THE WORLD BANK W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N T 2003 Annual Review of Development Effectiveness The Effectiveness of Bank Support for Policy Reform OED PUBLICATIONS Study Series 2002 Annual Review of Development Effectiveness--Achieving Development Outcomes: The Millennium Challenge Agricultural Extension: The Kenya Experience Agricultural Extension and Research: Achievements and Problems in National Systems Bangladesh: Progress Through Partnership Bridging Troubled Waters: Assessing the World Bank Water Resources Strategy Debt Relief for the Poorest: An OED Review of the HIPC Initiative Developing Towns and Cities: Lessons from Brazil and the Philippines The Drive to Partnership: Aid Coordination and the World Bank Financial Sector Reform: A Review of World Bank Assistance Financing the Global Benefits of Forests: The Bank's GEF Portfolio and the 1991 Forest Strategy and Its Implementation Fiscal Management in Adjustment Lending IDA's Partnership for Poverty Reduction India: The Challenges of Development India: The Dairy Revolution Information Infrastructure: The World Bank Group's Experience Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Lesotho: Development in a Challenging Environment Mainstreaming Gender in World Bank Lending: An Update The Next Ascent: An Evaluation of the Aga Khan Rural Support Program, Pakistan Nongovernmental Organizations in World Bank­Supported Projects: A Review Paddy Irrigation and Water Management in Southeast Asia Poland Country Assistance Review: Partnership in a Transition Economy Poverty Reduction in the 1990s: An Evaluation of Strategy and Performance Promoting Environmental Sustainability in Development Reforming Agriculture: The World Bank Goes to Market Social Funds: Assessing Effectiveness Uganda: Policy, Participation, People The World Bank's Experience with Post-Conflict Reconstruction The World Bank's Forest Strategy: Striking the Right Balance Zambia Country Assistance Review: Turning an Economy Around Evaluation Country Case Series Bosnia and Herzegovina: Post-Conflict Reconstruction Brazil: Forests in the Balance: Challenges of Conservation with Development Cameroon: Forest Sector Development in a Difficult Political Economy China: From Afforestation to Poverty Alleviation and Natural Forest Management Costa Rica: Forest Strategy and the Evolution of Land Use El Salvador: Post-Conflict Reconstruction India: Alleviating Poverty through Forest Development Indonesia: The Challenges of World Bank Involvement in Forests Uganda: Post-Conflict Reconstruction Proceedings Global Public Policies and Programs: Implications for Financing and Evaluation Lessons of Fiscal Adjustment Lesson from Urban Transport Evaluating the Gender Impact of World Bank Assistance Evaluation and Development: The Institutional Dimension (Transaction Publishers) Evaluation and Poverty Reduction Monitoring & Evaluation Capacity Development in Africa Public Sector Performance--The Critical Role of Evaluation Multilingual Editions Allègement de la dette pour les plus pauvres : Examen OED de l'initiative PPTE Appréciation de l'efficacité du développement : L'évaluation à la Banque mondiale et à la Société financière internationale Determinar la eficacia de las actividades de desarrollo : La evaluación en el Banco Mundial y la Corporación Financiera Internacional Côte d'Ivoire : Revue de l'aide de la Banque mondiale au pays Filipinas: Crisis y oportunidades Reconstruir a Economia de Moçambique : http://www.worldbank.org/oed W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N T 2003 Annual Review of Development Effectiveness The Effectiveness of Bank Support for Policy Reform 2004 The World Bank http://www.worldbank.org/oed Washington, D.C. © 2004 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone 202-473-1000 Internet www.worldbank.org E-mail feedback@worldbank.org All rights reserved Manufactured in the United States of America First edition July 2004 The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in the work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Cover photo by Ray Witlin, World Bank Photo Library, Burkina Faso. ISBN 0-8213-5938-X e-ISBN 0-8213-5939-8 Library of Congress Cataloging-in-Publication data has been applied for. World Bank InfoShop Operations Evaluation Department E-mail: pic@worldbank.org Partnerships & Knowledge Programs (OEDPK) Telephone: 202-458-5454 E-mail: eline@worldbank.org Facsimilie: 202-522-1500 Telephone: 202-458-4497 Facsimilie: 202-522-3125 Printed on Recycled Paper Contents v Acknowledgments vii Foreword ix Executive Summary xiii Abbreviations and Acronyms 1 1 Development Policy Reform 2 Policy Reform Is Difficult Business 3 Are Developing Countries' Policies Improving? 8 Why and How Does the Bank Support Policy Reform? 13 2 Ownership: Linking Bank Support to Policy Reform 13 Is Bank Support Linked to Policy Reform? 16 How Does the Bank Handle the "Tough Cases"? 22 Are Partnership Approaches Working? 25 3 Country Fit: Adapting Bank Support to Country Conditions 25 Do Bank Country Strategies "Fit"? 33 Is the Bank Effective in Advancing Corporate Advocacy Priorities and Harnessing Global Knowledge? 39 4 Conclusions 39 The Bank Has Been Reasonably Good at Aligning Its Support with Countries That Are Improving Their Policies 39 The Bank's Record In Customizing Its Support for Reform to Client Needs Is Mixed 40 Implications for the Future 43 Appendix: Project Performance Results 61 Annexes 61 A: Indicators of Economic Policies 67 B: Technical Notes 77 C: World Bank Operational Instruments 81 D: Outcomes and Bank Contributions to Country Cases Listed in Figure 2.2 91 E: Effectiveness of Bank Support for Policy Reform: An Overview from the CODE Chairperson i i i 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S 95 Endnotes 99 Bibliography Boxes 4 1.1 Indicators of Country Policies 7 1.2 Yemen Improved Its Macroeconomic Policies after Emerging from Conflict 8 1.3 Brazil's Policies Were Highly Rated, but Its Growth Was Disappointing 10 1.4 Customizing Policies to Country Contexts 14 2.1 The Bank Definition of a "Relatively Good" Policy Environment 16 2.2 Bank Support Contributed to Improved Educational Outcomes in Ghana 20 2.3 Conflicting Incentives Sometimes Make Conditionality Difficult to Enforce 21 2.4 Outcomes-Based Conditionality Shows Promise in a Pilot in Burkina Faso 23 2.5 CDF Principles Foster Country-Owned Strategies 26 3.1 The Uncertainties of Policy Reform Depend on the Policy Area 26 3.2 Adapting Support to Policy Reform Styles in Tunisia 28 3.3 Bank Support for Policy Reform in China Relies Primarily on Dialogue and Demonstration 29 3.4 Fostering Ownership of Education Reforms 30 3.5 Specific Country Knowledge Is Necessary for Project Success 33 3.6 Inadequate Debt Sustainability Analysis Can Contribute to Excessive Accumulation of Debt 35 3.7 New Paradigms: Private Sector Participation in Water and Sanitation 36 3.8 Electricity Development in Ukraine: Good Idea, Bad Fit Figures 5 1.1 Developing Countries Have Made Gradual But Definite Progress in Improving Their Policies 6 1.2 Policies Improved in All Regions 6 1.3 Countries with Relatively Large and Consistent Recent Changes in Policy Indicators (1999­03) 7 1.4 Most Areas of Developing-Country Policies Improved 8 1.5 Countries with Better/Improving Policies Grew Faster 11 1.6 World Bank Instruments Available to Support Policy Reform 14 2.1 Bank Lending Is Concentrated in Countries with Good Policy Environments 15 2.2 Good Outcomes of Bank Assistance Are Associated with Progress in Policy Reform 17 2.3 Improved Project Outcomes Are Associated with Improvements in Related Policies 31 3.1 High-Quality, Timely ESW Contributes to the Bank's Impact on Policy Reform i v Acknowledgments T his Review was prepared by a team led by from the advice of Ravi Kanbur. In addition, Robert J. Anderson. The core team com- helpful comments were received from the fol- prised Mirafe Marcos and Irina Tratch. lowing Bank staff: Yaw Ansu, Shawki Barghouti, Team members and background paper authors Daniela Gressani, Roumeen Islam, Aloysius Ordu, were Soniya Carvalho, Howard Nial White, Deepa and Padmanhaba Hari Prasad. Chakrapani, Hans Genberg (consultant), James The team was assisted in production of the re- W. Fox (consultant), and Miguel Kiguel (con- port by Julia Akumu Ooro. sultant). Contributions from numerous OED The Review was prepared under the direction staff are also gratefully acknowledged. of Victoria Elliott, Manager of OEDCM. Peer review was provided by David Dollar and Sanjay Pradhan. The Review also benefited Director-General, Operations Evaluation: Gregory K. Ingram Director, Operations Evaluation Department: Ajay Chhibber Manager, Corporate Evaluation and Methods: Victoria Elliott Task Manager: Robert J. Anderson v Foreword O ver the years, the Bank's strategy has broadened its focus beyond phys- ical investments to policies and institutions. Policy reform and insti- tutional development are seen not as ends in themselves, but rather as necessary to creating an environment that is--based on the Bank's and oth- ers' analyses of development experience--conducive to pro-poor growth and widespread, sustainable poverty reduction. Recognizing the inherently po- litical character of policy reform and the diversity of country contexts, the Bank espouses an approach to policy reform that is grounded in country leadership and ownership of the development agenda, with Bank support customized to country circumstances. The 2003 Annual Review of Development Ef- number of policy indicators. Policies improved fectiveness examines the effectiveness of Bank in most policy areas. The most widespread in- support to borrower countries to help them put creases occurred in policies related to gender, in place policies conducive to sustainable poverty financial stability, efficiency of resource mobi- reduction. The Review focuses primarily on the lization, and quality of budget and financial years from 1999 through 2003, which may have management. been an unusually favorable period for reform · Policy reform paid off. Countries whose policies due to the large number of countries coming out improved grew in per capita terms on average of crisis or advancing with transition. This period at more than twice the rate of those that did not. is chosen to facilitate the juxtaposition of re- · Bank lending, both overall and on a per capita cent evaluation evidence concerning the Bank's basis, was concentrated in countries that had support for policy reform with trends in a num- "relatively good" policy environments. Bank ber of policy indicators. support was also linked to improvements in policy at both the country and project levels, Findings and to higher levels of indicators related to Mil- · Two thirds of developing countries--both low- lennium Development Goals. and middle-income--have improved their de- · The Bank was less successful in linking its sup- velopment policies as reflected by trends in a port to policy reform in countries with no or v i i 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S weak track records, or in countries where the ceilings on adjustment lending or efforts to environment for reform was fluid or highly ramp up IBRD infrastructure lending that could uncertain. The instruments the Bank uses to result in increased lending levels. link lending to policy performance--Country · The knowledge content of Bank products Assistance Strategy triggers and conditionality, needs to be adapted to country circumstances. which pre-specify Bank responses to borrower The Bank is perceived to have been too nar- performance--have not resulted in good out- rowly focused in the analyses and "best prac- comes in highly uncertain situations. In such tices" that it presents, with insufficient attention situations, large-scale lending should normally to alternative perspectives or to individual be undertaken only when there are clear sig- country circumstances. Networks and regions nals that policy reform is under way. should strengthen their respective procedures · Outcomes of Bank country assistance have in the capture, validation, and application of les- been moderately satisfactory or better about 70 sons learned and good practices. The search percent of the time. Good results have been for "generic best practices" should be altered obtained with very different policies and in- in favor of intensified efforts to customize and stitutions, supported by different combina- adapt knowledge to specific localized prob- tions of Bank instruments and thematic/sectoral lems--often done best in collaboration with in- mixes, with the mix tailored to the country sit- country expertise. uation and country preferences, and usually evolving over time. However, a number of fac- Implications tors appear to play an important role in de- To support countries' efforts to deepen reforms, termining outcomes of Bank assistance. the Bank may need to adapt its processes and in- · Economic and sector work is critical for good struments--particularly those for managing the outcomes. It informs the Country Assistance risks associated with policy reform. One possi- Strategy, contributes to the policy dialogue, bility for doing so would be to experiment with and shapes the design of Bank operations. approaches that would complement intermedi- The Bank should, normally, not engage in lend- ate indicators and conditions with indicators of ing before economic and sector work has es- direct poverty reduction results or other out- tablished an adequate base of country and come-related indicators in Bank programming sector knowledge. decisions. To date, there has been only limited · The amount and terms of financing of devel- experience with this type of approach. That ex- opment assistance need to be calibrated care- perience has been promising, but has also re- fully to countries' debt carrying capacity. The vealed a number of practical problems, including Bank's Debt Sustainability Analysis needs to be lack of timely availability of data. The Bank should upgraded to require more critical examination promote further piloting and experimentation of the realism of assumptions and examination with outcomes-based lending and other ap- of the sensitivity of the analysis to key as- proaches to strengthening country leadership, sumptions. This should be done prior to any ownership, and results orientation in willing change in lending policies, such as removing countries. Gregory K. Ingram Director-General, Operations Evaluation v i i i Executive Summary O ver the years, as more has been learned about the conditions for suc- cessful development assistance, the Bank's strategy has broadened its focus beyond physical investments to policies and institutions. Pol- icy reform and institutional development are seen not as ends in themselves, but rather as necessary to creating an environment conducive to pro-poor growth and to widespread, sustainable poverty reduction. The 2003 Annual Review of Development Effectiveness (ARDE 2003) examines the effectiveness of Bank support to borrower countries to help them put in place such poli- cies. The Review focuses primarily, although not exclusively, on the years 1999 through 2003. This is done to facilitate the juxtaposition of recent eval- uation evidence with trends in policy indicators. Policy reform entails a high degree of uncer- dimensions of policy and institutional per- tainty since it depends not only on the policy sci- formance compiled by four organizations agree ences, but also on politics. Uncertainty, that developing countries' policies have, on av- unpredictability, and surprise have been a cen- erage, improved modestly. Improvement is seen tral part of the challenge of development since in all regions and among both low- and middle- the early days of aid, when development strate- income countries. Moreover, recent reforms gies focused on seemingly simple project in- have touched most areas of development pol- vestments. What is different about policy reform icy. In most countries that accomplished strong is that the uncertainties can be even greater, reform over the period 1999-2003, change was and prior analysis and contingency planning driven by necessity or opportunities such as generally have limited power to reduce them. transition from socialism, economic crisis, Eu- ropean Union accession, and change of gov- Recent Trends in Developing Countries' ernment. Development research suggests that Policies such factors tend to galvanize support for Developing countries have improved their poli- change and thus make the politics of policy re- cies in the last few years. Indicators of various form easier. i x 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Such circumstances occurred often during In countries with poor policy environments, the 1990s. Twenty-one countries emerged from therefore, the Bank should limit or postpone crises, and 26 countries in Europe and Central lending before there are clear signals that reform Asia were in various stages of transition to a mar- is under way. Assistance programs for these coun- ket economy. Moreover, as aggregate aid flows tries should initially emphasize analytical work, di- shrank and donors emphasized development alogue, and technical assistance, and they should performance over foreign policy in allocating modulate lending carefully. Premature lending aid, recipient countries faced strengthened in- may not only be wasted, but counterproductive. centives to reform. These trends may account for It weakens the linkage between sound policies and the fact that--despite the difficulties of policy re- access to development finance. It adds to the form--fairly widespread progress seems to have country's debt burden without a strong likeli- been made. hood of development payoff. This would suggest Policy reform paid off in terms of higher a gradual approach to lending re-engagement in growth. Countries whose policies improved over countries in which the Bank starts a new rela- the period 1999-2003 tended to grow on average tionship or re-activates a program after a long at more than twice the rate of those whose poli- hiatus. In non-crisis/non-emergency situations, cies did not improve. adjustment lending should be provided only when a solid track record of reform implemen- Linkage of Bank Support to Policy Reform tation has been established. The Bank was successful in linking its support The uncertainties associated with policy reform with good and/or improving policy environments are reduced when the Bank's support is aligned during the period covered by this Review. with country priorities that have been set and val- IBRD/IDA lending over the period 1999-2003, idated through country-led, inclusive political both overall and on a per capita basis, was con- processes. A start has been made in this direction centrated in countries that had "relatively good" in some cases in the development of country- policy environments. owned programs through the process of prepar- Evaluation evidence links Bank support to re- ing Poverty Reduction Strategy Papers (PRSPs). cent improvements in policy. Countries with good The PRSP's potential as a vehicle for fostering or improving policies as reflected in policy indi- country leadership and ownership is under- cators were also countries where Bank assis- mined, according to some borrowing countries, tance--as evaluated by OED--has contributed by its role as condition for access to conces- to good outcomes. Countries with satisfactory sional assistance from the Bank and Fund. Bank assistance outcomes and improving policies also scored higher on indicators related to the Mil- Adapting Bank Support to Country lennium Development Goals than did countries Conditions with unsatisfactory outcomes and deteriorating How effective is the Bank in customizing its sup- policies. At the project level, too, good country pol- port to country conditions? The outcomes of icy ratings tend to be associated with good eval- Bank country assistance as rated by OED in Coun- uation ratings on project outcomes. The Bank try Assistance Evaluations (CAEs) are moderately was less successful in linking its support to pol- satisfactory or better about 70 percent of the icy reform in countries with no or weak track time. There is thus room for improvement. An im- records or with deteriorating policy environments. portant aspect of success is tailoring the pace and Triggers and conditionality have helped keep instruments of reform to country policymaking lending support aligned with reform perform- processes. Even when country ownership of re- ance in cases where the environment for reform form is strong, a mismatch between Bank strat- was stable--either favorable or unfavorable. egy and country conditions creates tensions and But when the environment for reform is in flux, can result in poor outcomes of Bank assistance. and especially when it is deteriorating, triggers Economic and sector work (ESW) is critical for and conditions have been less effective. good outcomes. It informs the Country Assis- x E X E C U T I V E S U M M A R Y tance Strategies (CASs), contributes to the pol- consideration and recalibration of the Bank's icy dialogue, and shapes the design of Bank op- assistance as necessary to ensure progress toward erations. In countries where recent outcomes of or maintenance of debt sustainability. This should Bank assistance were evaluated as satisfactory, be done prior to any change in lending policies high quality, relevant, and timely ESW generally that could result in increased lending levels. made a substantial contribution. Where outcomes were not satisfactory, deficiencies of ESW in one Implications for the Future or more of these dimensions was a contributing The recent environment for policy reform may factor. The Bank should not normally engage in have been unusually favorable due to the large lending before ESW has established an adequate number of countries coming out of crisis or ad- base of country and sector knowledge. vancing with transition. To support the deeper There is considerable unexploited potential and wider reforms needed to maintain recent for applying ESW through better adaptation to trends will require the Bank to further adapt its specific country contexts and dissemination. processes and instruments--particularly those The Bank should ensure that adequate resources for managing the uncertainties associated with and managerial attention are devoted to realiz- policy reform. ing this potential. The Bank has also not been ef- Evaluations of early experience with Com- fective in giving operational value to its various prehensive Development Framework (CDF) knowledge initiatives in country contexts and sec- processes and policy research on development tor strategies. There is inadequate oversight and effectiveness identify three obstacles to doing so. quality assurance of the process by which pos- First, as noted above, some borrower countries sibilities suggested by new approaches become consider that the PRSP's multiple roles are in- "best practices." These inadequacies limit the compatible. Second, there is a potential tension effectiveness of the Bank's networks and their in- between triggers and conditionality--particu- teractions with the regions, and represent a risk larly when these are not aligned with country- to the Bank's reputation. This risk is potentially owned objectives--and the development of most serious in those areas--such as competi- country ownership. And third, the use of policy tive supply of infrastructure services and gover- indicators that appraise countries' policies against nance--where thinking on policy is evolving uniform criteria may discourage the customiza- very rapidly. Networks and regions should tion of policies to country circumstances. Good strengthen their respective procedures in the outcomes can be fostered by a range of policy capture, validation, and application of lessons and institutional options, which can differ learned and good practices. The development across countries based on their institutional of generic "best practice" should be augmented, history, norms, and cultural attributes. and where necessary replaced, by development Many of these tensions are addressed by cur- of knowledge adapted to local problems gen- rent Bank processes. The Bank is, for example, erated in collaboration with local expertise. endeavoring to align CASs and operational con- A properly customized country assistance ditionality with country-owned programs, and program requires a medium-term macroeco- supporting those programs through program- nomic framework that includes a sustainable matic lending instruments. It also aims to debt burden. The Bank and its donor partners strengthen the country focus by using differ- have not always calibrated the size and financing entiated approaches for countries at different of their assistance to recipients' debt carrying ca- stages of development (e.g., initiatives for low pacity. The Bank's Debt Sustainability Analysis income countries under stress, or LICUS, and (DSA) needs to be upgraded to require more crit- middle-income countries), decentralization of ical examination of the realism of assumptions operations, and doing joint analytical work. It and of the sensitivity of the analysis to key as- applies the CPIA flexibly in programming deci- sumptions. In addition, the Bank's monitoring sions. There is as yet not enough experience, indicators and CAS triggers should call for re- however, to determine whether these efforts x i 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S are paying off in terms of improved development The Bank could begin to experiment with ad- effectiveness. ditional mechanisms to introduce a more direct The Bank also has a number of mechanisms linkage between its support and results in terms designed to link its support to the achievement of poverty reduction or other Millennium De- of results. These include new lending instru- velopment Goals. Such a link would be added to ments such as Adaptable Program Loans and or blended with existing mechanisms, in order Poverty Reduction Support Credits, as well as to help sharpen the results focus, provide coun- ongoing efforts to ground triggers and condi- tries latitude during implementation in pursuing tionality in country-owned programs. In these their objectives, and help take account of the un- mechanisms, support is usually linked to com- certainties associated with policy reform. There pletion of intermediate steps such as legal and are practical problems to be addressed to link aid regulatory changes, organizational improvements, to results operationally, notably the difficulty of and expenditure or other input targets. This link- obtaining timely data. To find solutions to these age to intermediate steps--while relatively prac- problems, the Bank should promote pilot proj- tical to monitor--has one main weakness: It does ects in willing countries with the potential to not address the "qualitative" dimensions--"due increase aid effectiveness. The challenge facing diligence" or "best effort" in implementation in the Bank remains to continually re-examine and pursuit of the objectives of the development pro- modify its approaches to delivering develop- gram. While strong county ownership would mit- ment support so as to yield better results in igate this problem, it would not eliminate it. terms of sustainable poverty reduction. x i i ABBREVIATIONS AND ACRONYMS AAA Analytic and Advisory Services AFR Africa Region APL Adaptable Program Loan ARDE Annual Review of Development Effectiveness CAE Country Assistance Evaluation CAS Country Assistance Strategy CBO Community-based Organization CDF Comprehensive Development Framework CEM Country Economic Memorandum CIS Commonwealth of Independent States CODE Committee on Development Effectiveness CPIA Country Policy and Institutional Assessment DSA Debt Sustainability Analysis EAP East Asia and Pacific Region ECA Europe and Central Asia Region ESW Economic and Sector Work FY Fiscal Year GDP Gross Domestic Product HIPC Heavily Indebted Poor Countries HNP Health, Nutrition, and Population IBRD International Bank for Reconstruction and Development (World Bank) ICR Implementation Completion Report IDA International Development Association IEO Independent Evaluation Office (of the International Monetary Fund) IMF International Monetary Fund LCR Latin America and the Caribbean Region LICUS Low-Income Countries Under Stress LIL Learning and Innovation Loan MDGs Millennium Development Goals MNA Middle East and North Africa Region NGO Nongovernmental Organization ODA Official Development Assistance OED Operations Evaluation Department PPAR Project Performance Assessment Report PRSC Poverty Reduction Support Credit (also PRC) PRSP Poverty Reduction Strategy Paper PSAL Programmatic Structural Adjustment Loan (also PSL) QAG Quality Assurance Group SAR South Asia Region SAL Structural Adjustment Loan SSAL Special Structural Adjustment Loan (also SSL) SSP Sector Strategy Paper SWAP Sectorwide Approach UNDP United Nations Development Program WDR World Development Report x i i i 1 Development Policy Reform Today's World Bank Group is very different from the institution that first opened its doors at the end of World War II. In those early days, the Bank ...focused very much on the reconstruction of war-torn Eu- rope. As the immediate postwar reconstruction needs subsided, the Bank's focus shifted to financing infrastructure projects in Europe, Latin America, and Asia, in line with then-current thinking about de- velopment gaps and constraints. Over the years, as more was learned about the necessary conditions for successful development and poverty reduction, the paradigm for development assistance changed--to focus less on bricks and mortar, and more on policies and institutions, country ownership of the development process, the role of the private sector, partnerships, knowledge, and learning. The Bank's approach and instruments evolved along with this paradigm. --World Bank (2000) The 2003 Annual Review of Development policy reform, the Bank focuses on policies Effectiveness (ARDE 2003) examines the that: (a) improve the climate for investment and effectiveness of Bank support in helping growth (e.g., fiscal and monetary policies borrower countries put in place policies that are conducive to macroeconomic stability, trade conducive to sustainable poverty reduction.1 policies conducive to integration into the global The Bank views policy reform not as an end in economy, policies conducive to the develop- itself, but as necessary for creating an environ- ment of a stable and efficient financial system ment that is conducive to pro-poor growth and and supporting infrastructure services); (b) widespread sustainable poverty reduction. Its promote social inclusion in growth and concept of appropriate policy reform is based development (e.g., access of the poor to quality on its distillation of a substantial body of basic services in health and education, gender research and evaluation evidence. In promoting equity); (c) promote environmental sustainabil- 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S ity (e.g., environmental standards and regula- cascade of political, social, and economic tions); and (d) improve governance (e.g., consequences. In all but the most monolithic of policies with respect to civil service, informa- political environments, it is frequently impossi- tion disclosure). ble to foresee how the process will unfold or ARDE 2003 focuses primarily, although not where and when it will end. exclusively, on the years from 1999 through The process does not end when a policy is 2003. This allows examination of both recent adopted. Policy implementation determines evaluation evidence concerning the Bank's whether incentives really change and whether support for policy reform and independently costs and benefits actually flow. The various produced policy indicators available for interests must thus continue to struggle to approximately the same time period. ARDE control the administrative regulations and 2003 does not distinguish between "policy" and processes that govern implementation, which "institutional" reform. Although these are in ultimately determine how policies will impinge fact distinct, they are not distinguishable in the on those interests. The timing and nature of data over the short period considered here. the outcomes--there may be several, and they Institutions change more slowly than policies, may change over time--are also unpredictable. so the data examined here--while they contain Indeed, this second act--implementation--has institutional components--primarily reflect no natural end as long as a policy is in force. policy developments. The report addresses the Another dimension of uncertainty that following questions: complicates policy reform is disagreement among experts about the most appropriate · What distinguishes support for policy reform policy package for any given situation. Although from support for investments? there is substantial professional consensus · Have developing countries been making about many broad policy areas, the myriad progress in reforming their policies? options available for implementation (e.g., · Has the Bank been effective in targeting its timing, sequence of related policies) mean that support to countries that are improving their few universally accepted "engineering policies? blueprints" exist that can be relied upon to focus · Has the Bank been effective in tailoring its and limit the scope of debate and discussion. support to country conditions? Uncertainty, unpredictability, and surprise have been a central part of the challenge of This chapter briefly considers the challenge development since the early days of develop- posed by supporting policy reforms (in ment assistance, when development strategies contrast to physical investments) and recent were focused on seemingly simple project trends in developing countries' economic and investments.2 In the case of policy reform, the social policies as reflected by a number of uncertainties can be even greater, and prior policy indicators. In general, the indicator data analysis and contingency planning have limited show recent improvements in most developing power to reduce many of them. countries' policies, although this trend is by no The degree of "policy science" and political means universal. The chapter also briefly uncertainty associated with policy reform reviews the Bank's rationale for its emphasis on depends upon the particular kind of policy at policy reform. issue. The challenge to effective support for policy reform, given these uncertainties, is Policy Reform Is Difficult Business achieving the proper balance between (a) prior Policy reform generally entails a high degree of analysis of the likelihood of various possible uncertainty since it, inevitably and appropriately, outcomes and planning for contingencies (e.g., depends not only on the policy sciences, but the policy outcomes of a country's political also crucially on politics. Policy reform attempts processes, the social or environmental effects can and frequently do set off an unpredictable of alternative proposed changes in policies) 2 D E V E L O P M E N T P O L I C Y R E F O R M and (b) monitoring of outcomes and adjust- probably would have received low scores on ment of policies and their implementation as some dimensions of these indicators. To ensure events unfold. rough comparability over time, the focus here Prior analysis (e.g., use of a computable is on the period since 1999, when the CPIA general equilibrium model to analyze the alloca- underwent its most recent major revision.4 tive and distributive outcomes of a tax reform)3 can be costly, and its predictions are subject to Developing countries' policies are gradually error. This raises the question of whether the improving. benefits of such analysis--in terms of its utility Even though a country enters difficult terrain in improving the outcomes of policies-- when it undertakes to improve its policies, in outweigh the costs of conducting the analysis. general, the last few years have seen progress In general, the greater the uncertainty, and the on reform. The various indicators of policy and more limited the power of prior analysis and institutional performance examined here contingency planning to reduce it, the greater indicate that developing countries' policies the emphasis that needs to be given to monitor- have, on average, improved modestly. Improve- ing and midcourse adjustment of policies. ment is seen in both low- and middle-income countries (figure 1.1). Although there are some Are Developing Countries' Policies distinctive regional patterns, average CPIA Improving? ratings for every one of the Bank's regions This section uses four widely used policy improved--a pattern reflected in the other indicators to briefly review recent trends in three indicators as well (figure 1.2). developing countries' policies. These four In all four indicators, the average improve- indicators are the Bank's internal Country ment is quite small in relation to plausible Policy and Institutional Assessment (CPIA), the margins of variation in the judgments made in Heritage Foundation's Index of Economic applying rating criteria. Can we be confident Freedom (Heritage), the Economist Intelli- that these modest increases reflect real gence Unit's Index of Country Risk (EIU), and improvements, or are they mere statistical the Political Risk Services Group's International artifacts? Country Risk Guide (ICRG). These indicators, Three observations tend to support the used to appraise--inter alia--countries' conclusion that the observed increases reflect economic prospects, are described in box 1.1. real improvements. First, all four indicators point Although there are overlaps in coverage, in the same direction even though they are based each indicator also captures some unique on different criteria.5 Second, the modest scale elements of countries' policy environments, of the improvements is plausible because the and the criteria and weights used to arrive at indicators are based on variables that tend to countries' scores differ. All--no matter how change slowly. Third, significantly more comprehensive--give only partial and stylized countries' ratings increased than decreased.6 assessments of policy. In-depth case studies suggest that the specific features of the policies The strongest and most consistent indications of and institutions that have driven growth over reform are associated with transition countries. sustained periods--while embodying certain Although improvements in general have been underlying common features--are highly widespread and small, some countries stand country specific and are not easily captured in out (figure 1.3).7 The countries with improving any simple set of cross-country indicators policies are mostly transition countries in (Rodrik 2003b). For example, China's early Europe and Central Asia. These countries strategic policy of permitting market incentives started the reform effort from a considerably to operate for a relatively small number of lower base than the average, and many indeed transactions and maintaining most transactions were in crisis in the early to mid 1990s. Five through the "command and control" system countries aspire to European Union accession 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S B o x 1 . 1 I n d i c a t o r s o f C o u n t r y P o l i c i e s World Bank Country Policy and Institutional Assessment International Country Risk Guide (ICRG). The ICRG risk (CPIA).TheCountryPolicyandInstitutionalAssessmentassesses rating system is a proprietary service that rates the risks faced borrower countries' policy and institutional frameworks annually by investors in a country. It is used primarily by businesses as with regard to their capacity to foster poverty reduction, sustain- a predictive tool to analyze investment options. The system able growth, and the effective use of development assistance. The assigns a numerical value (risk points) to a set of risk CPIA evaluates 20 dimensions (each with the same weight in the components, according to a preset weighted scale, for each overall rating) of policy encompassing economic management, country covered by the system. ICRG rates countries monthly private and financial sector development, social inclusion and on risk of political instability, economic vulnerability, and equity, and public sector management and institutions. ability to meet financial obligations. Heritage Foundation Index of Economic Freedom Economist Intelligence Unit Country Risk Service (Heritage).The Heritage Foundation has produced the Index of (EIU). The EIU Country Risk Service is a proprietary service Economic Freedom annually since 1995. The Heritage Founda- designed to rate risks for commercial bankers, institutional tion/Wall Street Journal 2003 Index measures the performance investors, and corporate executives who invest in emerging of 161 developed and developing countries with respect to 50 markets. This service monitors and rates risks in 100 countries, policy dimensions, divided into 10 broad factors of economic including political, economic policy, economic structure, and freedom: trade policy, fiscal burden of government, govern- liquidity risks as well as currency, sovereign debt, and ment intervention in the economy, monetary policy, capital banking sector risks. Countries are rated monthly. flows and foreign investment, banking and finance, wages and prices, property rights, regulation, and black market. Source: Annex A. and are striving to close the gap between their these kinds of factors are frequently associated present policies and those required for with episodes of policy reform (Devarajan, membership. Three countries, in addition to Dollar, and Holmgren, 2001), as in the case of the reforms they have undertaken, have Yemen (box 1.2). Such factors tend to galvanize received a substantial macroeconomic boost support for change and thus make the politics from the increase in oil prices since 1999. of policy reform easier. Countries whose ratings have deteriorated The environment for policy reform in the are more diverse, although several have been 1990s may have been unique in some respects. A characterized by vacillation or backtracking on number of countries had been or were afflicted policy reform over extended periods of time. by crises (conflict, natural disaster, and/or Argentina's ratings are clearly related to the financial)8 and 26 countries in Europe and economic and political crisis that has gripped Central Asia were in various stages of transition the country since 2001. Five of the ten to a market economy.9 With greater competition countries (including Argentina) are in Latin for diminishing aid flows and the disappearance America or the Caribbean, a region that has of cold-war-motivated flows, these factors may in been hit by several crises and has grown very part account for the fact that--despite the slowly over the last several years (1999-2003). difficulties of policy reform--fairly widespread progress seems to have been made. Strong reform occurs in conducive environments. Most of the cases of strong reform in figure 1.3 Progress is occurring on most dimensions of policy. occurred in environments that were conducive Recent reform has also been fairly widespread to change; they were driven by necessity (e.g., across the different areas of development transition and/or crisis) and/or opportunity policy, as reflected in the various components (e.g., accession to the European Union, new of the CPIA (figure 1.4). In 1999, the highest regime). Development research suggests that rated policy areas were generally those that had 4 D E V E L O P M E N T P O L I C Y R E F O R M D e v e l o p i n g C o u n t r i e s H a v e M a d e G r a d u a l F i g u r e 1 . 1 B u t D e f i n i t e P r o g r e s s i n I m p r o v i n g T h e i r P o l i c i e s Value Rescaled Indicator 100 1999 2003 90 80 70 60 50 40 30 20 10 0 IDA IBRD IDA IBRD IDA IBRD IDA IBRD CPIA Heritage ICRG EIU Notes: CPIA = World Bank Country Policy and Institutional Assessment, Overall Rating; Heritage = Heritage Foundation/Wall Street Journal Composite Index of Economic Freedom; ICRG = International Country Risk Guide Composite Index of Country Risk; EIU = Economist Intelligence Unit Country Risk Service Composite Country Risk. IDA refers to low-income countries that are eligible to receive assistance from the International Development Association (IDA). IBRD refers to middle-income countries that are eligible to borrow from the International Bank for Reconstruction and Development (IBRD). All indices are re-scaled to range between 0 (lowest value) and 100 (highest value). Different indices cover different sets of countries, with the CPIA, which covers 134 countries in both 1999 and 2003, being the most inclusive. Sixty-seven countries are common to all three indicators. Source: Annex A. been on the Bank's agenda for many years, sector depth and efficiency. Debt management such as macroeconomic management, goods is the only indicator for which an absolute and factor markets, trade and foreign decrease was registered.12 exchange, and management of public debt10-- areas long addressed by the Bank in its support Patterns of reform are time and country specific. for stabilization and structural adjustment. It is unusual for a country to improve in some Newer items on the Bank's agenda--such as areas of reform and regress in others. Country financial stability, transparency, environment, factors explain by far the largest portion of the public administration, and property rights and variation in changes in policy ratings. Policy governance--had lower average ratings. area factors, while significant in a statistical Rates of progress, however, differed among sense, explain a smaller fraction of the variation policy areas.11 Progress was most pronounced in changes in policy ratings.13 with regard to gender, financial stability, revenue Policy area patterns of reform differ among efficiency, and macroeconomic management. countries, differ among policies within any The smallest increases were registered in trade country, and also change over time within any and foreign exchange (which remains the most country. Between 1999 and 2003, for example, highly rated policy area), fiscal policy, property many European and Central Asian countries rights and rules-based governance, and financial had already made substantial progress on 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S F i g u r e 1 . 2 P o l i c i e s I m p r o v e d i n A l l R e g i o n s Scaled CPIA 100 1999 2003 90 80 70 60 50 40 30 20 10 0 AFR EAP ECA LCR MNA SAR Region C o u n t r i e s w i t h R e l a t i v e l y L a r g e a n d F i g u r e 1 . 3 C o n s i s t e n t R e c e n t C h a n g e s i n P o l i c y I n d i c a t o r s ( 1 9 9 9 ­ 0 3 ) Policy Improvement Policy Deterioration Azerbaijan Argentina Brazil Egypt Croatia Dominican Republic Estonia Lebanon Kazakhstan Malawi Lithuania Panama Nicaragua Trinidad & Tobago Romania Uruguay Russian Federation Zambia Slovak Republic Zimbabwe Ukraine Yemen, Republic of Note: Based on concordance of two or more indicators, as per endnote 7. macroeconomic and trade policy and were Policy improvement is associated with faster changing their pensions and social assistance growth. policies to make them more affordable and Countries with better policies and/or improving improve targeting of the poor (OED 2004a). In policies grew more rapidly than did countries the area of pension reform, for example, some with weak policies and/or deteriorating policies chose to introduce funded systems or pillars (figure 1.5).14 Cumulative per capita growth within existing systems, and some opted for between 1998 and 2002 (the most recent period changes to existing non-funded systems. for which growth data are available that overlap 6 D E V E L O P M E N T P O L I C Y R E F O R M Y e m e n I m p r o v e d I t s M a c r o e c o n o m i c B o x 1 . 2 P o l i c i e s a f t e r E m e r g i n g f r o m C o n f l i c t Having achieved political stability after years of civil war and devaluing the currency, and introducing a market exchange- exogenous shocks, the government of Yemen embarked on an rate regime. Inflation was brought down to 6.3 percent and real ambitious economic stabilization and structural reform non-oil and overall GDP growth reached 5.3 percent and 5.2 program in the mid-1990s. It succeeded in bringing the fiscal percent, respectively, in 1997. deficit down from 17 percent of GDP in 1994 to 1 percent in 1997, liberalizing interest rates while containing the money supply, Source: OED (2001g). M o s t A r e a s o f D e v e l o p i n g - C o u n t r y P o l i c i e s F i g u r e 1 . 4 I m p r o v e d CPIA Rating 4.0 1999 2003 3.8 3.6 3.4 3.2 3.0 2.8 eff mgt. devt mgt res mkts debt Gender admin depth Policy StabilityReven res & anal pub reform regime of & laborof sec & govern Fin. publ public Transparency Macro. monit Environment & finan prot Competition & factorof Fin rght Fiscal of HumanPov. Sust. & forex Bugt EquitySocial Qlty Goods Prop Mgt. Trade CPIA Components Ranked by Amount of Improvement with the period covered by the CPIA data) in during the period 1998-2002. Statistical countries whose policies were improving was relationships linking policy indicators to two to six times greater than in countries with growth typically explain only 40 to 60 percent deteriorating policies. This was attributable of the variation in countries' growth rates over both to the effect of policy on growth and to the the very limited time period covered by the greater effectiveness of aid in good and/or data examined here. Shocks and other factors improving policy environments. also play a role. Those other factors could in But good policies--at least as they are some instances be elements of policy that may reflected in broad policy indicators--do not not be fully captured in broad policy indica- guarantee that a country will do well (box 1.3). tors. For example, Brazil's low investment rate Brazil's policies, for example, were rated highly could have been a result of its relatively high on a number of indicators yet its growth was cost of doing business--as reflected in the only about average for Latin America, and well Bank's new Cost of Doing Business indicators below the average for developing countries, (World Bank 2004). 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S C o u n t r i e s w i t h B e t t e r / I m p r o v i n g P o l i c i e s F i g u r e 1 . 5 G r e w F a s t e r Cumulative GDP Growth Per Capita (1998­2002) 18 CPIA Decreasing CPIA Increasing 16 14 12 10 8 6 4 2 0 1999 CPIA < = 3.0 1999 CPIA > 3.0 B r a z i l ' s P o l i c i e s W e r e H i g h l y R a t e d , B o x 1 . 3 b u t I t s G r o w t h W a s D i s a p p o i n t i n g Brazil has had disappointing growth performance for over two 1990s failed to restore high growth rates not because they decades. Studies have indicated that the country's low rate of failed to generate productivity gains, but because they failed capital accumulation during this time may explain as much as to raise the levels of investment to a significant degree. The 70 percent of Brazil's decline in output after 1980. The growth 2001 energy crisis provides an example of how insufficient studies also show that the structural reforms implemented in investment in infrastructure has affected the country's growth the 1990s actually generated significant productivity gains -- performance--energy rationing of in 2001 reduced GDP growth total factor productivity (TFP) increased more in the 1990s than in that year alone by a full percentage point. in any other period and explains a large share of GDP growth in this period. The literature concludes that the reforms in the Source: OED (2004e). Why and How Does the Bank Support and attainment of the Millennium Develop- Policy Reform? ment Goals (MDGs). This emphasis is based on the Bank's distillation of a substantial and The Bank's strategy for poverty reduction rests growing body of policy research and evaluation on three premises. evidence. The fundamental premises of the The Bank's corporate strategy for poverty Bank's strategy are that: reduction emphasizes mutually reinforcing developed- and developing-country policy · Aid is more effective in promoting poverty re- reforms and developing-country-led partner- duction and growth in good developed- and ships as keys to scaling up development impact developing-country policy/institutional envi- 8 D E V E L O P M E N T P O L I C Y R E F O R M ronments (World Bank 1997; Burnside and Dol- is also a wide range of potential outcomes (e.g., lar 2000, 2004).15 not all smokers contract these diseases, and · Aid is neither necessary nor sufficient for good not all non-smokers are spared them). In the policy. Formulation and implementation of case of growth, the evidence suggests that policies are the result of complex, uncertain, countries that maintain prudent fiscal balances, and risky political and administrative appropriate exchange rates, and reasonably processes--i.e., the evolution of "ownership"-- open trade appear to have a higher probability in adopting countries. Aid can complement of good growth outcomes than do countries and catalyze these processes by fostering and that do not (Rodrik 2003b). But these relation- reinforcing the development of ownership, ships do not hold for all times and all places. but aid per se is no substitute for ownership The specifics of policy design and the sequence (Dollar and Svensson 1998; Killick 1998; Mosley in which countries implement policies in order et al.1995). to maintain fiscal balances, trade openness, and · Good policy is context specific. All successful appropriate exchange rates vary widely across reforms align incentives to reward growth, but countries. what works well in one country at a specific time to accomplish this alignment may be in- Country ownership is crucial. effective or counterproductive in another (Ro- Meaningful policy change (i.e., change that drik 2003a). Donors can help by ensuring that alters incentives and affects the distribution of aid recipients have access to the knowledge and income and power) does not happen unless a experience of others. strong enough consensus for change can be forged among the various interests. In many The statistical evidence relating policy and aid recent successful reform cases, like the effectiveness is not robust. countries with policy improvement shown in The proposition that aid is more effective in figure 1.3, there has been some event, predica- good recipient-country policy environments-- ment, or opportunity that unexpectedly shifted notwithstanding the additional corroborating the center of political gravity in the direction of evidence presented here--has been policy change. Whatever the cause, a consen- challenged by a number of researchers sus of sufficient cohesiveness to deliver and (Hansen and Tarp 2000, 2001; Easterly et al. sustain change through the country's political 2003). The challengers have shown that the and administrative processes is a necessary empirical evidence relating policy to growth is condition for meaningful change to occur. This not robust, and have argued that there are "sufficient consensus" for change is what is equally plausible, no less robust relationships meant here by "ownership." between aid, growth, and poverty absent policy factors. Other research finds positive, non- Good policy is context specific. policy-related linkages between aid and The highly differentiated experiences of the improvements in non-income dimensions of most successful developing countries suggest welfare (Lensink and White 2000). that many policy approaches--including a Given the numerous factors that potentially variety of instruments and sequences--are affect growth, the multiple ways in which each conducive to growth and poverty reduction. can be measured, and the possibility that There is some degree of agreement on underly- growth relationships may change over time, ing common elements that may account for empirical analysis cannot fully resolve these success--an incentive environment that issues.16 Just as in health, certain factors seem rewards growth and poverty reduction, to be associated with higher probabilities of medium-term macroeconomic balance, and good outcomes (e.g., people who don't smoke stability and predictability of property rights have a lower probability of contracting certain and the rule of law. There is much less diseases than people who do smoke), but there agreement on the specific design of policies 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S that will achieve these in any particular country policies but also possibilities for improvement of context at any point in time. The key to success these policies with Bank assistance. Indeed, is finding an approximately (not "the") right policy reform objectives figure prominently in country policy fit in view of the country's almost all Country Assistance Strategies (CASs)-- institutions and politics (box 1.4).17 irrespective of whether policy-based lending is planned--to promote better long-term develop- Policy reform plays a critical role in the Bank's ment outcomes. Virtually all planned Bank strategy. country operations--economic and sector work Country assistance programs are calibrated to the (ESW), project lending, donor coordination, Bank's assessment of the capability of the recipi- etc.--have policy reform objectives, and the Bank ent country's policies and institutions to use aid uses its full range of instruments (shown in figure effectively as embodied in its system for perform- 1.6) to support policy reform objectives. The CAS ance-based allocations. However, in practice, the structure indeed is frequently conditional, Bank's assistance strategies also typically through triggers that link Bank assistance to emphasize not only the status of a country's policy, on policy developments. B o x 1 . 4 C u s t o m i z i n g P o l i c i e s t o C o u n t r y C o n t e x t s The principle that property rights should be protected implies system as one can imagine. Similarly, the principle that very little about what is the best way to do this under a private incentives should be aligned with social costs and society's existing institutional preconditions. It certainly benefits hardly results in unconditional support for policies does not imply that a system of private property rights and of trade liberalization, deregulation, and privatization. Anglo-American corporate governance is the right approach Finally, debt sustainability, fiscal prudence, and sound money for all countries at all times. Look at the tremendous amount are also obviously compatible with diverse institutional of investment and entrepreneurial activity that China has arrangements. managed to elicit through a hybrid system of property rights and a legal regime that is as far from the Anglo-American Source: Excerpted from Rodrik (2003a), pg. 3. 1 0 D E V E L O P M E N T P O L I C Y R E F O R M W o r l d B a n k I n s t r u m e n t s A v a i l a b l e t o F i g u r e 1 . 6 S u p p o r t P o l i c y R e f o r m Financial Non-financial Services Services (AAA) Investment Adjustment Other Diagnostic Advisory/TA APL, ERL DDO, DRL, Grants CEM, DPR, CFAA, Sector/Thematic Studies FIL, LIL, SIL, PRSC, PSAL, Guarantees CPAR, PA, Special Studies SIM, TAL RIL, SAL, Trust Funds PER, IFA Non-Lending TA SECAL, Policy Notes SNAL, SSAL Specific Instruments Note: Acronyms for specific investment and adjustment lending instruments and non-lending instruments are defined in Annex C. 1 1 2 Ownership: Linking Bank Support to Policy Reform We can do it with or without the Bank, but with you we can do it faster. --Deng Xiaoping China Country Assistance Evaluation (OED 2004f) Chapter 1 argued that meaningful policy covenants of specific lending operations that reform happens only where a strong enough link disbursements to satisfaction of specific consensus for change--ownership--is present, conditions--often conditions related to or can be forged, and that the politics and policies. How effectively have these instru- outcomes of the consensus-forming processes ments worked in focusing Bank support on underlying reform are uncertain. Policy reform countries that are improving their policies? is therefore high risk. The payoff is high, too-- improved prospects for growth and sustainable Bank lending is focused on countries with poverty reduction. good and improving policy environments. This chapter examines how the Bank links Bank lending over the period 1999-2003 was its support to policy reform in this highly concentrated in countries that had "relatively uncertain environment. It shows that the Bank good" policy environments (box 2.1), with 89.4 has, in general, been effective in directing its percent of Bank lending going to countries lending support to countries that have been with a CPIA ranking of 3.0 or better in 1999 and improving their policies. It also shows that, in a 96.6 percent going to such countries in 2003 number of cases, evaluation evidence associ- (figure 2.1).1 Lending during the period thus ates these improvements in part with Bank tended to be associated with countries that assistance. were improving their development policy environments as captured by the CPIA and Is Bank Support Linked to Policy Reform? other indicators.2 To some extent, this could Since ownership is not directly observable, the have been a result of the general improvement Bank attempts to link its support to ownership in policy indicators that was taking place during of policy reform indirectly through several this time. mechanisms: the CPIA, which reflects cumula- This same pattern of increasing concentra- tive policy reform achievements; CAS triggers, tion of assistance on countries with relatively which link Bank support to, inter alia, progress good performance holds when comparisons on the policy agenda; and conditions or are done in terms of countries' policy ranks. 1 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S T h e B a n k D e f i n i t i o n o f a " R e l a t i v e l y B o x 2 . 1 G o o d " P o l i c y E n v i r o n m e n t The Bank defines "relatively good" policies for productive use major imbalances"; a country rated "4.0" is one "applies of development assistance for poverty reduction as an overall measures to address major imbalances, but its program is not score of at least 3.0 on the Bank's Country Policy and Institu- fully consistent," tional Assessment (CPIA) index. Of 136 countries rated in 2003, On environmental policy, a "3.0" indicates that gaps "exist in the 112 had CPIAs of 3.0 or greater. coverage of policies concerning pollution and natural resources. A "3.0" rating on the CPIA rating scale in a policy area Funding levels are low and not well matched to environmental indicates that there are deficiencies in one or more of the priorities. Capacity to implement and enforce policies is weak." A criteria being rated in that area, while a rating of "4.0" "4.0" indicates that, "Policies concerning pollution and natural indicates that a country is dealing quite effectively with most resources are fairly broad, reasonably matched to environmental issues in the policy area. prioritiesandfundedatminimallyacceptablelevels.Basiccapacity For example: to implement and enforce policies exists." A country rated "3.0" on macroeconomic management is a country that "sporadically or partially attempts to correct Source: World Bank (2003b), World Bank (2003d). B a n k L e n d i n g I s C o n c e n t r a t e d i n C o u n t r i e s F i g u r e 2 . 1 w i t h G o o d P o l i c y E n v i r o n m e n t s Percent* 100 CPIA >= 3.0 CPIA >= 3.5 80 60 40 20 0 1999 CPIA Rating 2003 CPIA Rating *% of FY99-03 loans to countries with CPIA of 3 or higher, 1999 and 2003. Notes: Countries with a CPIA ranking of 3.0 are defined as having "relatively good" policy environments (see box 2.1 and Annex A). Only the 134 countries rated in both the 1999 and 2003 rankings are included here. Using 1999 CPIA ranks, 12 percent of the Bank lending is correlated with the policy Bank's FY99-03 lending was to countries environment, much more so than official ranking in the top quintile, and 73 percent of development assistance (ODA). CPIA ranking is Bank lending was to countries above the correlated positively and significantly with median CPIA rating. Using 2003 CPIA ranks, lending per capita, although there is a great 23.5 percent of FY99-03 lending was to deal of variability in this relationship--as would countries ranked in the top quintile, and 79 be expected given the other factors that affect percent to countries ranked above the median. Bank lending decisions.3 Changes in lending 1 4 O W N E R S H I P : L I N K I N G B A N K S U P P O R T T O P O L I C Y R E F O R M per capita over the period are also correlated indicator data examined here. Of these, the positively (although not significantly) with outcomes of Bank assistance were judged to be changes in the CPIA. By way of comparison, "moderately satisfactory" or better in 13 cases, ODA per capita during this period is correlated and "unsatisfactory" in three. Of the 13 positively (but not significantly) with the countries, the CPIA rating increased in 11; CPIA average CPIA.4 ratings declined in all three of the countries whose outcomes were rated "unsatisfactory" Good Bank country assistance outcomes are (figure 2.2). associated with policy improvement. The eleven countries with satisfactory Bank Countries with good policies--as reflected in assistance outcomes and with improving CPIA ratings--tended to be countries in which policies had better development outcomes Country Assistance Evaluations (CAEs) rated than did the three countries with both unsatis- the outcome of Bank assistance as satisfactory. factory outcomes and deteriorating policies CAEs issued since 1999, when the practice of (figure 2.2). GDP growth over the period 1998- rating country assistance outcomes was 2002 averaged 5 percent annually in the former initiated, have rated outcomes of a total of 544 group, versus 0 percent in the latter; the Bank "country assistance years"5 in 53 poverty headcount in the former averaged 30 countries, of which approximately 70 percent percent, versus 75 percent in the latter. Life were rated as "moderately satisfactory" or expectancy in the former increased by about better. Countries' CPIA ratings in the terminal one year between 1998 and 2002, while life year of CAE evaluations are strongly correlated expectancy in the latter group declined by with CAE outcome ratings.6 about one year. Annex D presents these data in Countries in which policies improved also more detail and summarizes the role played by tended to be countries where Bank assistance Bank assistance--as analyzed in recent CAEs-- contributed to good development outcomes. in these countries. Sixteen (16) of the 53 most recent evaluations OED's recent impact evaluation of the Bank's rate Bank assistance outcomes for 2001 or later, support to education in Ghana illustrates at a the midpoint of the period covered by the CPIA micro level the linkage between effective Bank G o o d O u t c o m e s o f B a n k A s s i s t a n c e A r e F i g u r e 2 . 2 A s s o c i a t e d w i t h P r o g r e s s i n P o l i c y R e f o r m Outcome of Bank Assistancea Policy Improvement Policy Deterioration Armenia Guatemala Brazil Dominican Republic Bulgaria China Croatia Moderately Satisfactory Lithuania Or Higher Mongolia Russia Rwanda Tunisia Vietnam Moderately Unsatisfactory Haiti Or Lower Zambia Zimbabwe a. Based on sixteen countries for which CAEs rated Bank assistance in 2001 or later, the mid-year of the period covered by the CPIA data used in this report. 1 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S support and better sectoral outcomes. This improve where policies are improving. analysis shows, on the basis of a careful specifi- To test this relationship, the thematic cation of the results chain and examination of categories assigned to projects were data collected specifically for the study, that matched to the twenty components of the Bank support has had a substantial positive CPIA.8 Changes in thematic project evalua- impact on education outcomes in Ghana (box tion outcome ratings between FY94-98 and 2.2). Bank support increased access and FY99-03 exits are fairly strongly correlated resulted in improved educational outcomes. (correlation coefficient of roughly 0.5 with 20 observations) with changes in the Good policies and good project evaluation corresponding mapped CPIA components outcomes go together. between 1999 and 2003 (figure 2.3). CPIA ratings are relatively highly correlated with country-level project evaluation results-- How Does the Bank Handle the "Tough with correlation coefficients of the order of Cases"? about 0.5, which is significant at the 1 percent The Bank thus has done reasonably well in the level. This is true for MDG-related projects, as aggregate in associating its support with was demonstrated in ARDE 2002, which opportunities for policy reform. Bank examined the relationship between the overall- assistance has been concentrated in reforming CPIA index and disbursement weighted project countries, and improved project performance outcomes.7 Projects aimed at Education for All, has been associated with policy improvements Basic Health, and Environment, like all at both the country level and the policy theme projects, were much more successful in level. As noted in chapter 1, this may be due in medium and high CPIA countries than were part to the fact that during these last several comparable projects in low CPIA countries. years the Bank has been operating in an Irrespective of thematic focus, country factors environment that has been favorable for are thus important determinants of develop- reform. ment effectiveness (Dollar and Svensson 1998). How did the Bank handle country situations where there was no prior information on a Policy improvements are linked to project country's policy track record, where the track outcomes. record was brief (e.g., new borrowers), or Not only are project outcomes better in where the track record was mixed or bad? Has good policy environments, they tend to it used triggers and conditionality effectively? B a n k S u p p o r t C o n t r i b u t e d t o I m p r o v e d B o x 2 . 2 E d u c a t i o n a l O u t c o m e s i n G h a n a OED studied how Bank-supported educational reforms class as was common before the reforms. Increased school introduced in Ghana affected educational outcomes, and how quality can in turn be linked to the Bank's support, which financed these changes were related to Bank-supported interventions. the construction of 8,000 classroom blocks and provided 35 Between 1988 and 2003, enrolments in basic education million textbooks over the 15-year period. . . . In one community increased by over 10 percent and learning outcomes increased surveyed in which a new school was built, enrolments rose from substantially. These improvements are associated with higher less than a quarter to 80 percent of children. Bank support helped income, better nutrition, and reduced mortality. sustain initially unpopular reforms, demonstrating the efficacy of The gains in educational outputs can be attributed directly to working in partnership with a government committed to a well- better school quality, manifested in improved infrastructure and defined sectoral strategy. greater availability of school supplies. Today it is the norm to have one textbook per child for math and English, rather than one per Source: OED (2004d). 1 6 O W N E R S H I P : L I N K I N G B A N K S U P P O R T T O P O L I C Y R E F O R M I m p r o v e d P r o j e c t O u t c o m e s A r e A s s o c i a t e d F i g u r e 2 . 3 w i t h I m p r o v e m e n t s i n R e l a t e d P o l i c i e s Change in CPIA 60 40 20 R2 = 0.36 0 -20 -10 0 10 20 30 Change in Percentage of Projects Rated Highly Satisfactory and Satisfactory Note: Change in CPIA is change between 1999 and 2003. Change in percentage of projects rated highly satisfactory and satisfactory is change in ratings mapped to the 20 CPIA themes for FY94-98 exits versus FY99-03 exits. Country assistance outcomes are less good-- adopted at least in part in response to strong at least initially--when there is no track encouragement from G-7 shareholders to build record to guide engagement. Bank programs quickly and to help ensure that The Bank gained a number of new clients in the the transition process was a success. Special early 1990s with the breakup of the former allocations of IDA beyond those indicated by Soviet Union and the realignment of countries IDA's system of performance-based allocations in Central and Eastern Europe. These countries were made for the poorer CIS transition had no established policy track record, and countries. the Bank had little country knowledge-- The early results were not good in compari- whether basic economic or practical country son with overall Bank averages. In six of ten operational knowledge--to inform its lending countries evaluated (60 percent), the early decisions. outcomes of Bank assistance were unsatisfac- The Bank initially approved substantial tory compared to a Bank-wide average of 30 volumes of adjustment lending to support percent unsatisfactory outcomes in recent ambitious programs of stabilization and Bank CAEs. Institutional development impact structural reform--particularly in many of the was rated high or substantial in only three twelve Commonwealth of Independent States transition countries (and modest in the other (CIS) transition countries. Total lending to CIS seven) and sustainability was rated likely in six countries amounted to $16 billion between cases and uncertain in four. FY92 and FY98, equivalent to 12 percent of Outcomes of Bank assistance to three total IDA and IBRD lending. This stance was countries have been rated as satisfactory in 1 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S subsequent years. Performance in two others period with a relatively low level of lending, and has not been evaluated by OED, but there has providing for gradual increases in lending levels been a substantial improvement in the policy if certain policy triggers were met. Early and outcome indicators for these countries. In implementation under the 1998 CAS was each of these five cases, improvements promising, but was not sustained and did not followed a shift by the Bank to more cautious fully meet the policy triggers for increased lending stances. lending. Nonetheless, the Bank scaled up In Bulgaria (one of the three countries in lending levels. This sent the wrong signal to the which the CAE found improved outcomes), for government concerning the importance the example, in the year before the 1996 financial donors attach to reform in Kenya. crisis, the Bank--foreseeing crisis and In the Dominican Republic, in spite of a inadequate government action to forestall it-- legacy of difficult relations, the Bank maintained adopted a cautious stance and only gradually an active and influential policy dialogue in the re-launched a full lending program in 1997. In late 1980s, helping to shape the government's the interim, the Bank worked closely with the "New Economic Program." Bank ESW in the International Monetary Fund, providing advice early 1990s identified an agenda for reform that on the Fund's structural reform program for would consolidate initial efforts (OED 2003b).9 the country. With growing evidence of govern- But unconvinced of the country's ability to ment commitment and achievement of sustain the reform effort, the Bank operated macroeconomic stabilization, the Bank began without an official country assistance strategy to support a broad reform program through a from FY92 to FY95. It did not lend during a series of sectoral reform loans. crucial period of successful economic reform and has since had difficulty in re-establishing a Countries with poor track records: dialogue with the government.10 an opportunity to jump-start reform or another In the Dominican Republic, the Bank was program that could fail in the making? probably too cautious; in Kenya, not cautious Sometimes countries with poor or mixed track enough. While each country situation must be records on policy reform request Bank support addressed on its merits, cases of a poor track for a reform program. These requests may be record call for at least as much caution as cases triggered by crisis or its aftermath; a change of of no track record. Unless the underlying politi- government; the opportunity to join a regional cal economy has changed, the odds are against free trade agreement, the World Trade Organi- success. In the transition economies, as zation, or the European Union; or perhaps, mentioned above, the political economy of unusually well-coordinated donor pressure. reform eventually changed, but this is not The difficulty facing the Bank is in knowing always the case. whether ownership will be real and durable "this time," as opposed to all those other times. Can triggers help link Bank support to reform? The Bank has to balance the risk of lending that The Bank has long used program CAS is not validated by subsequent reform perform- "triggers," as it did in Kenya, to adjust its ance with the opportunity to support reform country programs during CAS implementation that might otherwise fail. to events as they unfold.11 These are intended The case of Kenya illustrates the difficulties to ensure that the volume and composition of the Bank faces in assisting countries with a poor Bank support provided during the CAS period track record (OED 2003f). For over two decades adjust appropriately in view of developments and over nine adjustment operations prior to in the borrowers' internal and external circum- the 1998 CAS, disappointing progress had been stances. recorded in most Bank operations. The lending There is no quantitative evidence on the role program in the 1998 CAS recognized the risks played by triggers (as distinct from other to program implementation by starting the CAS instruments) in maintaining the aggregate 1 8 O W N E R S H I P : L I N K I N G B A N K S U P P O R T T O P O L I C Y R E F O R M linkage of Bank support to performance that is debt while net transfers for development were documented in figure 2.1. A review of recent small. Minimal creditworthiness was maintained, CAS progress reports suggests that triggers play but stabilization was complicated by inflationary a constructive role in linking Bank support to measures to service debt, and the development policy reform at the country level when events impact of Bank assistance was unsatisfactory evolve more or less in line with the range of (OED 2002j). expectations at the time the CAS was prepared. Even where triggers have been specified, the Sometimes, demand/capacity factors also play Bank may need to override them to respond important roles in limiting support to a country adequately to unforeseen events. The 1997 whose policy performance meets triggers for Brazil CAS did not foresee the possible need higher lending levels. for a substantial amount of adjustment lending Experience in the few cases in which in the period covered by the CAS. But the programs have not been triggered effectively financial crises of 1997 in Asia and 1998 in suggests that the absence of provisions for Russia revealed the need for a change in adjusting Bank support to evolving conditions Brazil's policies and for increased policy-based can contribute to low effectiveness of Bank lending. The Bank submitted a Framework assistance. In Peru, the 1997 CAS identified the Paper that effectively constituted a new CAS, risks to the reforms supported by the proposed approved by the Board in December 1998 country assistance program, but argued that (OED 2004e). calibrating the Bank's lending program at that These cases underscore the important but time was not appropriate in view of the govern- limited role that CAS triggers can play in ment's strong track record and a planned keeping support aligned to performance. halving of Bank lending to Peru. In effect, the Triggers are most useful in dealing with possible CAS committed the Bank to a single lending future developments and responses that are strategy irrespective of progress in implement- clear at the time of CAS preparation. They are ing policy reforms; the environment for reform less useful in cases where future developments deteriorated sharply. The absence of triggers to and appropriate Bank responses are highly reduce lending and of an alternative strategy uncertain. The difficulty in detailing meaningful reduced the leverage and the margin of triggers in such situations may account for the maneuver the Bank had in its negotiations with finding in the 2003 CAS retrospective that, while the Peruvian government once the downside improving in their identification of risks, about risks started to materialize. As a result, the Bank 30 percent of the CASs reviewed did an unsatis- was slow to adjust its operations and commit- factory job of mapping out alternative Bank ted considerable sums into a rapidly deteriorat- responses to possible developments and ing policy environment. outcomes during CAS implementation, and that In Zambia, the 1996 CAS argued that a high- roughly 40 percent of the macroeconomic case lending scenario was the only plausible triggers and 45 percent of the structural triggers strategic option. But the FY96-99 period was reviewed were not sufficiently specific (World characterized by repeated slippage in IMF-agreed Bank 2003c). Pre-specified automatic program programs, delays in key structural reforms, and a adjustments may be best suited to only a few halving of net ODA commitments, including the marginal adjustments that the Bank is certain it withholding of virtually all balance-of-payments would want to make in all events. Changes not support from the international community. The foreseen by the CAS or new information that Bank continued--indeed, modestly increased-- suggests that the Bank's response should be its high-case lending (committing 5 percent different from that envisaged in the CAS should more than originally planned) while interna- normally call for a reconsideration of the Bank's tional support waned. The result was an under- strategy. Where uncertainty is high at the time implemented, under-financed program in which of CAS preparation, a short CAS horizon is an exceptional allocations of IDA helped to service appropriate response. 1 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Conditionality is effective in linking Bank development of country-led partnerships and support to reform only in certain circumstances. ownership of the reform agenda, particularly in Conditionality, in principle, "automatically" links the later stages of reform. Bank action (disbursement under adjustment One way that has been proposed to address instruments or, in the case of covenants in some of the ownership, accountability, investment loans, continuation of implementa- incentive, and ambiguity problems with tion of the loan) to a borrower satisfying conditionality as it has usually been applied is specified conditions. Policy research evidence is to formulate conditions in terms of variables very strong in its conclusion that, in practice, nearer the outcome end of the results chain conditionality per se has been ineffective in rather than input or intermediate steps linking aid with reform (Collier 1997). Countries (Koeberle 2003).13 Properly formulated, such that do not want to reform don't and can still conditions are less ambiguous (in the sense avoid any serious loss of funding (Thomas that the Bank, the borrowing country, and the 2003). Conditionality is often ineffective in larger development community can determine modulating aid flows because the incentive more objectively whether they have or have not structure makes it difficult for donors to been met). In principle, they could also provide disengage when conditions are not met (Kanbur more flexibility to the borrowing country to 2000), as the case of Ghana demonstrates (box meet objectives in the best way under the 2.3). Other factors cited are the difficulty of circumstances.14 specifying clear and appropriate conditions with Outcomes-based conditionality was pilot regard to reform given the uncertainties of tested by Burkina Faso and its development meaningful policy reform processes and the partners with promising early results (box 2.4). relative ease with which borrowers often can The pilot facilitated donor coordination in the comply with the letter, but not the spirit, of context of Burkina Faso's first Poverty conditions. When countries do want to change Reduction Strategy Paper (PRSP), and the their policies, conditionality may be useful in the monitoring and evaluation framework for the early stages of reform in bolstering the position PRSP derives many elements from the test of reform supporters.12 Researchers and practi- framework. The European Union currently tioners alike, however, point to a tension links a portion of its support to progress as between donor-imposed conditionality and the reflected in these indicators. C o n f l i c t i n g I n c e n t i v e s S o m e t i m e s M a k e B o x 2 . 3 C o n d i t i o n a l i t y D i f f i c u l t t o E n f o r c e ...Up to 1992, Ghana had been called the "star pupil" of the holding up as much as one eighth of the annual import bill of Bretton Woods Institutions, with an adjustment program that the country. was proclaimed by the IMF and the World Bank as perhaps One would think that holding one eighth of the annual the most successful in Africa.... But in 1992, Ghana consum- import bill of a poor cash strapped economy would give mated its transition to democracy and, in the process, the enormous leverage to the World Bank and the donors to dictate government gave in to pressures to grant enormous pay terms to the Ghanaians. In fact, ... the World Bank ... came increases to civil servants and the military...As a result, the under pressure from several sources, some of them quite budgetary conditionality in the World Bank's then current surprising, to release the tranche with minimal attention to Structural Adjustment Credit was violated, and the impending conditionality. tranche release was suspended. Through its own tranche, and through co- financing tied to it, the World Bank found itself Source: Excerpted from Kanbur (2000), pg. 5. 2 0 O W N E R S H I P : L I N K I N G B A N K S U P P O R T T O P O L I C Y R E F O R M Subsequent experience with the implemen- Conditionality based on completion of tation of the Burkina Faso PRSP has suggested actions prior to disbursement within a program- that the approach used to link disbursements matic framework has been suggested both as a to outcomes developed in the pilot may need way to reduce the severity of some of the some refinement. The government notes in its problems of traditional conditionality and as a PRSP progress reports that it is difficult to way to retain some dimensions of outcome- monitor and evaluate progress on the basis of based conditionality while attenuating some of outcome or impact indicators that change little its problems. There is evidence, reflected in on an annual basis and that are subject to rapid growth of the use of programmatic instru- factors outside the government's control, and ments and in client surveys, that borrowers like it is refining its indicators in view of these programmatic lending. There is, however, little problems. The Bank's support to the govern- research or evidence yet on the effectiveness of ment's program is still based primarily on programmatic approaches, and some concern implementation of agreed intermediate policy that conditions may not be as clearly linked to measures. outcomes as they should be. OED's evaluation O u t c o m e s - B a s e d C o n d i t i o n a l i t y S h o w s B o x 2 . 4 P r o m i s e i n a P i l o t i n B u r k i n a F a s o A pilot exercise in Burkina Faso initiated by the European agement" from donors. In areas where the government's Commission and the Burkina Faso government demonstrated policymaking capacity requires strengthening, the donors' that outcomes-based conditionality could attenuate many of responsibilities lie in providing advice should the govern- the problems of traditional conditionality. In the pilot program, ment seek it. donors--including Belgium, Denmark, France, Germany, the · Ownership--a good start: The results-based approach IMF, Japan, The Netherlands, Switzerland, and the World enhanced ownership because there was increasing partic- Bank, along with the European Commission ­"simulated" the ipation at the highest political levels as the pilot linkage their aid to achievement of development outcome progressed. The exercise changed the manner of thinking objectives identified jointly with the government. The exercise within the ministries as the focus on results grew. reached the following conclusions: · Broad-basedstakeholderinvolvement--stilladifficult issue: The government was initially reluctant to involve the · Shift the focus to results: Outcome indicators shift the civil society and make the outcome targets public. Progress focus of reforms to results and change the nature of the was made, and there is fairly widespread dissemination of government/donor dialogue from negotiating specific data and participation in monitoring under the PRSP process. reform measures to examining options to improve results. · Donor coordination--worth the logistical difficulties. They also highlight areas where policies are not deliver- Despite some logistical difficulties, donors were able to ing. For example, in the pilot, despite 10 years of increasing draw on each others' sector expertise and to arrive at a sectoral budget allocations and donor support as well as common understanding on the direction and mechanisms new cost recovery schemes in the health sector, to follow in the pilot. The government has particularly attendance rates at health centers had steadily decreased. appreciated the joint approach, because it has helped to · Partnership--shared responsibilities: The government free scarce capacity and ensure coherence in relations pointed out that donors shared responsibility for poor with various donors. results of policies they had imposed as conditions of past · Linking disbursements to outcomes: The pilot identi- support. The pilot included few conditions on how the fied a number of options for calibrating disbursements to results were to be achieved. If the government is to be results as reflected in outcome indicators. judged on its results, it should have freedom to determine the scope and timing of policy changes without "microman- Source: Adapted from Zongo et al. (2000). 2 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S of the Bank's assistance to Brazil--which has Through the PRSP process, good progress made heavy use of programmatic lending over has been made in some countries in developing the last several years--found that programmatic country-owned strategies. The CDF evaluation lending in Brazil needed strengthening through found that the strongest sense of ownership closer linkages to real economic outcomes, appears to have developed where a highly quantified and time-bound benchmarks, and a inclusive participatory process is conducted in framework for measuring results (OED 2004e). tandem with and through the country's normal These considerations raise a number of political processes and institutions (box 2.5). To questions: Why is traditional conditionality still foster ownership, the evaluation recommends so widely used? Have conditionality reforms broader participation where participation has such as grounding it in country-owned been limited. Another factor noted by the programs, shifting to programmatic lending evaluation is the need to lessen the tension approaches that disburse on completed between the PRSP's role as a process for actions, reducing the number of conditions, building domestic consensus and ownership and providing debt reduction options as an and the roles that its plays with regard to debt alternative to adjustment lending in highly relief and access to aid resources (OED 2003j). indebted countries made conditionality more Progress has been slower in the other effective (Killick 2003)?15 These questions dimensions of aid processes addressed by the deserve attention as the Bank moves forward CDF. Efforts to develop long-term holistic to develop new and more flexible instruments frameworks and to adopt a results orientation and approaches to supporting policy reform. have been slowed by capacity constraints, continuing asymmetries in donor-recipient Are Partnership Approaches Working? relationships, and the difficult politics of aid in Uncertainty about ownership is reduced when many donor countries. the Bank's support is aligned with country To support country-owned development priorities that were set and validated through frameworks, donors should provide both long- country-led, inclusive political processes before term assistance for capacity building and any Bank support was provided. This does not predictable, transparent, and reliable financing. assure success--consensus is by its nature Budget support and sector program aid should usually shifting and evolving--but it does link be based on mutually agreed country perform- Bank assistance more closely to the policy ance criteria. Successful implementation of CDF process. While a good start has been made in principles will require both donors and recipient this direction with the promotion of aid countries to make significant changes in their processes based on Comprehensive Develop- behaviors and processes, changes that may be in ment Framework (CDF) principles, this is still a conflict with the political environments of most work in process, with solid progress along some donors and with the incentives and development dimensions and difficulty along others. goals of both donors and recipient countries. 2 2 O W N E R S H I P : L I N K I N G B A N K S U P P O R T T O P O L I C Y R E F O R M C D F P r i n c i p l e s F o s t e r C o u n t r y - O w n e d B o x 2 . 5 S t r a t e g i e s Toward Country-Led Development, the multi-stakeholder systematic manner. By itself, this tends to produce a lobby evaluation of the Comprehensive Development Framework for increased spending rather than a process where (CDF), found that: tradeoffs are made within budget constraints because representatives of sectoral interests will understate the · Change driven by domestic interests is usually more priority of policies that produce diffused nationwide sustainable than change induced by external sources. benefits such as trade liberalization and exchange rate · The shift from Policy Framework Papers designed by the reforms. international financial institutions to Poverty Reduction · A participatory process can be captured by vested interests Strategy Papers designed primarily by governments, with that use the opportunity to delay widely beneficial reforms. inputs from other domestic stakeholders, has promoted In such cases, the government needs to inform and guide a country ownership. participatory process, not simply listen. · Country ownership of strategy and reforms is more likely to · While the shift from international financial institution be sustained when, consistent with the CDF country ownership of the reform process to country ownership has ownership principle, there is regular broad-based dialogue been both considerable and beneficial, the attempt to that addresses tradeoffs and priorities. Of the six country broaden participation within society beyond government cases evaluated, Uganda is perhaps closest to good practice. was perhaps insufficiently thought through. Because of its mandate to link the planning and budgeting functions, the Ministry of Finance, Planning and Economic A strong reform process is indeed more likely to be Development has taken the lead in the PRSP process. sustained if the decisions for reform are seen as legitimate. · The mechanism the CDF uses to encourage country ownership is to give voice to diverse interest groups in a Source: OED (2003j). 2 3 3 Country Fit: Adapting Bank Support to Country Conditions Try this bracelet: if it fits you wear it; but if it hurts you, throw it away no matter how much it sparkles. --Kenyan Proverb Chapter 2 showed that the Bank has been cases where major issues were missed, it was reasonably effective in the aggregate in because the Bank lacked knowledge about new supporting countries that are improving their borrowers. OED's recent evaluation of support policy environments. This chapter examines to transition countries found, for example, that the Bank's effectiveness in tailoring its support the Bank largely missed the poverty and to meet the particular needs of individual governance problems in the Europe and countries--focusing on the right priorities and Central Asia region until the latter part of the using the right combination of instruments, 1990s (OED 2004a). with adequate space for political processes to The record with regard to details of specific evolve, and understanding of specific country policy reforms is more mixed. The Brazil CAE contexts. In this regard, the Bank's perform- notes, for example, that earlier and increased ance has been mixed, hampered in some cases Bank emphasis on fiscal measures to increase by inadequate country knowledge, in others by the national savings rate, financial sector reform inadequate sensitivity to country policymaking to improve the efficiency of intermediation, and styles, and in still others by attempts to regulatory reform to improve the environment transplant policies and institutions--some of for investment in infrastructure could have which "travel" more readily than others, as helped Brazil achieve higher investment rates argued in box 3.1--without adequate analysis and thus higher growth during the 1990s (OED of the context. 2004e). The evaluation of Bank support to transition economies notes that, in retrospect, Do Bank Country Strategies "Fit"? it is clear that the sequencing of reforms was OED Country Assistance Evaluations rate often wrong. The Bank, along with other approximately 70 percent of the outcomes of donors, should have tried harder to persuade Bank support as moderately satisfactory or borrowers to devote greater attention to better. At the broadest level of strategic focus improving the institutional framework for and allocation of resources, the Bank has business and investment in parallel with generally addressed the right issues. In many promoting privatization (OED 2004a). 2 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S T h e U n c e r t a i n t i e s o f P o l i c y R e f o r m B o x 3 . 1 D e p e n d o n t h e P o l i c y A r e a Some reforms are grounded in well-established technical take some months or years to put into effect. Such reforms also models, while others have multiple models to choose from. On require considerable detailed information, and mechanisms to the spectrum of policy reforms, macroeconomic reforms, such collect and analyze the needed data often must be created or as devaluation or interest rate adjustments, that are guided by upgraded. Postponing reforms implies direct and clear costs an established analytical framework are at one end. Such for some groups, but only indirect and/or delayed and often reforms can be put into effect by a handful of high economic unrecognized costs for many others. officials (with the backing of top political authority), and they At the other end of the spectrum are systemic reforms in take effect rapidly. They require certain basic economic education and health services and social protection. Multiple indicators, which are available in all but the least sophisti- models are available, influenced by different national and cated countries. Postponing action has high costs, affecting regional histories. More important, there is only limited much of the population immediately and directly. consensus among technical specialists regarding basic princi- Toward the center of the spectrum are reforms entailing ples of reform. Ministries and legislatures at national, state, organizational restructuring and legal changes, such as and local levels are usually involved. Despite widespread financial sector reforms. Broad principles of these reforms are dissatisfaction with the status quo, there are no compelling fairly well established, but there is considerable room for costs to politicians of postponing action. Once launched, many debate and adjustment of design to specific country circum- social sector reforms require years to implement. stances. A wider range of government and private sector actors must collaborate on the reforms, and they are likely to Source: Adapted from Nelson (2000). Country fit--particularly fit with policy initially to support policy reform through processes--matters. heavily conditioned multi-tranche adjustment Successful Bank strategies typically are well loans. The authorities were genuinely commit- attuned to country policy processes, as the case ted to reform and initially shared the Bank's of Tunisia demonstrates (box 3.2). Even when enthusiasm for ambitious, detailed programs. country ownership of reform is strong, a It quickly became clear, however, that the mismatch between Bank strategy and country Bank's program was premised on a pace of policymaking styles creates tensions and can reform more rapid than could be handled by cause Bank assistance to have poor results. In the Uruguayan system, where a slow but steady Uruguay, for example, the Bank attempted pace of reform reflects a consensus style of A d a p t i n g S u p p o r t t o P o l i c y R e f o r m S t y l e s B o x 3 . 2 i n T u n i s i a Tunisian authorities have followed a gradualist, cautious move prudently and deliberately to ensure that all concerned approach to reform to ensure as broad a political consensus as were on board. possible in sensitive areas. This has led to some delays, from In the words of an official: "We wait until things ripen and the standpoint of the Bank, and it entails some risks, as the then we move, step-by-step. This is why there has been no economy faces more competitive pressures. On the other hand, reversals or slippages in this phase. For example, after seven the gradual approach has allowed the government to build the years of partial trade liberalization and five years of hesitant necessary consensus and has served the overall objective of privatization, both have now accelerated." reform well. Even when the authorities were themselves convinced of the reforms, they have chosen to continue to Source: OED (2004k). 2 6 C O U N T R Y F I T: A D A P T I N G B A N K S U P P O R T T O C O U N T R Y C O N D I T I O N S policymaking in a traditionally democratic conditionality. This preference led to Bank society. By the time the CAS was developed in strategies for supporting reform based on 1997, the policy dialogue had deteriorated. "dialogue" and "example," with a heavy focus Recognizing that the dialogue was bogging on infrastructure. In fact, the Bank has made down, the Bank moved to single tranche adjust- only one adjustment loan to China, a single ment operations that recognized policy tranche loan in the mid-1980s to support rural achievements as and when they materialized. development (box 3.3). This strategy permitted the government to In Chile, the Bank played a key supportive pursue agreed objectives at a pace and by the role, with lending and advice, in helping the means dictated by the domestic policy process country out of the mid-1980s crisis and in facili- rather than as pre-specified by Bank lending tating the economic transformation that operations. The CAE found this strategic switch followed. The Bank's assistance evolved from a to have been critical to the success of the program based on heavy use of adjustment Bank's program (OED 2000b). lending to help Chile re-establish access to Such flexibility led to good results because international capital markets to a program that the government assumed full ownership of its included infrastructure and agriculture invest- program. The lesson here is that the Bank and ment loans to reduce bottlenecks hindering the government should agree on broad indica- economic and export growth. Once stability tors of the effectiveness of development initia- and economic growth were secured, and at tives. But the government itself should have Chile's behest, the Bank emphasized social greater scope to decide how best to achieve the welfare and human resources development, agreed reform objectives. with loans for education and health increasing In Costa Rica, the Bank's phasing and design their share in Bank lending (OED 2002c). of the 1993 CAS jeopardized what had been a As at the country level, successful cases of productive relationship. The lessons of experi- Bank support for sector reform typically also ence at that time showed that Costa Rica has involve combinations of instruments. Ownership, always been a slow reformer--with the pace of Bank flexibility, and responsive adaptation to reform dictated by the complicated politics of country circumstances are important factors consensus in a pluralistic, democratic political contributing to sectoral success, as education system. An adjustment loan taken to the Board reform efforts in Ghana and Uganda illustrate with the CAS required Costa Rica to pass (box 3.4). legislation for the loan to become effective. This was a mistake. The Bank had to extend the Adjustment lending provides effective support effectiveness deadline six times while waiting to reform in the right environment, but is not for the required passage of legislation, and always associated with overall improved ultimately canceled the loan two years after policy performance. Board approval. The policy dialogue deterio- Since the mid-1990s, adjustment lending has rated, negatively affecting the overall lending accounted for a steadily increasing share of and non-lending program of the 1993-99 Bank lending. This increasing share is attribut- period (OED 2000a). able to steady increases in adjustment lending volumes accompanied by simultaneous sharp Good policy reform results can be achieved declines in investment lending. Both the with a variety of instrument mixes. increase in adjustment lending and the decline Good results have been obtained with a variety in investment lending were concentrated in of Bank instruments, themes, and sectors, with IBRD countries relative to IDA countries-- the mix tailored to the country situation and adjustment lending accounted for 51 percent preferences, and usually evolving over time. In of IBRD commitments during FY99-03, almost China, officials prefer gradual and pragmatic double the share (27 percent) of such commit- reform and reject approaches based on ments in FY94-98. In contrast, the share of 2 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S B a n k S u p p o r t f o r P o l i c y R e f o r m i n B o x 3 . 3 C h i n a R e l i e s P r i m a r i l y o n D i a l o g u e a n d D e m o n s t r a t i o n The China CAE identifies four factors underlying the Bank's were not. The Bank continued to work with Shanghai in other effectiveness in supporting China's policy reforms: sectors, however, and Shanghai later introduced housing Finding common ground through ESW, workshops, and other reforms on its own. AAA activities was a key method. The target audience was Experimentation and demonstration. The Bank relied on the mainly senior officials and their policy advisers, and the demonstration effect of successful project experience in objective was to persuade them to implement top-down reforms. introducing new technologies, management methods, or Dual-track approach of building trust through lending while policy reforms to leverage the impact of project outcomes. This carrying the policy dialogue through sector work. Each project was largely a bottom-up process of diffusion by example, was intended to push the frontier of policy or institutional though China also uses pilot projects to test policy changes change, but the approach was gradualist and depended less before they are rolled out nationally. on conditionality than persuasion. One drawback of this pragmatic engagement strategy is Working with willing partners. The Bank worked with the that important issues may be omitted if key actors on the power sector, which wanted access to the technology Bank government side are unwilling to engage. The dual track lending could provide and which was receptive to policy approach can also lead to overly modest project objectives, suggestions. In contrast, the Bank withdrew from working in and accountability for the Bank's performance can be difficult the coal sector because the ministry responsible was not to assess when policy issues are left to the uncertain timing interested in policy dialogue. In housing reform, the Bank and outcome of the policy dialogue track. worked with municipalities that were willing to introduce cost recovery pricing and excluded areas such as Shanghai that Source: OED (2004f). adjustment commitments for IDA countries ments, of supporting policy reform in the right remained relatively flat at about the 25 percent environment. In Armenia, for example, adjust- upper limit set by the IDA replenishment ment lending accounted for almost half of Bank agreements in both the FY94-98 and the FY99- lending over the period FY99-03. However, it 03 periods. was complemented by Bank investment Is this shift toward adjustment lending lending and was based on a country-owned associated with the observed improvement in agenda previously mapped out with Bank policy performance as reflected in recent policy analytical support (OED 2004c). indicator evidence? The answer is "No."1 The Strong reform performance can also be increase in adjustment lending shares was achieved without adjustment lending, as driven primarily by macroeconomic crises in illustrated by Chile and China. These cases several large countries, in conjunction with a underscore the point that it is the match of the decline in investment lending. Some large crisis instrument mix to the country context, not the countries improved their policies during the choice of instrument per se, that is paramount. period under review, but some did not. Several countries made fairly heavy use of adjustment New "learning-friendly" lending instruments lending--e.g., many of the poorer CIS yield promising early returns. countries during the mid-1990s--with mixed The Bank introduced several new instruments results with regard to policy reform but with in the late 1990s that were intended to increase substantial effects on indebtedness. flexibility in implementation and to improve Adjustment lending can be an effective country fit. The most important of these new means, in conjunction with other Bank instru- instruments are Adaptable Program Loans 2 8 C O U N T R Y F I T: A D A P T I N G B A N K S U P P O R T T O C O U N T R Y C O N D I T I O N S B o x 3 . 4 F o s t e r i n g O w n e r s h i p o f E d u c a t i o n R e f o r m s The Bank has supported successful reforms in the education productive relationship between Bank and government sectors of Ghana and Uganda by working closely--and often education staff. compromising--with government officials on specific reform In Uganda, the Bank's rapid response to the government's measures. program to rapidly expand access to education through the In 1986, Ghana started to implement reforms first Education Sector Adjustment Credit in 1998 enabled the proposed in 1971 but shelved because of political opposition. government to double primary school enrollments while The Bank supported the reforms through a combination of averting a crisis in the education system. The Bank fostered investment and adjustment financing; it compromised on a country ownership by involving inter-ministerial working number of issues. For example, the government proposed an groups in project preparation to develop a country-owned increase in the vocational element of the curriculum at all sector strategic plan. Its subsequent support to a sectorwide levels, and Bank staff advised against this. In the end, the approach (SWAP) enabled the Ministry of Education to develop government's proposal was somewhat reduced, but and take charge of consultative processes for ongoing policy- remained substantially intact. While subsidies to tertiary making, planning, and budgeting in the sector. The Bank's role education were reduced, as recommended by the Bank, this in the SWAP has been to support--rather than lead--other happened with some delay and they were replaced with a agencies while contributing high-quality technical inputs in subsidized student loan scheme. Bank staff, aware of the critical areas. This approach has supported Uganda in political sensitivities manifested in frequent student unrest, developing and agreeing on a national strategy for secondary did not press the government for speedier action and did not education that accounts for a range of stakeholder interests hold up loan disbursements for the technical violation of a and also benefits from strong technical analysis. loan condition relating to student subsidies. This accommo- dating position allowed the continuation of a close and Source: OED (2004d, 2004l). (APLs), Learning and Innovation Loans (LILs), recent exits, but the numbers are small and Programmatic Structural Adjustment Loans none has yet been subjected to the more (PSLs), and Poverty Reduction Support Credits thorough evaluation entailed in the prepara- (PRSCs). tion of a Project Performance Assessment Utilization of these instruments, particularly Report. APLs, PSLs, and PRSCs, has grown relatively rapidly, accounting for 16 percent of IDA and Timely, well-focused country ESW contributes 17 percent of IBRD commitments over the to better development assistance outcomes period FY99-03. Increased APL and PRSC usage and reduces risk. accounted for most of the new instrument The Bank stresses the importance of country growth among IDA borrowers, while PSLs and and sector knowledge to reduce uncertainty APLs accounted for the largest shares of new and risks in the processes of country strategy instrument growth among IBRD and blend design, policy dialogue, and design of lending borrowers. operations (box 3.5). Evaluation evidence Forty-one of these operations have closed corroborates the key strategic role of ESW in and been evaluated by OED.2 The early ratings shaping the Bank's country assistance strategies are very promising. Eighty-one percent of rated and its roles in the policy dialogue and project projects are rated as satisfactory, 65 percent are design. Although evaluations have given mixed rated as having a substantial institutional reviews to the Bank's use of ESW, in countries development impact, and sustainability is rated where outcomes of Bank assistance were as likely in 86 percent of the projects. These evaluated as satisfactory, high quality, relevant, ratings compare favorably with those for other timely ESW frequently made a substantial 2 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S contribution. In countries where outcomes The Zimbabwe case dramatizes the risks that were not satisfactory, deficiencies in ESW are inadequate information can lead to costly frequently cited as a contributing factor. In an errors. Analyses of the relationship between evaluation of 34 countries, ESW had a satisfac- ESW and performance of Bank operations tory impact on 15 of the 23 countries where confirm the importance of an adequate country outcomes were rated as satisfactory. information base to successful project designs. Out of 11 countries where outcomes of Bank Research carried out at the Bank indicates that assistance were rated as unsatisfactory, ESW was its ESW has had a significant positive impact on found to have had a satisfactory impact on the various measures of the quality of World Bank country dialogue in only four cases (OED 2001a, projects (Deininger et al. 1998). Table 4.2). Figure 3.1 shows additional examples In 2002, Bank management issued a require- from six CAEs in which ESW influences the ment that all active countries have a Poverty outcome of Bank assistance. Assessment, a Country Economic Memoran- ESW can also help to identify avoidable risks. dum or Development Policy Review, a Public The Zimbabwe CAE, for example, traces the Expenditure Review, a Country Financial country's debt problem in the mid-1990s in Accountability Assessment, and a Country part to faulty sequencing of reforms under its Procurement Assessment Report, or their Economic Structural Adjustment Program, functional equivalents, that is not more than which the Bank supported with two Structural five years old. In the case of IDA countries, the Adjustment Credits. Tax rates were reduced Bank undertook to meet this requirement by and interest rates deregulated before expendi- the end of FY04. As of end FY03, of the 87 active tures were reduced. The effect was an immedi- borrowers, all five diagnostics had been ate increase in government interest payments completed for 19 countries, and four had been equivalent to 5 percent of GDP, forcing the completed for an additional 28 countries.3 government to borrow more, and leading to Spending on ESW has increased sharply in the rapid accumulation of additional debt. The recent years, as have the number of ESW evaluation found that, had the Bank's analytical products produced. Spending on ESW products work been adequate at the time these reforms (on a direct cost basis) doubled between FY98 were being planned, the risks in this sequenc- and FY03, from $40 million to about $80 million. ing could have been foreseen and the problem The number of ESW outputs delivered increased could largely have been avoided (OED 2003k). by about 80 percent during that time; this S p e c i f i c C o u n t r y K n o w l e d g e I s N e c e s s a r y B o x 3 . 5 f o r P r o j e c t S u c c e s s The 2004 OED review of social development in Bank activities project is doing things that put cultural heritage at risk. Similarly, finds that awareness of the socio-political realities in borrowing a lack of understanding of nongovernmental and community- countries is crucial to effective social development assistance based organizations and their relations with borrowers hindered and that a lack of specific country knowledge hinders project project success. The evaluation recommends that the Bank success. For instance, most projects in the Bank's water "improve significantly" in specific country contexts and that it resources portfolio involved analyses of technical institutions, become more knowledgeable in country issues by conducting systems, and local government institutions, but failed to analyze inventories, performing local institutional analyses, and social and informal private institutions--an oversight that, in the developing good practices, among other things, in order to avoid case of watersheds and river basins, can result in the dominance blunders and improve performance. of some power groups in key decisions. Many task managers lack important country knowledge of the warning signs that a Source: OED (2004b). 3 0 C O U N T R Y F I T: A D A P T I N G B A N K S U P P O R T T O C O U N T R Y C O N D I T I O N S H i g h - Q u a l i t y , T i m e l y E S W C o n t r i b u t e s t o F i g u r e 3 . 1 t h e B a n k ' s I m p a c t o n P o l i c y R e f o r m Satisfactory Outcomes Unsatisfactory Outcomes Armenia. The Bank, in collaboration with the Fund, provided advice Croatia. Important components of the proposed ESW program could and technical assistance in macroeconomic management--a very not be implemented through the 1990s because of a lack of government successful aspect of the country's performance. The Bank highlighted agreement and because the impact of the work that was completed for the government the linkages between the large deficits in the was low. Although much of this program was of high quality, some enterprise and energy sectors and the country's fiscal and quasi-fiscal components such as the Fiscal Sustainability Study were not useful. deficits. Officials recall the relevance of informal analytical and techni- From 2000 on, both the overall quality and the impact of the program cal support provided in the areas of macroeconomics, treasury were satisfactory (OED 2004g). functions, aid coordination, and the energy sector (OED 2004c). Brazil. Government officials state that the Bank is the multilateral Zambia. Overall, the Bank's analytic work in the second half of the institution that the government most resorts to for technical advice. 1990s is judged moderately satisfactory, reflecting substantial They say that the Bank can play an important "pedagogical" role, relevance but only modest efficacy. The findings of analytic work in the informing Brazilian society about long-term structural issues, their mid-1990s were not fully incorporated in Bank strategy, nor were they potential solutions, and the trade-offs involved. To play this role sufficiently disseminated and discussed in order to shape public and effectively, however, the Bank must make a greater effort to dissemi- international opinion and influence the government's policies. nate its work among the several potential audiences and to the Thereafter, no formal analytical work was finalized in the FY97-01 population at large (OED 2004e). period (OED 2002k). China. Macroeconomic advice in the early to mid-1990s and more Zimbabwe. Substantive analytical work on some key issues either was recent work on pension reform were repeatedly cited in interviews not undertaken or was not timely. The public expenditure review (PER) as examples of highly relevant and effective work. Other areas cited in end-1995 came too late to highlight sequencing issues in the govern- as important Bank contributions to debate and policy change in the ment's fiscal program or to inform the design of the Structural Adjust- past decade included VAT reform, WTO entry, public utility reform, ment Credits (SACs) . No substantive analytical work on poverty has environmental policy, and preparation of the most recent Five-Year been completed, and land reform was addressed only sporadically and Plan (OED 2004f). not treated as a priority area until late in 1998 (OED 2003k). included a doubling of Core Diagnostic reports, Evaluation evidence also shows, however, a 50 percent increase in Country Advisory that ESW is not a panacea. Client demand is Reports, and a ten-fold increase in Policy also an important factor in determining Notes/Other Products.4 ESW has become whether ESW is effective in promoting reform. increasingly participatory. Quality has improved An OED evaluation of private sector develop- across the board--the Quality Assurance Group ment in the power sector found that the Bank's (QAG) rated 94 percent of the FY02 cohort of ESW Analytic and Advisory Services (AAA) facili- reports as satisfactory or better. The gap between tated the reform process but that its contribu- the quality of IBRD and IDA country reports and tion at the country level varied widely. Country the gap between the quality of new diagnostics case studies suggest that substantial ESW/AAA (e.g., of procurement and financial manage- alone does not necessarily lead to better sector ment) and other ESW products found in earlier outcomes. Rather, it is a combination of "just- QAG reviews has been eliminated. QAG also in-time" advice, leveraged by commitment rated 90 percent of FY02 products "satisfactory" from government and support from a broad or better in terms of their likelihood of achieving spectrum of civil society, that facilitates reform. their stated objectives. ESW products prepared Another review of evidence in various country with the participation of the client are rated assistance evaluations on the role of ESW in particularly highly for quality and likely impact. supporting reform found that ESW was less 3 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S likely to have a satisfactory impact on the institutions; the decentralization of many country dialogue in low-income countries and responsibilities to state and municipal govern- in countries with relatively low CPIA scores ments, with which Bank staff engage less than in other countries. This finding raises frequently; the emergence of alternative some question about the impact of ESW in low- domestic and international sources of good income countries under stress (LICUS) analysis about the Mexican economy and policy environments. alternatives; and also perhaps the nature of The potential of the Bank's investment in remaining reforms, which often require deep ESW could be better realized, according to a and complex changes in institutional behaviors range of evidence, including CAEs (OED 2002j, for which the Bank's expertise may be less 2004a), QAG reviews (QAG 2003), and OED's relevant (OED 2001d). recent knowledge evaluation (OED 2003i). Clients surveyed for the knowledge evaluation Country debt sustainability analysis has been a said that dissemination of the Bank's problem. knowledge at the country level needs improve- A fundamental dimension of "good country fit" ment. Beyond central government personnel is a sustainable medium-term macroeconomic and a narrow circle of other well-informed framework (box 3.6). OED's recent evaluation persons, the reach of Bank knowledge is said of the Heavily Indebted Poor Countries (HIPC) to be limited. Two contributing factors Initiative found systematic over-optimism in frequently cited are that Bank information can the debt sustainability analysis underpinning be easily accessed only by computer and that it HIPC programs and questioned whether is not available in local languages. Clients programs would meet debt sustainability indicate that there is also a "government bias" objectives (OED 2003a). The Independent in disseminating information. Evaluation Office (IEO) of the International Looking ahead, according to recent CAEs, the Monetary Fund found that IMF programs are Bank's ESW risks losing relevance for some not adequately analyzing debt sustainability in clients. In China, for example, there is a calibrating fiscal adjustments (IEO 2003). widespread view that the Bank's AAA, which was OED's evaluation of the HIPC Initiative uniformly valued in the 1980s, in some instances found the Bank's treatment of debt sustainabil- declined in relevance and quality during the ity analysis (DSA) lacking in a number of 1990s (OED 2004f). Chinese officials and respects. While conceding the appropriateness researchers felt the Bank was slow to recognize of allowing for some effect of reform on and take advantage of the growing knowledge growth, the evaluation found that (a) growth and sophistication of Chinese counterparts in assumptions in HIPC DSAs were optimistic some areas. The Bank continued to provide (more than twice historical averages); and (b) generic advice5--a textbook approach--rather that the Initiative, as implemented, was not than discussing policy options relevant to underpinned by credible country strategies China's needs. As one researcher put it, the consistent with these assumed levels of growth. Bank's glossy reports were thorough and In particular, although the strategies competent, but "Where is the value added?" emphasize macroeconomic stability and There was also a feeling that the Bank made too human capital development, they give little use of Chinese collaborators.6 relatively little attention to other factors The Mexico CAE notes that it will become important for growth--e.g., the investment increasingly challenging for the Bank to have climate and infrastructure. The evaluation also substantial developmental impact through its flagged inadequacies in DSA risk analysis--i.e., advice and technical assistance. This prognosis examination of the sensitivity of projected reflects several factors: the deepening human results to deviations from baseline assump- capital available to the government and the tions; DSAs give particularly inadequate gradual strengthening of central government treatment to export volatility. Finally, while 3 2 C O U N T R Y F I T: A D A P T I N G B A N K S U P P O R T T O C O U N T R Y C O N D I T I O N S I n a d e q u a t e D e b t S u s t a i n a b i l i t y A n a l y s i s B o x 3 . 6 C a n C o n t r i b u t e t o E x c e s s i v e A c c u m u l a t i o n o f D e b t As a result of a special arrangement with the Russian Federa- fact, debt to the Bank and the IMF accounts for a larger share of tion that cancelled their debts, the low-income members of the these countries' debt than it does for HIPC countries. Commonwealth of Independent States--Armenia, Azerbaijan, This rapid accumulation of debt was caused by several Georgia, Kyrgyz Republic, Moldova, Tajikistan, and factors, including "bad luck" and lagging policy reform. In Uzbekistan--started the transition from planned to market addition, however, research indicates that inadequate economies in 1992 with no outstanding debt. Over the balance of analysis of the sensitivity of the macroeconomic frameworks the 1990s, all but Azerbaijan rapidly accumulated external debt. underpinning these countries' programs to uncertainties about By 2000, excluding Azerbaijan, these countries' net present initial levels of GDP growth, the rate of policy implementation, value of debt to exports ranged between 100 and 250 percent of and the supply response to policy reforms played a key role in GDP, and between 500 and 700 percent of government revenues. the debt buildup problem. All but Azerbaijan and Uzbekistan received substantial volumes of policy based lending support from the Bank and the Fund. In Source: Helbling, Mody, and Sahay (2003). recommending increased focus on initiatives countries. The quality of countries' fiscal to attract foreign investment, the evaluation policies has improved much less, as reflected noted that growth will probably also require in an increase in the average rating of this area some element of additional borrowing, and of the CPIA of about one fifth of the overall that debt and debt sustainability would average increase (see figure 1.4). The average therefore need to be monitored closely. level of the fiscal policy CPIA rating is among A recent OED evaluation of the Bank's policy- the lowest. The quality of debt management based lending activities in Guinea since 1988 actually deteriorated--the only CPIA found that IDA had substantially increased its component to register an absolute decrease portfolio exposure, shifting from being one of relative to 1999 levels--although its level the smaller external creditors to being the remains relatively high.7 This deterioration in principal official external creditor (OED 2003e). debt management occurred in both IDA and The consequence was a major increase in IDA's IBRD countries. Outcome data reflect a exposure, a phenomenon that was not general deterioration in the debt positions of adequately assessed and communicated to IDA's developing countries. The IMF's 2003 World management and board in a timely manner. Economic Outlook notes that public debt in Despite its rising exposure as an official external emerging market economies has risen sharply creditor, IDA did not give sufficient considera- since the mid-1990s to about 70 percent of tion to the country's likely sustainable external GDP, more than reversing the decline that debt burden. It also did not discuss the implica- took place in the first half of the 1990s. tions of sending to the board relatively large lending operations that were increasing the debt Is the Bank Effective in Advancing burden. IDA did not identify or formulate the Corporate Advocacy Priorities and "trigger" for an "exit" or the orderly reduction in Harnessing Global Knowledge? its risk exposure. As a result, the country While emphasizing the importance of country required access to HIPC debt relief. priorities and country fit in the design of Bank Shortcomings in DSA may also be reflected assistance, the Bank has also identified a few in weak policy indicators and rapidly areas--"corporate advocacy priorities"--where mounting debt levels in many developing performance has been weak and which the 3 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Bank believes to be "under-represented" in at good government and a competent civil many countries' development agendas. Three service. By encouraging open debate, carrying of these are rated and were among the lowest out policy research, and disseminating good rated dimensions of the CPIA in 1999--gender, practice to member countries and international environment, and governance (see figure 1.4). organizations, the Bank helped to raise the Recent evaluations of the Bank's efforts to profile of governance issues on the develop- mainstream these priorities are mixed. Two ment agenda. At the country program level, the OED evaluations found, for example, that the evaluation found that despite Bank support for Bank had not acted as forcefully as it might in highly relevant anti-corruption activities only mainstreaming gender. Moreover, while Bank modest success had been achieved so far. assistance addressed women's/gender issues in There was a gap between relevance and efficacy the human development sectors in some because the Bank's entry into this area is countries, particularly in countries where there relatively recent, the task is complex, and the is gender disparity in education enrollments, it challenge is daunting (OED 2003g). did not address the problems of increasing the The evaluation was cautious about the economic participation of women or of prospects for effective Bank engagement in strengthening the institutional framework for anti-corruption activities, noting that corrup- gender issues. These evaluations thus tion is grounded in political contexts and social concluded that Bank carry-through on both the fragmentation over which the Bank has limited letter and spirit of the Bank's gender policy, influence. Therefore, greater reliance on and also the gender results achieved, were at development partners may be warranted in the best mixed (OED 2001b, 2002i). future provided, of course, that borrowers are An evaluation of the Bank's efforts to receptive. mainstream environment was similarly critical Notwithstanding the Bank's mixed perform- of the Bank's efforts. It noted that environmen- ance in these three priority areas, developing tal sustainability was not adequately integrated countries appear to have made progress on into the Bank's core objectives and country each since 1999. Gender has posted the largest assistance strategies, that the linkages between average increase of the 20 CPIA component macroeconomic policy, poverty alleviation, and indicators and now averages 3.7, the highest environmental sustainability were not explicitly among social components of the CPIA. forged, and that the institution's environmen- Transparency, accountability, and corruption in tal efforts had neither been consistent nor held the public sector registered the fourth largest to uniform quality standards. The evaluation increase, and environment the sixth. The traced these shortcomings to fundamental contribution of the Bank's operational support differences of views among member countries to these positive developments, with the about the Bank's role and to a lack of consis- possible exception of anti-corruption, has tent management commitment to environmen- probably been modest. The Bank's advocacy, tal issues (OED 2002h). on the other hand, may have contributed to The OED evaluation of anti-corruption raising the profile of these issues, and thereby initiatives concluded that the Bank had rapidly contributed to the progress that has occurred. integrated corruption concerns into operational activities and internal processes, Best practice, good fit, or fad? and that understanding of the causes and The Bank's creation of networks in 1996 was dynamics of corruption in different country intended, inter alia, to harness the develop- settings has improved. Bank activities over the ment knowledge gained through the Bank's period 1997 to 2002 addressed key factors research and worldwide operations in order to affecting corruption, including state interven- incorporate global state of the art knowledge in tion, rule of law, greater transparency in public its services. To this end, Bank networks are sector operations, and capacity building aimed supposed to cull, on an ongoing basis, from the 3 4 C O U N T R Y F I T: A D A P T I N G B A N K S U P P O R T T O C O U N T R Y C O N D I T I O N S Bank's experience concerning effective The risks of promoting inappropriate processes and designs for different contexts. policies appear to be most pronounced in These are disseminated in various ways, includ- policy areas where progress on reform has been ing Sector Strategy Papers (SSPs),8 internal and difficult and where developed-country policies external websites, and various knowledge (perhaps driven in part by new technology, as is management and learning activities. the case in many infrastructure sectors) are The potential gains from this initiative have evolving rapidly. They are also present in not yet been fully realized. There are no routine "frontier" areas in development thinking, such practices for validating either lessons learned as governance and empowerment. or promising practices. Only a third of fifteen The Bank's experience in supporting private network and regional units surveyed for OED's sector development in power illustrates well knowledge evaluation reported having such some of the risks involved. The Bank's initial practices in place (OED 2003i). Only one of the strategy called for a sequenced process of five units has a process for regularly eliminating commercialization and corporatization of state- outdated lessons or good practice cases; the owned power companies, followed by privatiza- other four report that their materials have no tion as a means to introduce competition and time limits. This undercuts the ability of innovation. This orderly sequence of steps from network and regional knowledge activities, public to private--which was based on experi- including those of thematic groups, to serve as ence in Chile, England, and Wales--proved reliable sources of community knowledge and unworkable in many developing-country poses a risk that the Bank will advocate, or be contexts. Commercialization turned out to be perceived to be advocating, practices that may incompatible with the kinds of political and not turn out to be "best" or even "good"--at social considerations to which many govern- least in all cases (see box 3.7 for an example ments gave higher priority than commercial from the water and sanitation sector). success. N e w P a r a d i g m s : P r i v a t e S e c t o r B o x 3 . 7 P a r t i c i p a t i o n i n W a t e r a n d S a n i t a t i o n Government-provided water and sanitation services proved a developing countries. It took a number of PSP contracts that convenient means for the World Bank to lend to the sector, did not live up to expectations for the Bank to correct this since the Bank's statutory requirement of sovereign guaran- inappropriate use of and excessive faith in the private sector tees was easily satisfied when service providers were as operator. government agencies. Bank lending to the water and sanita- As a consequence of this experience, governments, tion sector began in the late 1960s and expanded rapidly in the international lending agencies including the Bank, and the 1970s. Lending to state providers, however, delivered poor private sector have acquired a more nuanced view of PSP. It is development results and became increasingly difficult in the not a panacea to deep-seated problems and cannot be 1980s as a result of the debt crisis. At the same time, the private expected to substitute for decisions--particularly about tariffs sector alternative for provision of water and sanitation gained and service standards--that only governments have the power prominence following the privatization of these services in and obligation to make. PSP is better likened to a sharp tool. A England and Wales under the Thatcher government. What had capable government can use it to great advantage to improve until then been a phenomenon limited to France and Spain the water supply and sanitation situation, but an inept govern- became a model that was suddenly thought to be worth consid- ment can make matters worse through an injudicious use of ering everywhere. PSP without providing clear quality and price regulation along Based on these developments, the Bank and others quickly with strong and sustained support. adopted "private sector participation" (PSP) approaches as the preferred solution to water and sanitation problems in Source: OED (2003c). 3 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S The Bank's response was to rely increasingly in increases both the opportunity to accelerate its operations on privatization (as well as private development through use of this knowledge participation through management contracts) as a and the risk of making a mistake. It is often meansforachievingcommercialization.Thismodel difficult to reach definitive judgments on what reflected rapid developments in power policy in works and what doesn't work, when, where, the developed world that were reducing the role of and why. With respect to private sector regulation and increasing the role of competition. development in the electric power sector, for The Bank's privatization model--at least as it was example, the OED evaluation notes that contin- widely perceived--began to advocate privatization ually evolving practices make it difficult to strategies based on new developed-country establish convincing theories about the optimal models that unbundled production, transmission, sequencing of reforms and that the catalogue and distribution, and introduced private sector of things to avoid continues to expand. competition where possible. As noted above, the Bank's networks are This strategy was effective in some contexts. In expected to synthesize and communicate "best others, it was clearly a bad fit (box 3.8), and may practice" in development. But there may be an in some cases have retarded the development of inherent conflict between the achievement of the sector. OEDs evaluation concludes that this objective and the Bank's efforts to improve despite widespread adoption of the many the country-fit of its services. In a survey variants of this concept in a wide range of conducted for the knowledge sharing evalua- industrial and developing countries, power sector tion, 121 knowledgeable persons in five client reform remains a work in progress (OED 2003h). countries rated the technical soundness of the Bank's knowledge activities highly but were The Bank has done a good job in aggregating less positive about how well the Bank's knowledge, but has been less successful in knowledge is applied in individual country fostering adaptation to country contexts. circumstances. The majority of respondents The explosion of development knowledge that from all countries, professions, and sectors has resulted from the Bank's and others' efforts agreed that the information provided by the E l e c t r i c i t y D e v e l o p m e n t i n U k r a i n e : B o x 3 . 8 G o o d I d e a , B a d F i t The Electricity Market Development Loan to Ukraine, approved cancelled at government request in 1999 due to the impact of the in 1997, aimed at supporting critical reforms in the power Russian financial crisis on the Ukrainian economy. sector--improved collections, access to working capital, A key lesson from the project is that there is little merit in ability to meter , and financial management--leading to the pursuing comprehensive power sector reform policies development of a competitive power pool based on the British (legislation, regulation, unbundling, competition, privatization, model of unbundling. The reforms were expected to increase regulation) in a country suffering a major economic crisis. In the quality and reduce the cost of electricity supply by develop- an economy that was barter based, with salaries and pensions ing a competitive electric power market and operating in arrears, and where the government condoned the culture of conditions that would encourage electric power companies to nonpayment, there was no way to make consumers pay for seek full cost recovery. But political interference prevented any electricity in cash. In such an environment, the attempt to improvement in payment collections; in fact, collection levels introduce a competitive power market was bound to fail. declined, preventing the generating companies from recovering Project objectives should have been more modest and targeted their costs. The loan was suspended shortly after approval due to improving well-delineated technical, institutional, and both to the unsatisfactory financial performance of the entire financial problems. power sector and to a new government prohibition on increased electricity tariffs for household consumers. The loan was Source: OED (2003h). 3 6 C O U N T R Y F I T: A D A P T I N G B A N K S U P P O R T T O C O U N T R Y C O N D I T I O N S World Bank is of unquestionably high quality-- solving economic and social problems in their as long as quality is defined strictly in terms of country. Several said that the Bank's insistence technical soundness and sophistication. that its approach is the only correct approach But the majority of respondents also generates mistrust and suspicion of the Bank and complained that the Bank is too narrowly substantially decreases receptivity to Bank focused in the analyses and "best practices" that information overall. Most respondents suggest it presents, with little or no attention to alterna- that increased use of local expertise as well as tive perspectives or to individual country circum- more local involvement in generating knowledge stances. Many respondents expressed frustration would improve the alignment of Bank about the Bank's insistence that its models and knowledge with country circumstances (OED solutions represent the only viable approach to 2003i). 3 7 4 Conclusions The Bank Has Been Reasonably Good at knows that reform is under way. Country Aligning Its Support with Countries That assistance programs in such countries initially Are Improving Their Policies could emphasize analytical work, dialogue, and Recent Bank support has generally been associ- technical assistance, and modulate lending very ated with improving policy environments. carefully. Premature lending may not only lack Evaluation evidence suggests that in many development impact, but it can also weaken the cases the Bank's support has contributed to linkage to policies conducive to productive use these reforms. of resources and access to development finance. The environment for policy reform may have In non-crisis/non-emergency situations, adjust- been unusually favorable over the last several ment lending should be provided only after a years. The large number of countries in transi- solid track record of reform implementation has tion, some on the path to European Union been established. accession, combined with a large number of countries emerging from conflict or other The Bank's Record in Customizing Its crisis, presented an environment that was Support for Reform to Client Needs Is favorable for change--particularly given the Mixed low starting points of many of these countries. The Bank has followed diverse approaches The Bank recognized these opportunities and, across countries, and over time within in many cases, provided effective development countries. Although the Bank's strategies have assistance. been broadly appropriate in context, there is Bank support for policy reform was less significant room for improvement in customiz- successful in countries where there was no ing Bank support to country conditions. favorable trend to reinforce. Triggers and Where Bank programs have not been well conditionality helped keep support aligned with aligned with country conditions, there has reform performance where the environment for been a substantial risk of failure even when the reform was stable--whether favorable or ownership environment was otherwise unfavorable. These tools have been less effective favorable for Bank support. when the environment for reform, after showing The Bank has provided effective support for some promise, had begun to deteriorate. policy reform through a wide range of instru- The overall lesson that emerges is not new: in ments and instrument mixes depending upon countries with poor policy environments, the country context. Although adjustment limited lending is appropriate before the Bank lending has played an important role in many 3 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S countries that have undertaken reforms over knowledge has not yet been harnessed in the last few years, its effectiveness is mainly sector strategies that are operationally useful at attributable to underlying circumstances in the the country program level. The Sector Boards countries that favored reform. The new instru- have not exercised adequate oversight over the ments created by the Bank to provide more process by which promising new developments flexibility to learn and modify operations in become transformed into "best practices." The implementation are also showing positive risk of inappropriate dissemination is returns in early evaluation results. potentially serious in areas where thinking on Corporate advocacy activities have been less policy is evolving very rapidly. Networks and effective in fostering change because they regions should strengthen their respective generally were not customized enough to procedures in the capture, validation, and generate good uptake at the country level. application of lessons learned and good While early uptake has been good with regard practices. to the Bank's efforts to mainstream anti-corrup- tion work, earlier efforts in the areas of gender Debt sustainability analysis needs upgrading. and environment were not satisfactory. Gender In some countries, the Bank has provided and environment sector strategies developed support in the context of macroeconomic subsequent to these OED evaluation findings frameworks that were not sufficiently robust addressed the implementation weaknesses with regard to risk--and thus contributed to found in the evaluations. Given the potential the emergence of unsustainable debt. The debt tensions between Bank and country driven sustainability analysis that has been routinely priorities, however, effectiveness in promoting carried out in the context of the design of Bank uptake of the Bank's corporate priorities should programs has not always adequately consid- be kept under continuing review. ered the sensitivity of debt accumulation to key uncertain parameters. The Bank's DSA needs "Best practice" is risky and sometimes to be upgraded to require better ex ante inappropriate. analysis involving more critical examination of ESW is a critical element in a successful instru- the realism of assumptions and examination of ment mix. ESW has frequently been found to the sensitivity of the analysis to key assump- be deficient in countries where outcomes of tions. In addition, the Bank's monitoring Bank assistance are not satisfactory, and an indicators and CAS triggers should call for important contributing factor in cases of reconsideration and recalibration of the Bank's successful outcome. In non-crisis/non- assistance as necessary to ensure progress emergency situations, the Bank should not toward and/or maintenance of debt sustainabil- engage in lending before ESW has established ity. Caution is indicated in expanding lending an adequate base of country and sector volumes until the Bank's debt sustainability knowledge. The evidence also suggests that analysis has been strengthened. there is considerable unexploited potential in ESW through better adaptation to specific Implications for the Future country contexts and dissemination, especially Maintaining the positive trends of the last few in countries that have access to strong analyti- years will require deepening of reforms. Develop- cal capacity. The Bank should ensure that ment research indicates that while many countries adequate resources and managerial attention have been able to achieve some degree of reform are devoted to realizing this potential. and periodic modest growth, only a few--those Looking beyond ESW to the Bank's able to sustain and deepen the development of knowledge resources more generally, the Bank supporting policies and institutions--have been has not been effective in bringing its able to achieve the long periods of sustained knowledge to bear on specific country growth needed to reduce poverty substantially contexts. The potential embodied in this and sustainably (Rodrik 2003b). 4 0 C O N C L U S I O N S Tension can occur between support of policy of results. These include new lending instru- reform and CDF principles. ments such as APLs and PRSCs as well as The instruments and processes that the Bank ongoing refinements in the use of triggers and currently uses to link its support to policy conditionality in traditional instruments. In reform are in some tension with its efforts to these mechanisms, support is usually linked to promote country-led partnership approaches to completion of intermediate steps such as legal development. OED's Comprehensive Develop- and regulatory changes, organizational ment Framework evaluation identifies tensions improvements, and expenditure or other input created by the PRSP's multiple roles--as a targets. This linkage to intermediate steps-- process for promoting country leadership of the while relatively practical to monitor--has one reform agenda, as a condition of debt reduction main weakness: it does not address the under HIPC, and as a basis for access to Bank "qualitative" dimensions--"due diligence" or and Fund finance (OED 2003j). Policy research "best effort" in implementing objectives of the on conditionality shows that there are also development program.1 Although strong tensions between CAS triggers and adjustment county ownership would mitigate this lending conditionality on the one hand, to the problem, it might not eliminate it. extent that these derive from objectives that are The Bank could begin to experiment with not country-owned, and the development of additional mechanisms to introduce a more country ownership of the development agenda direct linkage between its support and results in on the other. Finally, there are potential tensions terms of poverty reduction or of other MDGs between the use of policy indicators like the that would capture, at least in part, some of the CPIA--which are based implicitly on the qualitative dimensions of implementation. Such premise that their underlying criteria reflect a link would be added onto or blended with good policies at all times and in all places--and existing mechanisms. Countries that are deliver- the findings of evaluation and policy research ing good results in terms of poverty reduction that good policies have context specific would receive more support, other things being elements (White 2001; Rodrik 2002). equal, than countries that are not delivering The Bank is endeavoring to address these results. This approach could, in principle, tensions and tradeoffs through better alignment sharpen the results focus, provide countries of Bank CASs and operational conditionality with latitude during implementation in pursuing PRSPs (e.g., through medium-term programmatic their objectives, and provide flexibility during lending support of PRSP measures). The Bank is implementation to accommodate the uncertain- working to strengthen the country focus of its ties associated with policy reform. It would also support by differentiating its strategies by country provide additional incentives for due diligence characteristics (e.g., between LICUS countries and in pursuit of results. strongly performing middle-income countries), The Bank should promote pilots in willing decentralizing its operations, and increasing its countries using hybrid approaches with the reliance on borrowing-country participation in potential to increase aid effectiveness. The conducting analytical work. It applies the CPIA challenge facing the Bank going forward is to flexibly in decision-making. It is too early to tell, continually re-examine its approaches for however, whether these efforts are paying off in delivering development support to find modifi- terms of improved development effectiveness. cations and/or new approaches that yield the best possible bottom line results in terms of The Bank should explore approaches to sustainable poverty reduction. strengthen the link between Bank support for reform and development results. The Bank has a number of mechanisms designed to link its support to the achievement 4 1 APPENDIX APPENDIX: PROJECT PERFORMANCE RESULTS This Appendix presents long-term trends in Completion Report (ICR) Reviews, previously project performance based on evaluations called "Evaluation Summaries" (ES), and 63 conducted by OED. Consistent with past Project Performance Assessment Reports ARDEs, the Appendix uses the year 1990 as a (PPARs).2, This newly evaluated cohort consists 3 starting point in analyzing long-term trends in of 56 adjustment operations and 266 investment project performance. Data for the statistical operations; figure A.1 shows the evaluated tables and charts are taken from the Business projects by when they were approved. Warehouse. The data in this report were "frozen" on December 31, 2003. Performance Trends Composition of the ARDE 2003 Exiting Outcome Cohort Exit year FY02 marks the third year in which OED has evaluated 322 closed projects since project performance exceeded the Strategic ARDE 2002, 80 percent of which exited the Compact target of 75 percent satisfactory Bank's portfolio during FY02 and FY03.1 These outcomes.4 As figure A.2 illustrates, 79 percent evaluations cover US$30.4 billion in disburse- of projects in the FY02 exit cohort had satisfac- ments and consist of 266 Implementation tory outcomes. F i g u r e A . 1 E v a l u a t e d P r o j e c t s b y A p p r o v a l Y e a r s Number of Evaluated Projects 60 Adjustment 50 Investment 40 30 20 10 0 Pre - 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003* 1991 Approval fiscal year * Partial. Source: Business Warehouse, World Bank 2003. 4 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S F i g u r e A . 2 L o n g - T e r m T r e n d s i n P r o j e c t P e r f o r m a n c e Percentage Satisfactory Outcome 10 0 80 60 Weighted by disbursement By project 40 20 FY90 FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 FY00 FY01 FY02 FY03* * Partial (dashed line). Source: Business Warehouse, World Bank 2003. As of December 31, 2003, OED has evaluated better. As with outcome ratings for the FY03 44 percent of the 293 FY03 exits. For this partial (partial) cohort, it is too soon to tell whether FY03 cohort,5 outcomes are lower than in the these apparent departures from recent trends recent past (figure A.2). Outcome is rated will be borne out and, if so, whether they satisfactory for 72 percent of projects (for 68 represent significant changes in Bank project percent weighted by disbursements). This effectiveness (figure A.3). represents a decline with respect to outcomes in recent previous years, although it is unclear Regional Performance whether the decline will be borne out when the Figure A.4 reports the percentage of satisfac- evaluation results for the full 2003 cohort are tory project outcomes, weighted by disburse- available and, if so, whether this reflects a ment, for the FY99-03 (partial) cohort systematic departure from recent trends compared with the FY94-98 cohort for each toward improved results. region. The Latin American and Caribbean Region (LCR) and East Asia and Pacific (EAP) Sustainability and Institutional Development Region are the top performers for both Impact cohorts. The percentages of satisfactory Eighty-four percent (weighted by disburse- outcomes for both LCR and EAP Regions ments) of the FY02 exiting cohort are rated exceed the Bank-wide satisfactory outcomes of "likely" or "highly likely" to be resilient to 76 percent for the FY94-98 cohort and of 80 future risks,6 and 51 percent (weighted by percent for the FY99-03 (partial) cohort. disbursements) were determined to have a "substantial" or "high" institutional develop- Sectoral Performance ment impact.7 Both these findings represent Compared with FY94-98 exits, the outcome modest increases over the FY01 exit cohort. performance weighted by disbursement for the For the FY03 (partial) cohort, sustainability FY99-03 (partial) exits improved in 9 of the 15 ratings weighted by disbursements decreased Sector Boards--education, energy and mining, to 70 percent "likely" or "highly likely" to be financial, global information and communica- sustainable. Institutional development impact, tions technology, public sector governance, however, increased to 54 percent rated rural development, transport, urban develop- (disbursement weighted) "substantial" or ment, and water supply and sanitation (table 4 4 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S L o n g - T e r m T r e n d s i n S u s t a i n a b i l i t y a n d F i g u r e A . 3 I n s t i t u t i o n a l D e v e l o p m e n t Sustainability Institutional Development Percentage Likely or Better Percentage Substantial or Better 100 100 Weighted by disbursement Weighted by disbursement By project By project 80 80 60 60 40 40 20 20 FY90 FY92 FY94 FY96 FY98 FY00 FY02 FY90 FY92 FY94 FY96 FY98 FY00 FY02 Note: FY03 is partial (dashed line). Source: Business Warehouse, World Bank 2003. F i g u r e A . 4 P r o j e c t O u t c o m e s b y R e g i o n EAP Distribution ofDisbursements by Region (FY94­03*) LCR Bankwide AFR SAR 14% 26% ECA EAP SAR MNA 24% 6% MNA LC R ECA 15% 15% AFR 40 50 60 70 80 90 100 % satisfactory outcome (weighted by disbursement) F Y94­98 exits F Y99­03*exits Active portfolio "not at risk" * Partial. Source: Business Warehouse, World Bank 2003. A.3 and figure A.5). The largest increases in ment, environment, economic policy, and outcome ratings were in the financial, water supply and sanitation sectors remain education, and public sector and governance below the Bank-wide average for FY99-03. sectors. The largest declines in performance The reasons for improvement and deteriora- were in the social protection; health, nutrition, tion differ from sector to sector, and perform- and population (HNP); and economic policy ance during any period is frequently highly sectors. The HNP, energy and resource manage- sensitive to the outcomes of a few large loans. 4 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S In the economic policy sector, for example, the tory in FY00 exits, then dipping to 78 percent decline in outcome ratings reflects the results satisfactory for FY01 exits, outcome ratings of a few large risky loans undertaken in rebounded again to 88 percent satisfactory for macroeconomic crises in Argentina (a FY02 exits (figure A.6). The low disbursement- structural adjustment loan, or SAL, and the weighted performance at 53 percent for FY03 Special Repurchase Loan) and Russia (SAL III). (partial) exits is largely accounted for by an In the social protection sector, most unsatisfac- unsatisfactory large adjustment loan of $2.5 tory projects in the period FY99-03 were identi- billion to Argentina. fied as having weaknesses in design such as The percentage of satisfactory outcomes by underestimation of the capacity and political disbursement for investment lending support to the implementing agency; too much operations has improved consistently since exit complexity; lack of clarity or consensus on the FY00, reaching 84 percent for the FY02 cohort. main design aspects; and weak monitoring and FY03 (partial) exits, however, show a slight evaluation systems. decline to 82 percent (disbursement weighted). Lending Instrument Performance New Lending Instruments Outcomes of adjustment lending operations Forty-one operations employing four new have been volatile in recent years. Having lending instruments (see box A.1 for descrip- reached a record high of 96 percent satisfac- tions of the new lending instruments) were F i g u r e A . 5 P r o j e c t O u t c o m e s b y S e c t o r B o a r d Distribution of Disbursements SD by Sector Board (FY94­03*) GCI PSG WSS UD 4% Tran 4% FS EP Tran 13% Edu SP 11% Edu SD4% 7% SP 0% E&M PSD RS 17% 13% UD PSG Bankwide Avg 6% FS 12% Env RS PSD 1% 3% HNP GCI HNP 4% 1% E&M Env Note: The Sector classification provides a single rating for the en- tire project rather than rating individual project components that EP cover particular sectors or themes. The Sector Board classifica- tion applies to the whole project and enables outcomes to be WSS matched to it. EP = economic policy; Edu = education; E&M = en- ergy and mining; Env = environment; FS = financial sector; GCI = 40 50 60 70 80 90 100 global information/communication technology; HNP = health, nu- trition, and population; PSD = private sector development; PSG = % satisfactory outcome (weighted by disbursement) public sector governance; RS = rural sector; SD = social develop- F Y94­98 exits F Y99­03* exits Active portfolio "not at risk" ment; SP = social protection; Tran = transpot; UD = urban devel- * Partial. opment; WSS = water supply and sanitation. SD Sector Board did Source: Business Warehouse, World Bank 2003. not exist before FY98. 4 6 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S L o n g - T e r m T r e n d s i n A d j u s t m e n t a n d F i g u r e A . 6 I n v e s t m e n t L e n d i n g Adjustment Investment Percentage Satisfactory Outcome Percentage Satisfactory Outcome 100 100 90 90 80 80 70 70 60 60 50 Weighted by disbursement 50 Weighted by disbursement 40 By project 40 By project 30 30 20 20 FY90 FY92 FY94 FY96 FY98 FY00 FY02 FY90 FY92 FY94 FY96 FY98 FY00 FY02 Note: FY03 is partial (dashed line). Source: Business Warehouse, World Bank 2003. evaluated by OED as of December 2003.8 Of operations had satisfactory outcomes. One of these, 81 percent had satisfactory outcomes every two projects had substantial or better (figure A.7). Weighted by disbursements, the institutional development impact. About 70 new lending instruments achieved a very high percent were rated as having likely or highly level of 98 percent satisfactory outcomes. likely resilience to future risks. Eighty-six percent of the new instrument projects were rated to have likely or highly likely Conclusions resilience to future risks. Sixty-five percent of Project performance continued to meet the projects were assigned substantial or better corporate targets through FY02. The picture institutional development impact ratings. for FY03 is as yet uncertain. The performance of Sector Boards is varied. The evaluations of Bank Managed Special Programs operations funded by new lending instruments OED has evaluated 70 operations financed so far indicate a strong performance but do not under four Bank-managed special programs constitute a large enough sample to make any that have exited the Bank's portfolio since FY95 reliable analysis of performance of the new (box A.2). Eighty-seven percent of the special lending instruments as a whole. 4 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S B o x A . 1 T h e B a n k ' s N e w L e n d i n g I n s t r u m e n t s The Bank has five new lending instruments: Adaptable Pro- port for a medium-term government program of policy reforms gram Loans and Learning and Innovation Loans are investment and institution building. lending instruments. Programmatic Structural Adjustment Loans, ThePovertyReductionSupportCredit(PRC)programcon- Poverty Reduction Support Credits, and Special Structural Ad- sists of a series of operations, typically two or three, which together justment Lending are adjustment lending instruments. support IDA countries' medium-term policy and institutional reform AdaptableProgramLoans(APLs)provide phased support programs to help implement their poverty reduction strategies. Its for long-term development programs. They involve a series of specific structure depends on country circumstances, including the loans that build on the lessons learned from the previous loan(s) objectives and nature of the country's reform program that it sup- in the series. ports and the timing of the requirement for assistance. LearningandInnovationLoans(LILs)support small pilot- Special Structural Adjustment Loans (SSLs) support type investment and capacity-building projects that, if suc- structural and social reforms by creditworthy borrowers ap- cessful, could lead to larger projects that would mainstream the proaching a possible crisis, or already in crisis, and with ex- learning and results of the LIL. ceptional external financing needs. These loans help countries ProgrammaticStructuralAdjustmentLoans(PSLs)are to prevent a crisis or, if one occurs, to mitigate its adverse eco- provided in the context of a multi-year framework of phased sup- nomic and social impacts. O u t c o m e , S u s t a i n a b i l i t y , a n d I n s t i t u t i o n a l F i g u r e A . 7 D e v e l o p m e n t I m p a c t o f N e w L e n d i n g I n s t r u m e n t s Outcome Sustainability Percentage Satisfactory Outcome Percentage Likely or Better 100 100 80 80 60 60 40 40 20 20 0 0 All Investment Adjustment All Investment Adjustment Institutional Development Impact Percentage Substantial or Better 100 80 By project Weighted by disbursements 60 40 20 0 All Investment Adjustment Source: Business Warehouse, December 31, 2003. 4 8 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S B o x A . 2 B a n k - M a n a g e d S p e c i a l P r o g r a m s This year ARDE begins to report on the evaluation of operations RainforestInitiative(RAIN).The RAIN was established in financed under special programs. Seventy such operations ex- 1992 out of concern about the deforestation of Brazil's humid rain ited the Bank's portfolio during the FY95-03 period. forests in the Amazon and on the Atlantic coast. Its purpose is Global Environment Facility (GEF). The GEF was estab- to demonstrate ways towards conservation and sustainable lished in 1991 as a pilot program to assist in the protection of the use of the natural resources of the rain forests. Overall, 12 proj- global environment and to promote environmentally sound and ects have been approved for a total commitment of US$54.5 sustainable economic development. Overall, 325 projects have million. been approved, for a total commitment of US$4.6 billion. SpecialFinancingGrants(SPF).The SPF grants cover spe- Montreal Protocol Fund (MONT).The World Bank is one cial emergency assistance provided to recent post-conflict of four implementing agencies (along with UNDP, UNIDO, and countries such as Bosnia-Herzegovina, East Timor, Kosovo, UNEP) for the Multilateral Fund for the Implementation of the and West Bank and Gaza. Overall, 80 special financing op- Montreal Protocol to reduce ozone-deleting substances. Over- erations have been approved for a total commitment of US$1.7 all, 38 projects have been approved under this fund, for a total billion. commitment of US$895 million. Performance of Bank-Managed Special Programs No. of ID Impact Special Program Evaluated Outcome Sustainability % Substantial Type Programs % Satisfactory % Likely or Better or Better GEF 42 83 63 60 MONT 6 100 100 50 RAIN 1 100 100 0 SPF 21 90 71 52 Overall Result 49 87 69 55 4 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S O u t c o m e , S u s t a i n a b i l i t y , a n d I n s t i t u t i o n a l T a b l e A . 1 D e v e l o p m e n t ( I D ) I m p a c t b y V a r i o u s D i m e n s i o n s , b y P r o j e c t , F Y 9 4 ­ 0 3 a E x i t s Exit FY94­98 Sustainability ID impact Number of Share Outcome % likely % substantial Number of Share projects % % satisfactory or better or better projects % Sector Board Economic Policy 116 9.2 73.9 53.1 29.5 59 4.8 Education 100 7.9 72.4 43.3 28.6 122 10.0 Energy and Mining 190 15.1 61.7 51.3 34.0 128 10.5 Environment 17 1.3 70.6 58.8 23.5 74 6.1 Financial Sector 72 5.7 62.3 46.4 36.8 65 5.3 Global Information/ Communications Technology 26 2.1 84.6 76.9 46.2 12 1.0 Health, Nutrition, and Population 60 4.8 66.7 55.0 25.0 99 8.1 Private Sector Development 33 2.6 81.8 63.6 34.4 69 5.6 Public Sector Governance 65 5.2 63.1 46.9 30.8 95 7.8 Rural Sector 301 23.9 68.1 42.5 37.8 182 14.9 Social Development N/A N/A N/A N/A N/A 2 0.2 Social Protection 35 2.8 82.9 25.7 42.9 78 6.4 Transport 129 10.2 78.9 54.7 35.9 126 10.3 Urban Development 63 5.0 66.1 38.7 21.0 59 4.8 Water Supply and Sanitation 55 4.4 53.7 33.3 22.2 52 4.3 Overall Result 1,262 100.0 68.9 47.6 33.0 1,222 100.0 Lending Instrument Type Adjustment 199 15.8 77.8 59.4 36.8 150 12.3 Investment 1,062 84.2 67.2 45.3 32.3 1,071 87.6 Not Assigned 1 0.1 100.0 100.0 0.0 1 0.1 Overall Result 1,262 100.0 68.9 47.6 33.0 1,222 100.0 Network ESSD 318 25.2 68.3 43.4 37.0 258 21.1 FSE 72 5.7 62.3 46.4 36.8 65 5.3 HDN 195 15.5 72.5 43.8 30.1 299 24.5 INF 115 9.1 72.3 56.3 41.1 377 30.9 PREM 181 14.3 70.0 50.8 29.9 154 12.6 PSDN 29 2.3 79.3 62.1 35.7 69 5.6 PSI 352 27.9 66.3 48.1 29.1 0 0.0 Overall Result 1,262 100.0 68.9 47.6 33.0 1,222 100.0 5 0 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S Exit FY99­03a Exit FY94­03a Sustainability ID impact Sustainability ID impact Outcome % likely % substantial Number of Share Outcome % likely % substantial % satisfactory or better or better projects % % satisfactory or better or better 72.9 74.1 33.9 175 7.0 73.6 59.9 31.0 83.6 74.1 45.1 222 8.9 78.6 59.8 37.7 70.9 66.9 48.8 318 12.8 65.4 57.4 40.0 74.3 75.0 56.8 91 3.7 73.6 71.9 50.5 75.4 79.7 55.4 137 5.5 68.7 61.7 45.9 100.0 100.0 58.3 38 1.5 89.5 83.8 50.0 67.7 65.9 43.8 159 6.4 67.3 61.6 36.5 61.2 66.7 36.9 102 4.1 68.0 65.6 36.1 87.1 84.7 61.3 160 6.4 77.2 68.5 48.7 66.5 56.0 40.2 483 19.4 67.5 47.4 38.7 100.0 100.0 100.0 2 0.1 100.0 100.0 100.0 83.3 70.3 44.9 113 4.5 83.2 54.5 44.2 86.9 74.8 67.2 255 10.3 82.8 64.2 51.2 71.2 57.1 30.5 122 4.9 68.6 47.5 25.6 65.4 49.0 34.6 107 4.3 59.4 40.8 28.3 75.1 68.9 47.5 2,484 100.0 71.9 57.7 40.1 82.7 80.9 49.3 349 14.0 79.9 68.2 42.3 74.0 67.2 47.2 2,133 85.9 70.6 55.9 39.8 100.0 100.0 0.0 2 0.1 100.0 100.0 0.0 75.1 68.9 47.5 2,484 100.0 71.9 57.7 40.1 69.0 62.1 45.5 576 23.2 68.6 51.5 40.8 75.4 79.7 55.4 137 5.5 68.7 61.7 45.9 78.4 70.4 44.6 494 19.9 76.1 59.3 38.9 76.3 66.5 50.3 492 19.8 75.4 64.0 48.1 81.6 80.6 50.7 335 13.5 75.3 63.9 39.5 61.2 66.7 36.9 98 3.9 66.7 65.1 36.6 N/A N/A N/A 352 14.2 66.3 48.1 29.1 75.1 68.9 47.5 2,484 100.0 71.9 57.7 40.1 (Table continues on the following page.) 5 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S T a b l e A . 1 ( c o n t i n u e d ) Exit FY94­98 Sustainability ID impact Number of Share Outcome % likely % substantial Number of Share projects % % satisfactory or better or better projects % Region Africa 410 32.5 57.0 30.5 25.5 314 25.7 East Asia and Pacific 199 15.8 82.2 66.0 42.1 189 15.5 Europe and Central Asia 132 10.5 72.9 61.7 42.2 259 21.2 Latin America and Caribbean 236 18.7 79.1 56.1 40.8 248 20.3 Middle East and North Africa 113 9.0 65.5 45.5 25.0 95 7.8 South Asia 172 13.6 67.3 46.5 28.2 117 9.6 Overall Result 1,262 100.0 68.9 47.6 33.0 1,222 100.0 Income Group High income: non-OECD 12 1.0 81.8 72.7 27.3 6 0.5 High income: OECD 19 1.5 84.2 73.7 44.4 10 0.8 Low income 654 51.8 61.7 35.8 29.1 561 45.9 Lower middle income 420 33.3 73.1 54.9 34.5 458 37.5 Upper middle income 153 12.1 83.9 73.8 45.0 179 14.6 Not assigned 4 0.3 50.0 25.0 25.0 8 0.7 Overall Result 1,262 100.0 68.9 47.6 33.0 1,222 100.0 Notes: Exit FY denotes the year in which the project leaves the World Bank's active portfolio, normally at the end of disbursements. Percents exclude projects not rated. In July 2000, the rating scale for sustainability was changed from a 3-point scale (Likely, Uncertain, Unlikely) to a 4-point scale (Highly Likely, Likely, Unlikely, Highly Unlikely), with the new scale used in rating projects exits between FY00 and FY02. To monitor the effects of this change, OED continued using the old 3-point scale in parallel with the new until May 2003. a. The data for FY03 exits represent a partial lending sample (130 out of 293) and reflects all OED project evaluations posted to the Business Warehouse as of December 31, 2003. The processing of the remainder of the FY03 exits is ongoing and is expected to be completed by the end of FY04. Source: Business Warehouse, World Bank 2003. Income group designations are taken from World Development Indicators 2002. 5 2 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S Exit FY99­03a Exit FY94­03a Sustainability ID impact Sustainability ID impact Outcome % likely % substantial Number of Share Outcome % likely % substantial % satisfactory or better or better projects % % satisfactory or better or better 60.8 50.9 37.3 724 29.1 58.7 39.0 30.6 80.9 70.5 48.9 388 15.6 81.6 68.1 45.5 82.4 79.8 54.3 391 15.7 79.2 73.4 50.3 79.9 77.1 55.2 484 19.5 79.5 66.4 48.2 74.2 69.0 41.9 208 8.4 69.4 55.6 32.7 78.6 74.8 46.2 289 11.6 71.9 57.4 35.5 75.1 68.9 47.5 2,484 100.0 71.9 57.7 40.1 66.7 83.3 66.7 18 0.7 76.5 76.5 41.2 90.0 90.0 60.0 29 1.2 86.2 79.3 50.0 66.7 56.0 37.3 1,215 48.9 64.0 44.7 32.9 84.1 80.6 55.7 878 35.3 79.3 67.8 45.5 78.5 77.0 57.0 332 13.4 81.0 75.5 51.4 62.5 71.4 62.5 12 0.5 58.3 54.5 50.0 75.1 68.9 47.5 2,484 100.0 71.9 57.7 40.1 5 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S O u t c o m e , S u s t a i n a b i l i t y , I n s t i t u t i o n a l D e v e l o p m e n t ( I D ) I m p a c t b y V a r i o u s T a b l e A . 2 D i m e n s i o n s , W e i g h t e d b y D i s b u r s e m e n t , F Y 9 4 ­ 0 3 a E x i t s Exit FY94­98 Sustainability ID impact Disbursement Share Outcome % likely % substantial Disbursement Share $ millions % % satisfactory or better or better $ millions % Sector Board Economic Policy 13,249 13.4 71.7 63.7 26.6 12,172 13.1 Education 5,166 5.2 74.6 46.5 32.6 7,345 7.9 Energy and Mining 18,953 19.1 68.2 61.1 37.1 13,259 14.3 Environment 349 0.4 73.8 59.5 28.7 2,005 2.2 Financial Sector 14,755 14.9 78.4 70.7 43.2 7,485 8.1 Global Information/ Communications Technology 1,369 1.4 93.6 95.2 69.6 1,067 1.2 Health, Nutrition and Population 2,745 2.8 84.3 71.5 29.7 5,470 5.9 Private Sector Development 2,373 2.4 88.1 66.2 40.5 3,543 3.8 Public Sector Governance 2,907 2.9 80.5 64.1 44.2 7,643 8.2 Rural Sector 15,575 15.7 75.3 52.5 44.3 9,596 10.3 Social Development N/A N/A N/A N/A N/A 30 0.0 Social Protection 2,098 2.1 95.6 60.1 56.6 5,943 6.4 Transport 10,791 10.9 87.0 57.0 37.8 10,986 11.8 Urban Development 4,615 4.7 79.7 51.1 17.3 3,092 3.3 Water Supply and Sanitation 4,059 4.1 54.8 22.5 15.0 3,127 3.4 Overall Result 99,004 100.0 75.9 59.2 36.5 92,764 100.0 Lending Instrument Type Adjustment 33,968 34.3 79.2 70.6 39.0 33,438 36.0 Investment 64,979 65.6 74.2 53.2 35.4 59,326 64.0 Not Assigned 57 0.1 100.0 100.0 0.0 0 0.0 Overall Result 99,004 100.0 75.9 59.2 36.5 92,764 100.0 Network ESSD 15,924 16.1 75.3 52.7 44.0 11,631 12.5 FSE 14,755 14.9 78.4 70.7 43.2 7,485 8.1 HDN 10,010 10.1 81.6 56.2 36.8 18,758 20.2 INF 9,858 10.0 77.1 62.0 38.8 31,532 34.0 PREM 16,156 16.3 73.3 63.8 29.9 19,815 21.4 PSDN 2,188 2.2 87.1 68.4 41.8 3,543 3.8 PSI 30,113 30.4 73.4 54.0 32.2 0 0.0 Overall Result 99,004 100.0 75.9 59.2 36.5 92,764 100.0 5 4 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S Exit FY99­03a Exit FY94­03a Sustainability ID impact Sustainability ID impact Outcome % likely % substantial Disbursement Share Outcome % likely % substantial % satisfactory or better or better $ millions % % satisfactory or better or better 65.0 58.5 35.0 25,421 13.3 68.5 61.3 30.7 87.2 86.1 48.0 12,511 6.5 82.0 69.2 41.7 71.1 67.7 53.6 32,212 16.8 69.4 63.8 43.9 70.7 80.6 45.7 2,355 1.2 71.2 77.4 43.2 90.1 94.6 74.8 22,240 11.6 82.3 77.2 55.5 100.0 100.0 53.2 2,437 1.3 96.4 97.3 62.4 77.5 79.2 59.7 8,216 4.3 79.8 76.5 49.7 84.1 84.0 59.6 5,916 3.1 85.7 76.4 51.5 94.5 95.4 56.3 10,550 5.5 90.6 85.9 53.0 79.6 69.1 50.7 25,171 13.1 77.0 58.7 46.8 100.0 100.0 100.0 30 0.0 100.0 100.0 100.0 84.7 81.7 38.1 8,041 4.2 87.6 75.7 42.9 90.4 85.6 67.4 21,777 11.4 88.7 70.6 52.7 82.7 73.7 32.7 7,707 4.0 80.9 59.9 23.5 61.2 49.5 31.7 7,186 3.7 57.6 34.0 22.3 80.3 76.5 51.9 191,768 100.0 78.0 67.3 44.1 82.1 78.1 51.5 67,406 35.1 80.6 74.1 45.5 79.3 75.6 52.2 124,305 64.8 76.6 63.6 43.4 N/A N/A N/A 57 0.0 100.0 100.0 0.0 80.3 76.5 51.9 191,768 100.0 78.0 67.3 44.1 78.1 71.2 49.9 27,556 14.4 76.5 60.4 46.5 90.1 94.6 74.8 22,240 11.6 82.3 77.2 55.5 83.6 82.7 48.3 28,768 15.0 82.9 73.1 44.3 78.9 73.5 54.1 41,390 21.6 78.5 70.6 50.5 76.4 71.9 43.2 35,971 18.8 75.0 68.1 37.3 84.1 84.0 59.6 5,731 3.0 85.3 77.6 52.4 N/A N/A N/A 30,113 15.7 73.4 54.0 32.2 80.3 76.5 51.9 191,768 100.0 78.0 67.3 44.1 (Table continues on the following page.) 5 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S T a b l e A . 2 ( c o n t i n u e d ) Exit FY94­98 Sustainability ID impact Disbursement Share Outcome % likely % substantial Disbursement Share $ millions % % satisfactory or better or better $ millions % Region Africa 15,770 15.9 67.6 35.8 29.5 10,206 11.0 East Asia and Pacific 23,714 24.0 86.7 76.5 40.9 22,817 24.6 Europe and Central Asia 13,444 13.6 66.9 72.8 40.8 16,149 17.4 Latin America and Caribbean 22,099 22.3 82.9 61.1 49.1 26,420 28.5 Middle East and North Africa 7,271 7.3 67.3 37.0 19.9 4,672 5.0 South Asia 16,706 16.9 70.2 53.6 24.7 12,500 13.5 Overall Result 99,004 100.0 75.9 59.2 36.5 92,764 100.0 Income Group High income: non-OECD 201 0.2 95.0 91.4 54.2 104 0.1 High income: OECD 5,995 6.1 96.4 95.2 12.5 2,760 3.0 Low income 39,051 39.4 70.5 46.4 29.8 31,223 33.7 Lower middle income 36,951 37.3 74.5 60.5 37.8 41,242 44.5 Upper middle income 16,710 16.9 84.3 73.4 53.4 17,406 18.8 Not assigned 98 0.1 50.8 0.0 0.0 28 0.0 Overall Result 99,004 100.0 75.9 59.2 36.5 92,764 100.0 Notes: Exit FY denotes the year in which the project leaves the World Bank's active portfolio, normally at the end of disbursements. Percents exclude projects not rated. In July 2000, the rating scale for sustainability was changed from a 3-point scale (Likely, Uncertain, Unlikely) to a 4-point scale (Highly Likely, Likely, Unlikely, Highly Unlikely), with the new scale used in rating projects exited between FY00-02. To monitor the effects of this change, OED continued using the old 3-point scale in parallel with the new till May 2003. a. The data for FY03 exits represent a partial lending sample (130 out of 293) and reflects all OED project evaluations posted to the Business Warehouse as of December 31, 2003. The processing of the remainder of the FY03 exits is ongoing, and is expected to be completed by the end of FY04. Source: Business Warehouse, World Bank 2003. Income group designations are taken from World Development Indicators 2002. 5 6 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S Exit FY99­03a Exit FY94­03a Sustainability ID impact Sustainability ID impact Outcome % likely % substantial Disbursement Share Outcome % likely % substantial % satisfactory or better or better $ millions % % satisfactory or better or better 65.4 53.1 34.6 25,977 13.5 66.7 42.4 31.5 89.7 80.9 64.7 46,531 24.3 88.2 78.6 53.4 78.4 80.6 53.5 29,593 15.4 73.2 76.7 47.7 80.9 78.2 51.7 48,519 25.3 81.8 70.2 50.5 76.6 73.3 47.7 11,943 6.2 70.9 50.6 30.8 78.1 80.5 42.9 29,206 15.2 73.6 64.4 32.5 80.3 76.5 51.9 191,768 100.0 78.0 67.3 44.1 77.1 84.3 77.1 305 0.2 88.9 89.0 62.1 95.9 96.7 87.7 8,755 4.6 96.3 95.6 48.6 75.1 63.1 40.2 70,274 36.6 72.6 53.4 34.5 85.8 88.3 60.1 78,193 40.8 80.4 74.6 49.6 74.4 69.2 48.0 34,116 17.8 79.3 71.3 50.7 0.0 20.5 0.0 126 0.1 39.5 4.6 0.0 80.3 76.5 51.9 191,768 100.0 78.0 67.3 44.1 5 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S T a b l e A . 3 A c t i v e P o r t f o l i o : P r o j e c t s N o t a t R i s K Number of Projects Net Commitment % Not % Not No. Share, % at Risk $ millions Share, % at Risk Sector Board Economic Policy 22 2 73 1,678 2 79 Education 149 11 77 8,553 10 80 Energy and Mining 94 7 82 7,846 9 83 Environment 47 4 87 2,053 2 93 Financial Sector 51 4 86 3,632 4 96 Gender and Development 1 0 100 3 0 100 Global Information/ Communications Technology 12 1 100 214 0 100 Health, Nutrition and Population 146 11 78 8,968 10 77 Poverty Reduction 4 0 100 328 0 100 Private Sector Development 68 5 82 2,408 3 74 Public Sector Governance 102 8 77 4,312 5 76 Rural Sector 226 17 84 15,068 17 83 Social Development 27 2 81 1,071 1 95 Social Protection 64 5 77 4,255 5 72 Transport 153 12 86 18,960 21 88 Urban Development 68 5 81 5,015 6 87 Water Supply and Sanitation 86 7 74 5,268 6 74 Overall Result 1,320 100 81 89,635 100 83 Lending Instrument Type Adjustment 47 4 77 7,968 9 80 Investment 1,273 96 81 81,668 91 83 Overall Result 1,320 100 81 89,635 100 83 Network ESSD 300 23 84 18,192 20 85 FSE 51 4 86 3,632 4 96 HDN 359 27 77 21,777 24 77 INF 413 31 82 37,304 42 85 PREM 129 10 78 6,322 7 78 PSDN 68 5 82 2,408 3 74 Overall Result 1,320 100 81 89,635 100 83 Region Africa 337 26 74 16,097 18 73 East Asia and Pacific 215 16 89 20,568 23 92 Europe and Central Asia 268 20 85 13,295 15 87 Latin America and Caribbean 267 20 84 18,100 20 86 5 8 A P P E N D I X : P R O J E C T P E R F O R M A N C E R E S U LT S T a b l e A . 3 ( c o n t i n u e d ) Number of Projects Net Commitment % Not % Not No. Share, % at Risk $ millions Share, % at Risk Middle East and North Africa 93 7 80 3,959 4 84 South Asia 140 11 79 17,616 20 74 Overall Result 1,320 100 81 89,635 100 83 Income Group High income: non-OECD 3 0 100 40 0 100 Low income 652 49 78 41,699 47 81 Lower middle income 513 39 87 35,212 39 91 Upper middle income 144 11 74 12,454 14 67 Not Assigned 8 1 100 231 0 100 Overall Result 1,320 100 81 89,635 100 83 Note: Data as of February 15, 2004. Source: Business Warehouse, World Bank 2003. Income group designations are taken from World Development Indicators 2002. 5 9 ANNEXES ANNEX A: INDICATORS OF ECONOMIC POLICIES Country Policy and Institutional Countries are rated on the current status of Assessment1 their policies in relation to the guidelines and to The Country Policy and Institutional Assess- benchmark countries rated first and provided ment (CPIA) is intended to assesses the quality to staff rating other countries. The guidelines of a country's current policy and institutional direct staff to assess the countries on the basis framework. "Quality," according to the instruc- of their currently observable policies, and not tions for preparing the rating, means "how on the amount of improvement since the conducive that framework is to fostering previous rating nor on intentions for future poverty reduction, sustainable growth and the change, unless the latter are virtually in place. effective use of development assistance." Countries are rated on a scale of "2" (Unsatis- There are 20 broad areas assessed, each with factory) to "5" (Satisfactory), in one-half-point a 5 percent weight in the overall CPIA rating. steps, in each of the 20 areas.2 If a "5" has been Each of the 20 areas is further defined by a sustained for three or more years in an area, the number of criteria that are defined in the rating in that area is increased to a "6," signify- instructions for preparing the rating. The 20 ing a sustained commitment to and support for areas are grouped into four categories, the policy. If a "2" rating has been sustained for Economic Management, Structural Policies, three or more years in an area, a rating of "1" is Policies for Social Inclusion/Equity, and Public assigned, reflecting an entrenched and Sector Management and Institutions. intractable policy environment. POLICY AREA COVERAGE A Economic Management 1 Management of Inflation and Macroeconomic policies (exchange rate, monetary and fiscal policy) that addresses in- Macroeconomic Balances flation and internal and external imbalances. 2 Fiscal Policy Size of the fiscal balance and the composition of government revenue and spending to assess their compatibility with adequate provision of public services for economic growth, favorable macroeconomic outcomes, and a sustainable path of public debt. 3 Management of Public Debt (External Capacity to manage public debt, external and domestic, and service it now and sus- and Domestic) tainably into the future. Two separate but linked dimensions for assessment are: (a) debt service capacity and (b) debt management capacity. 4 Management and Sustainability of the Degree to which the management of the economy and the development program Development Program reflect three elements: technical competence, sustained political commitment and public support, and participatory processes. B Structural Policies 5 Trade Policy and Foreign Exchange Regime Degree to which policy framework fosters trade and capital movements. 6 Financial Stability Degree to which the structure of the financial sector, and the policies and regulations that affect it, are conducive to diversified financial services to be provided in a context of integrity and with a minimal risk of systemic failure. 7 Financial Sector Depth, Efficiency, and Degree to which policies and regulations affecting financial institutions foster the Resource Mobilization mobilization of savings and efficient financial intermediation. 6 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S 8 Competitive Environment for the Degree to which firms face competitive pressure to behave efficiently or be forced to Private Sector exit. 9 Goods and Factor Markets Policies that affect the efficiency of (a) goods markets and (b) factor markets for labor and land. 10 Policies and Institutions for Extent to which economic and environmental policies contribute to the incomes and Environmental Sustainability health status of the poor, by fostering the protection and sustainable use of natural resources and the management of pollution C Policies for Social Inclusion/Equity 11 Gender Extent to which the country has created laws and policies, and institutions to enforce them, that promote equal access of males and females to productive and economic re- sources, human capital development opportunities, and equal status and protection under the law. 12 Equity of Public Resource Use Extent to which the pattern of public expenditures and revenues favors the poor. 13 Building Human Resources Extent to which the programs and policies (a) affect access to and quality of health care and nutrition services, (b) access to and quality of education, training, and literacy, and (c) prevention of HIV/AIDS and other communicable diseases. 14 Social Protection and Labor Government policies in the area of social protection and labor market regulation reduce the risk of becoming poor and assist those who are poor to mitigate and cope with further risk to their well-being. 15 Monitoring and Analysis of Poverty Quality of systems to monitor poverty outcome/impact indicators and their use in Outcomes and Impacts formulating policies. D Public Sector Management and Institutions 16 Property Rights and Rule-Based Extent to which private economic activity is facilitated by an effective legal system Governance and rule-based governance structure in which property and contract rights are reliably respected and enforced. 17 Quality of Budgetary and Financial Extent to which there are (a) a comprehensive and credible budget, linked to policy Management priorities, which in turn are linked to a poverty reduction strategy; (b) effective finan- cial management systems to ensure that incurred expenditures are consistent with the approved budget, that budgeted revenues are achieved, and that aggregate fiscal control is maintained; (c) timely and accurate fiscal reporting, including timely and audited public accounts and effective arrangements for follow up; and (d) clear and balanced assignment of expenditures and revenues to each level of government. 18 Efficiency of Revenue Mobilization Overall pattern of revenue mobilization­not only the tax structure as it exists on paper, but revenues from all sources as they are actually collected. 19 Quality of Public Administration Extent to which civilian central government staffs (including teachers, health workers, and police) are structured to design and implement government policy and deliver services effectively. 20 Transparency, Accountability, and Extent to which (a) the executive can be held accountable for its use of funds and the Corruption in the Public Sector results of its actions by the electorate and by the legislature and judiciary, and (b) public employees within the executive are required to account for the use of resources, administrative decisions, and results obtained. 6 2 A N N E X A : I N D I C AT O R S O F E C O N O M I C P O L I C I E S Economist Intelligence Unit--Country with a set of criteria. Ratings on these factors Risk Service3 are then used to compile an overall score and The Economist Intelligence Unit--Country Risk rating for the country. EIU states that this Service (EIU) provides internationally compara- overall country risk assessment can be used for ble and regularly updated country risk analysis making a general assessment of the risk of a for 100 developing and highly indebted crisis in the country's financial markets, where countries to its subscribers. foreign investors may have exposure, and that EIU country experts evaluate four categories it is also useful for investors wishing to get a of risk--political, economic policy, economic snapshot of the generalized risk of investing in structure, and liquidity factors--in accordance the country. RISK CATEGORY COVERAGE Political Risk Political risk pertains to the risk of exposure stemming from the political environment. The factors in this cat- egory relate to the threat of war, social unrest, disorderly transfers of power, political violence, international disputes, regime changes, and institutional ineffectiveness, but also include the quality of the bureaucracy, the transparency and fairness of the political system, and levels of corruption and crime in the country in question. Economic Policy Risk Economic policy risk relates to the quality and consistency of economic management. Open economies with low inflation and low fiscal deficits are rewarded in the ratings. Among the subcategories considered are monetary policy (inflation performance and interest rates), fiscal policy (magnitude of public-sector deficits and public debt/GDP), exchange-rate policy (type of exchange-rate regime), trade (barriers to an open trading system), and regulatory policies (capital controls and regulations/attitudes towards foreign investment). Economic Structure Risk Economic structure risk examines economic variables central to solvency. Among the subcategories of vari- ables considered are growth and savings (growth performance, including volatility), the current account (deficit/GDP, magnitude and degree of sustainability), and debt structure (debt/exports, interest due/ exports). Liquidity Risk Liquidity risk considers potential imbalances between resources and obligations which could result in disrup- tion of the financial markets. Among the factors examined are the direction of movement of reserves, import cover, the ratio of money (M2) to reserves, the degree of a country's dependence on portfolio inflows, and the size of direct investment inflows. RISK SCORE EIU INTERPRETATION 0-20 points Contains countries which have no foreign-exchange constraints on their debt-service ability and no problems financing their trade activities. Their economic policies are deemed to be effective and correct in relation to the conditions they face (whether in a boom or a recession) and they have a working government (not always a multiparty democracy in the European mould) capable of effective policy implementation. These countries have no significant constraints on any international financial transactions. 21-40 points Contains countries which also have no significant foreign-exchange constraint, but whose economic policies or political structure may be a cause for concern. B-rated countries have access to commercial capital mar- kets. There are no major risks with respect to international financial transactions, but political risk and economic policy risk often need to be watched carefully. 41-60 points Contains countries which have a record of periodic foreign-exchange crises and political problems. Many of these countries will have negotiated external debt-rescheduling agreements and could be in the process of successfully carrying out an economic reform program. These economies will usually be in a state of flux with persistent, but controllable, internal and external imbalances. However, some will have access to com- mercial capital markets. With caution, this set of countries will often offer exciting opportunities for foreign investors. 61-80 points Contains countries which are currently suffering from serious economic and political problems. Arrears, debt rescheduling, and restricted access to official lending are common characteristics. Many have a narrow, commodity-dependent export base, resulting in potentially large and frequent fluctuations in export earnings and lengthening remittance delays. Many of these economies will be heavily regulated in the initial phases of restructuring or will have failed to implement such reforms. 81-100 points Contains countries which are likely to have a high and rising level of arrears. They will be characterized by severe fiscal imbalances and hyperinflation. Foreign exchange will be scarce, and their relations with multi- lateral lenders severely strained. Often they are in or on the verge of civil war or undergoing violent political change. Political risk is usually extremely high. 6 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Heritage Foundation/Wall Street Journal Each country's overall score is based on an Index of Economic Freedom4 average of the 10 individual factor scores. Each The Heritage Foundation/Wall Street Journal factor is scored against factor-specific criteria Index of Economic Freedom (referred to as on a scale of "1" (signifying a set of policies and Heritage here) measures the degree of govern- institutions that are judged to be most ment involvement in production, distribution, conducive to economic freedom) to "5" (for or consumption of goods and services. To rate policies and institutions that are least each country, raters consider 50 different conducive).5 variables grouped into 10 broad areas. FACTOR VARIABLES CONSIDERED Trade Policy · Weighted average tariff rate · Non-tariff barriers · Corruption in the customs service Fiscal Burden of Government · Top income tax rate · Marginal rate for average taxpayer · Corporate tax rate · Government expenditures/GDP Government Intervention in the Economy · Government consumption/GDP · Government ownership of businesses and industries · Share of government revenues from state- owned enterprises and govern- ment ownership of property · Economic output produced by the government Monetary Policy · Weighted average inflation rate from 1992 to 2001 Capital Flows and Foreign Investment · Foreign investment code · Restrictions on foreign ownership of business · Restrictions on the industries and companies open to foreign companies · Restrictions and performance requirements on foreign companies · Foreign ownership of land · Equal treatment under the law for both foreign and domestic companies · Restrictions on repatriation of earnings · Availability of local financing for foreign companies Banking and Finance · Government ownership of banks · Restrictions on the availability of foreign banks to open branches and sub- sidiaries · Government influence over the allocation of credit · Government regulations · Freedom to offer all types of financial services, securities, and insurance policies Wages and Prices · Minimum wage laws · Freedom to set prices privately without government influence · Government price controls and the extent to which government price con- trols are used · Government subsidies to businesses that affect prices · Government role in setting wages Property Rights · Freedom from government influence over the judicial system · Commercial code defining contracts · Sanctioning of foreign arbitration of contract disputes · Government expropriation of property · Corruption within the judiciary · Delays in receiving judicial decisions · Legally granted and protected private property 6 4 A N N E X A : I N D I C AT O R S O F E C O N O M I C P O L I C I E S Regulation · Licensing requirements to operate a business · Ease of obtaining a business license · Corruption within the bureaucracy · Labor regulations, such as established work weeks, paid vacations, and parental leave, a well as selected labor regulations · Environmental, consumer safety, and worker health regulations · Regulations that impose a burden on business Black Market · Transparency International Corruption Perceptions Index, OR the following factors: ­ Smuggling ­ Piracy of intellectual property in the black market ­ Agricultural production supplied on the black market ­ Service supplied on the black market ­ Transportation supplied on the black market ­ Labor supplied on the black market International Country Risk Guide 6 awarded to any particular risk component is The International Country Risk Guide (ICRG) pre-set within the system and depends on the is based on a set of 22 components grouped importance (weighting) of that component to into three major categories of risk: political, the overall risk of a country. The three financial, and economic, with political risk categories are made up of the following comprising 12 components (and 15 subcompo- components: nents), and financial and economic risk each comprising five components. The political risk · Political risk: government stability, socioeco- assessments are made on the basis of subjec- nomic conditions, investment profile, inter- tive analysis of the available information, while nal conflict, external conflict, corruption, the financial and economic risk assessments military in politics, religion in politics, law and are made solely on the basis of objective data. order, ethnic tensions, democratic accounta- Each component is assigned a maximum bility, and bureaucracy quality. numerical value (risk points), with the highest · Economic risk: GDP per capita, GDP growth, number of points indicating the lowest inflation, fiscal balance, and current account. potential risk for that component and the · Financial risk: external debt, external debt ser- lowest number (0) indicating the highest vice, current account deficit, reserves, and ex- potential risk. The maximum points able to be change rate stability. 6 5 ANNEX B: TECHNICAL NOTES The notes in this Annex provide some backup underlies the analysis of the linkage between detail on points made in the text. Note 1.1 shows improved project results and changes in CPIA the rank correlations between changes in the components. four policy indicators examined in this study. Note 1.2 explains a statistical test for shifts in the Note 1.1: Rank Correlations of Changes in distribution of policy indicator values. Note 1.3 CPIA, Heritage, EIU, and ICRG Indicators explains the results of an analysis of variance of The change in each indicator's value, scaled the factors affecting changes in the CPIA indica- from 0 ("worst" value) to 100 ("best" value) tor over the period 1999-2003. Note 1.4 explains between 1999 and 2003 were computed and a statistical analysis of the relationship between ranked. Correlations, t-statistics, and one-tail policy reform--as reflected in policy indica- significance levels for these correlations of rank tors--and growth during the period 1999-2003. are reported here. Correlations between ranks Note 2.1 explains the mapping of thematic codes are all positive, and three of the six correlations of evaluated projects to CPIA components that are significant at the 5 percent level or better. RANK CORRELATION CPIA Heritage EIU ICRG CPIA Heritage 0.208959 EIU 0.524123 0.252181 ICRG 0.142924 0.010811 0.141985 T STATISTICS CPIA Heritage EIU ICRG CPIA Heritage 1.722709 EIU 4.961722 2.101057 ICRG 1.164241 0.087168 1.156434 SIGNIFICANCE LEVEL (1-TAIL) CPIA Heritage EIU ICRG CPIA Heritage 0.044812 EIU 2.59E-06 0.019729 ICRG 0.124259 0.465401 0.125836 6 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Note 1.2: Sign Test for Shifts in the times the less frequent sign occurs. Observa- Distribution of Policy Indicators tions for which the difference is exactly "0" are The sign test for shift in the distribution of excluded from the sample. The null hypothesis policy indicators toward better values strongly is that each difference has a probability distri- suggest that the majority (i.e., more than half) bution (which need not be the same for all of countries improved their policies over the differences) with median equal to "0." The null period 1999-2003. hypothesis is rejected if the number of positive The result that so many more CPIA ratings and negative signs differ significantly from have increased than have decreased provides equality. When the null hypothesis is true, fairly strong statistical evidence that the distribu- differences are distributed as binomial with the tion of ratings has shifted slightly toward higher probability of a plus sign equal to 0.5. ratings. It could, of course, be the case that some We apply a variant here in which the null systematic bias toward higher ratings has crept hypothesis is a "compound hypothesis" that the in--perhaps induced by the role that the CPIA median of the distribution of differences is less plays in the Bank's IDA allocation, administra- than or equal to 0, i.e., that the parameter of the tive budgeting, and portfolio risk management. binomial distribution of differences of a plus The fact, however, that other indictors show the sign is <0.5, compared with the alternative that same general pattern of improvement suggests it is >= 0.5. For example, in the case of the that the observed improvement in the CPIA is overall CPIA, of 134 countries rated, 85 not due to any such systemic bias. registered increased ratings, 45 decreased The sign test is computed as follows: Let "A" ratings, and 4 did not change. The probability of and "B" represent respectively 1999 and 2003 getting 85 or more positive differences if the indicator ratings on a specific rated criterion and probability of a positive difference were 0.5 is let "X" and "Y" represent the value taken on by 0.0001. If the probability of a positive difference the indicator in 1999 and 2003 respectively. For were less than 0.5, the probability of getting 85 example, in the case of the CPIA, for each rating or more positives out of 130 values would be less variable we have 134 paired observations--one than 0.0001. Hence, 0.0001 is an upper bound value in 1999 and one value in 2003. The 134 on the probability of getting 85 positives in 130 pairs of observations and their differences may trials if the true probability were less than 0.5. be represented by (X1,Y1), (X2,Y2) ....(X134,Y134) Shown here are sign test results for each of and (Y1­X1), (Y2­X2).... (Y134­X134), respectively. the four indicators examined in this study. All The sign test is based on the signs of these show a statistically significant, although modest, differences. Let "r" represent the number of tendency for the country ratings to increase. Maximum probability of Observing I or more No. of No. of No. of increases if Developing Countries with Countries with probability of Countries Rated in Increased Decreased an increase Indicator Both 1999 and 2003 Ratings (I) Ratings (D) is <= 0.5 CPIA 134 85 45 0.0001 Heritage 108 59 39 0.0167 ICRG 88 55 33 0.0069 EIU 78 50 28 0.0044 6 8 A N N E X B : T E C H N I C A L N O T E S Note 1.3: Analysis of Variance of Changes that country factors and policy area factors in CPIA Ratings together explain about one third of the The variance of changes in CPIA ratings on each variation in changes in CPIA ratings. Of this one of 20 questions can be decomposed in variation third, country factors account for over 90 explained by country factors and variation percent, and policy area factors account for less explained by policy areas via an application of than 10 percent. analysis of variance. This decomposition shows The results of this analysis are the following: Observations = 2680 R-squared = 0.3626 Root MSE =0.5834 Adj R-squared = 0.3243 Source Partial SS Df MS F Prob > F Model 489.2 152 3.2187 9.46 0.0000 Country 456.3 133 3.4305 10.08 0.0000 Policy area 33.0 19 1.7355 5.10 0.0000 Residual 860.0 2527 0.3403 Total 1349.3 2679 0.5036 Note 1.4: Linkage Between Economic The pairings of the CPIA with EIU and ICRG, Policy Indices and Growth and of the latter two measures, show a high Policy indices combine different measures of correlation, particularly in 2003. At these levels of policy into a single number. Such indices thus correlation (0.7 and above), each indicator may provide a useful summary for comparing be expected to give similar results when used as countries or, providing the measure is consis- an explanatory variable in a regression equation. tently applied across time, how policies are The exception is Heritage, which is reasonably improving, or not, as the case may be. Indices correlated but does appear to be measuring may also be used in statistical analysis to something different from the other indices.1 determine the impact of policies on economic However, the changes in the indices are less performance. correlated than are the levels. The CPIA and ICRG move together reasonably closely. Comparing the Indices Changes in EIU are not too far adrift but, as As noted, there appears to be considerable with levels, Heritage appears to be measuring overlap between what the different indices are something else. trying to measure. Table 1 is a matrix of the correlations among the indices in both 1999 Policy Changes Over Time and 2003, and the change in each of them Table 2 presents the means of the different between these two years. Recall that for measures in 1999 and 2003. These are shown Heritage and EIU a high score implies a worse first for all countries for which the indicator is policy, so that these indices will be negatively available for the two years. The bottom part of correlated with the CPIA and ICRG. The the table shows the comparison only for those strength of the relationship is thus indicated by countries for which data are available for all the absolute size of the coefficient. four indicators for both years. 6 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S T a b l e 1 C o r r e l a t i o n M a t r i x f o r I n d i c e s 1999 2003 CPIA Heritage EIU ICRG CPIA Heritage EIU ICRG CPIA 1.00 CPIA 1.00 Heritage ­0.59 1.00 Heritage ­0.70 1.00 EIU ­0.77 0.52 1.00 EIU ­0.83 0.55 1.00 ICRG 0.69 ­0.53 ­0.75 1.00 ICRG 0.79 ­0.56 ­0.84 1.00 Changes in the index, 1999-2003 CPIA Heritage EIU ICRG CPIA 1.00 Heritage ­0.15 1.00 EIU ­0.57 0.18 1.00 ICRG 0.71 ­0.28 ­0.63 1.00 In both sets of samples all four measures only in the case of the CPIA and EIU, as show an improvement in policies, since a fall in measured by the test-t). Heritage and EIU indicates that policies have improved. However, these changes are small. It Policy and Growth is questionable whether the "full sample" This section presents regression estimates of should be treated as a sample, since it is the the determinants of growth defined as cumula- population of available data. The common tive growth over the period 1999-2003. Regres- sample is a sample, though not a random one. sions are reported using each of the policy The interpretation of the t-statistics is thus indices in turn. Previous research has questionable, though t-statistics are one way of suggested that there is a "growth boost" from normalizing the value of the change to show policy improvement over and above that that the change is of a significant size (although delivered by the impact of good policies. M e a n V a l u e o f P o l i c y I n d i c e s i n T a b l e 2 1 9 9 9 a n d 2 0 0 3 Full sample CPIA Heritage EIU ICRG 1999 3.36 3.26 55.99 64.18 2003 3.52 3.19 52.21 64.78 Change 0.16 ­0.07 ­3.78 0.59 No. of obs. 134 108 76 88 t-statistic 1.95 ­1.02 ­1.65 0.44 Common sample 1999 3.63 3.08 55.12 65.96 2003 3.82 3.07 51.87 66.37 Change 0.19 ­0.01 ­3.25 0.41 No. of obs. 67 67 67 67 t-statistic 1.92 ­0.06 ­1.46 0.28 7 0 A N N E X B : T E C H N I C A L N O T E S Hence the model includes both the level of the All policy variables are significant, with the index at the start of the period and its change exception of the level of Heritage and CPIA in the between 1999 and 2003. full sample regressions (though barely insignifi- cant). The change as well as the level of the Model Specification policy index is significant, suggesting there is Many variables have been used in growth indeed a "growth boost" from changing policies. regressions. It is important to incorporate the All other variables have the expected sign. most important of these in any model of Convergence is confirmed and high population growth. Previous studies have shown the growth dampens economic performance. importance of investment, which can be broken Inevitable resources promote growth but debt down into sources of investable resources (i.e., reduces it. In the full sample regressions aid is domestic and foreign savings). External debt always significantly negative and the interactive also matters, partly because it detracts from the aid policy term significantly positive. Care must use of savings for investment purposes. Conver- be taken in interpreting this result in two gence theory suggests that lower-income respects. First, the results do not mean that aid countries grow more rapidly, so that the start- actually damages growth in any country since of-period income per capita should be the total effect of aid is positive over the range included. The stock of physical and human of observed values of the independent capital also matters; the former can be captured variables. Take, for example, the full sample by social indicators such as life expectancy and CPIA results in which the coefficient on aid is - literacy and the latter by infrastructure variables 0.007 and that on the interactive term ­is 0.005. (e.g., roads). Since the level of output depends Aid will increase growth provided the value of on the capital stock, the regressions use the the policy index is above 1.2. The actual change in social indicators. Few physical minimum value of the CPIA in 1999 was 2.3. infrastructure variables are available for a large Second, the interactive term can be interpreted number of countries, and these were dropped as meaning either that aid works better in the from the final regressions as they unduly presence of good policies or that policy reform restricted sample size. Population growth, also has a bigger impact on growth when supported included, is typically found to exert a negative by aid. There is no way econometrically to tell impact on growth. In accordance with recent these two possible interpretations apart, and World Bank research findings, an interactive aid- the latter is at least as plausible as the first. policy variable is included. Figure 1 shows the large impact that policy can have on growth.2 A country with poor policy with Results a low aid inflow would be expected to grow by Table 3 presents the regression results. As with only three percent over the whole period from the means, two sets of results are reported. The 1999-2003. A country that changed to good first set covers all available observations for the policies during the period, supported by aid policy measure being used. The sample for the inflows, would be expected to grow over ten times second set of regressions is restricted to those as fast. The figure also shows how the impact of countries for which data on all four policy aid plus better policies would be expected to be indices are available, meaning that differences greater than that of just aid or policy alone, shown in results stem only from the different measure here as the larger estimated incremental impact of being used not sample selection. aid in a good policy regime. 7 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S *** *** * ** ** * *** * *** *** ** *** ** ** 3.96 4.10 ICRG 1.95­ 2.56­ 0.60 2.20­ 0.22­ 0.21 1.94­ 2.80 1.80­ 0.74 1.53 4.63 3.56 2.58­ 2.69­ 0.56 2.32­ 0.57 0.73 0.53­ 0.77 2.21­ 0.16 1.61 78 06 0.65 0.69 0.009 0.011 0.000 1.394­ 0.003 0.020­ 0.000 0.001 0.018­ 0.008 0.061­ 0.039 0.937 0.010 0.009 0.000 1.506­ 0.003 0.022­ 0.001 0.005 0.024­ 0.010 0.135­ 0.008 1.017 *** *** *** ** * ** *** *** *** *** *** *** *** *** 1 EIU 3.51­ 4.10­ 3.26­ 2.13­ 1.44 0.88­ 1.23 0.54 1.81­ 2.17 2.97­ 0.23 2.75 3.56­ 3.95­ 2.94­ 2.3­ 1.61 1.38­ 0.87 0.17­ 0.41 2.75­ 0.07 2.87 Results 65 06 0.68 0.68 0.004­ 0.006­ 0.000 1.138­ 0.005 0.008­ 0.002 0.003 0.051­ 0.017 0.159­ 0.011 1.546 0.005­ 0.006­ 0.000 1.335­ 0.007 0.014­ 0.001 0.006 0.008­ 0.005 0.165­ 0.004 1.765 ** ** *** *** *** ** *** ** * ** ** *** *** Regression 0.39 2.36­ 1.54­ 1.14­ 2.35 2.69­ 4.43 6.62 1.46­ 2.54 3.80­ 3.03 0.93 0.34 2.26­ 1.78­ 2.85­ 1.47 2.15­ 2.24 1.36 1.32­ 1.57 2.87­ 0.43 2.74 Heritage 62 06 0.61 0.61 Growth 0.012 0.099­ 0.000 0.568­ 0.010 0.021­ 0.006 0.016 0.012­ 0.006 0.111­ 0.155 0.498 0.013 0.109­ 0.000 1.763­ 0.007 0.023­ 0.004 0.010 0.073­ 0.024 0.190­ 0.025 1.853 level of 1% at ** *** *** *** *** *** * *** ** ** ** *** *** ** ** *** significant = *** 1.47 2.03 CPIA 2.92­ 1.40­ 2.70 2.66­ 3.62 6.80 0.83­ 1.80 4.19­ 2.55 1.02 0.58 104 2.22 2.47 3.16­ 3.12­ 1.67 2.3­ 1.03 1.31 0.58­ 0.77 2.48­ 0.16­ 2.72 level 0 5% 0.60 at 0.040 0.073 0.000 0.698­ 0.011 0.020­ 0.004 0.017 0.007­ 0.005 0.108­ 0.133 0.566 0.075 0.097 0.000 1.929­ 0.008 0.025­ 0.002 0.010 0.033­ 0.012 0.169­ 0.009­ 1.832 Determinants significant = ** 3 1998 1998 level expectancy expectancy 1999 10% policy capita 1999 growth life illiteracy policy capita growth life illiteracy at able in per in in rate debt sample in per in in rate debt T index sample policyx index dummy policyx dummy squared squared significant = Full Policy Change Income Population Change Change Savings FDI Aid Aid External ECA Intercept R n Common Policy Change Income Population Change Change Savings FDI Aid Aid External ECA Intercept R n6 * 7 2 A N N E X B : T E C H N I C A L N O T E S B e t t e r P o l i c i e s A r e L i n k e d w i t h H i g h e r F i g u r e 1 G r o w t h Cumulative GDP growth 1999­03 30 Low aid High aid 25 20 15 10 5 0 Poor policies Good policies With good policy growth boost Note 2.1: Mapping of Bank Themes to satisfactory or higher. The same theme has CPIA Categories been assigned to more than one CPIA category Shown here are Bank themes mapped to CPIA where the theme definition overlaps CPIA categories and the percentage of projects rated categories. Percentage of Projects Highly Satisfactory/Satisfactory CPIA Question/Theme FY94­98 FY99­03 1. Macroeconomic Management 50.5 45.7 ­ Macroeconomic management 2. Fiscal Policy 58.1 53.9 ­ Debt Management and fiscal sustainability 3. Debt Management 58.1 53.9 ­ Debt management and fiscal sustainability 4. Sustainability of Reform Program 61.1 56.4 ­ Analysis of economic growth ­ Conflict prevention and post-conflict reconstruction ­ Economic statistics, modeling and forecasting ­ Indigenous peoples ­ Other social development ­ Participation and civic engagement ­ Social analysis and monitoring 5. Trade and Foreign Exchange 51.9 49.9 ­ Export development and competitiveness ­ Other trade and integration 7 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S ­ Regional integration ­ Technology diffusion ­ Trade facilitation and market access 6. Financial Stability 53.0 53.6 ­ International financial architecture ­ Regulation and competition policy ­ Standards and financial reporting 7. Financial Sector Depth and Efficiency 51.6 50.6 ­ Other financial and private sector development ­ Small and medium enterprise support ­ State enterprise restructuring and privatization 8. Competitive Environment 51.2 54.0 ­ Corporate governance ­ Infrastructure services for private sector development ­ Other financial and private sector development ­ Other rural development ­ Other urban development ­ Regulation and competition policy ­ Rural markets ­ Rural non-farm income generation ­ Rural policies and institutions ­ Rural services and infrastructure ­ Small and medium enterprise support ­ State enterprise restructuring and privatization 9. Factor and Product Markets 57.6 59.0 ­ Land management ­ Other economic management ­ Social risk reduction 10. Environmental Sustainability 55.8 56.2 ­ Biodiversity ­ Climate change ­ Environmental policies and institutions ­ Land management ­ Other environment and natural resource management ­ Pollution management and environmental health ­ Water resource management 11. Gender 21.5 45.0 ­ Gender 12. Equity of Public Resource Use 50.0 56.9 ­ Public expenditure, financial management and procurement ­ Tax policy and administration 13. Building Human Resources 45.5 54.8 ­ Child health ­ Education for all ­ Education for the knowledge economy ­ Health system performance ­ HIV/AIDs ­ Injuries and non-communicable diseases 7 4 A N N E X B : T E C H N I C A L N O T E S ­ Nutrition and food security ­ Other communicable diseases ­ Other human development ­ Population and reproductive health 14. Social Protection and Labor 56.4 64.2 ­ Access to urban services for the poor ­ Natural disaster management ­ Other social protection and risk management ­ Social risk coping ­ Social risk mitigation ­ Social risk reduction ­ Vulnerability assessment and monitoring 15. Monitoring Poverty Outcomes 44.9 55.2 ­ Poverty strategy, analysis and monitoring ­ Vulnerability assessment and monitoring 16. Property Rights and Rules Based Governance 49.0 50.7 ­ Access to law and justice ­ Judicial and other dispute resolution mechanisms ­ Land management ­ Law reform ­ Legal institutions for a market economy ­ Legal services ­ Other rule of law ­ Personal and property rights 17. Quality of Budget and Financial Management 39.9 51.2 ­ Municipal finance ­ Public expenditure, financial management, and procurement 18. Efficiency of Revenue Mobilization 61.0 64.3 ­ Tax policy and administration 19. Quality of Public Administration 50.0 48.0 ­ Administrative and civil service reform ­ Decentralization ­ Municipal governance and institution building ­ Other economic management 20. Transparency and Accountability 55.3 56.8 ­ Other accountability/anti-corruption ­ Other public sector governance 7 5 ANNEX C: WORLD BANK OPERATIONAL INSTRUMENTS This annex describes the various types of programs. They involve a series of loans that lending and non-lending instruments used by build on the lessons learned from the previous the World Bank. loan(s) in the series. LENDING INSTRUMENTS Emergency Recovery Loans There are two types of lending instruments Emergency recovery loans (ERLs) support the facilitated by the World Bank: Investment Loans restoration of assets and production levels and Adjustment Loans. Investment loans have a immediately after an extraordinary event-- long-term focus (5-10 years) and finance goods, such as war, civil disturbance, or natural works, and services in support of economic and disaster--that seriously disrupts a borrower's social development projects in a broad range of economy. They are also used to strengthen the sectors. Adjustment loans have a shorter-term management and implementation of focus (1-3 years) and provide quick-disbursing reconstruction efforts, and to develop disaster- external financing to support policy and institu- resilient technology and early warning systems tional reforms. Both investment and adjustment to prevent or mitigate the impact of future loans are used flexibly to suit a range of emergencies. purposes; they are occasionally used together in hybrid operations. Financial Intermediary Loans Financial intermediary loans (FILs) provide Investment Lending long-term resources to local financial institu- tions to finance real sector investment needs. Adaptable Program Loans The financial institutions assume credit risk on Adaptable program loans (APLs) provide each subproject. phased support for long-term development INVESTMENT LOANS ADJUSTMENT LOANS Adaptable Program Loan (APL) Deferred Drawdown Option (DDO) Emergency Recovery Loan (ERL) Debt Reduction Loan (DRL) Financial Intermediary Loan (FIL) Poverty Reduction Support Credit (PRSC) Learning and Innovation Loan (LIL) Programmatic Structural Adjustment Loan (PSAL) Specific Investment Loan (SIL) Rehabilitation Loan (RIL) Sector Investment and Maintenance Loan Structural Adjustment Loan (SAL) (SIM) Sector Adjustment Loan (SECAL) Technical Assistance Loan (TAL) Sub-National Adjustment Loan (SNAL) Special Sector Structural Adjustment Loan (SSAL) 7 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Learning and Innovations Loans an environment conducive to private sector Learning and innovation loans (LILs) support investment, where foreign exchange is required small pilot-type investment and capacity- for urgent rehabilitation of key infrastructure building projects that, if successful, could lead and productive facilities. The focus is on key to larger projects that would mainstream the short-term macroeconomic and sector policy learning and results of the LIL. reforms needed to reverse declines in infrastruc- ture capacity and productive assets. Specific Investment Loans Specific investment loans (SILs) support the Poverty Reduction Support Credit creation, rehabilitation, and maintenance of The Poverty Reduction Support Credit (PRSC economic, social, and institutional infrastructure. or PRC) program is expected to consist of a In addition, SILs may finance consultant services series of operations, typically two or three, and management and training programs. which together support IDA countries' medium-term policy and institutional reform Sector Investment and Maintenance Loans programs to help implement their poverty Sector investment and maintenance loans reduction strategies. Its specific structure (SIMs) focus on public expenditure programs depends on country circumstances, including in particular sectors. They aim to bring sector the objectives and nature of the country's expenditures, policies, and performance in line reform program that it supports and the timing with a country's development priorities by of the requirement for assistance. helping to create an appropriate balance among new capital investments, rehabilitation, Programmatic Structural Adjustment Loans reconstruction, and maintenance. They also The programmatic structural adjustment loan help the borrower develop the institutional (PSAL or PSL) is provided in the context of a capacity to plan, implement, and monitor an multiyear framework of phased support for a expenditure or investment program. medium-term government program of policy reforms and institution building. Technical Assistance Loans Technical assistance loans (TALs) are used to Rehabilitation Loans build institutional capacity in the borrower The rehabilitation loan (RIL) supports govern- country. They may focus on organizational ment policy reform programs aimed at creating arrangements, staffing methods, and technical, an environment conducive to private sector physical, or financial resources in key agencies. investment, where foreign exchange is required for urgent rehabilitation of key infrastructure Adjustment Lending and productive facilities. The focus is on key short-term macroeconomic and sector policy Deferred Drawdown Options reforms needed to reverse declines in The Deferred Drawdown Option (DDO) is infrastructure capacity and productive assets. available to both IBRD and Blend countries to whom the Bank makes a single-tranche adjust- Structural Adjustment Loans ment loan. The DDO gives borrowers access to The structural adjustment loan (SAL) supports long-term IBRD resources to manage ongoing reforms that promote growth, efficient use of structural programs if market borrowing resources, and sustainable balance of payments becomes difficult and unforeseen financing over the medium and long term. needs materialize. Sector Adjustment Loans Debt Reduction Loans The sector adjustment loan (SECAL) supports The debt reduction loan (DRL) supports govern- policy changes and institutional reforms in a ment policy reform programs aimed at creating specific sector. 7 8 A N N E X C : W O R L D B A N K O P E R AT I O N A L I N S T R U M E N T S Special Structural Adjustment Loans ment by developing and following up action The special structural adjustment loan (SSAL) plans agreed with the government and (b) supports structural and social reforms by enable the World Bank to judge the level of creditworthy borrowers approaching a fiduciary risk to its funds that may be provided possible crisis, or already in crisis, and with to a country, particularly in the context of exceptional external financing needs. These adjustment lending. loans help countries to prevent a crisis or, if one occurs, to mitigate its adverse economic Country Procurement Assessment Report and social impacts. The Country Procurement Assessment Report (CPAR) diagnoses the health of a country's Sub-National Adjustment Loans procurement system and practices and aims to The Sub-National Adjustment Loan (SNAL) generate a dialogue with governments on supports reforms that promote growth, needed reforms. efficient use of resources, and sustainable balance of payments at a sub-national level. Poverty Assessment The Poverty Assessment (PA) provides informa- NON-LENDING INSTRUMENTS tion on the causes and consequences of poverty Analytic and Advisory (AAA) services include in a country and examines how public policies, standard economic and sector work (ESW) as expenditures, and institutions affect poor well as less formal just-in-time policy advice people. The Assessment describes the poor, such as Policy Notes and non-ESW activities details their living conditions, describes their including workshops, seminars, conferences, changing situation over time, and clarifies the etc. Described below are the core diagnostic main challenges facing poor people when they ESW tools for examining social and structural try to emerge from poverty. The Assessment policies and fiduciary issues in client countries. also considers how labor-intensive growth, human resource development, and social Country Economic Memorandum and protection programs can alleviate poverty. Development Policy Review The Country Economic Memorandum (CEM) Public Expenditure Review and the Development Policy Review (DPR) The Public Expenditure Review (PER) examines analyze key aspects of a country's economic government resource allocations within and development, such as growth, fiscal reform, among sectors and assesses the equity, public administration, foreign trade, financial efficiency, and effectiveness of those allocations sector development, and labor markets. They in the context of the macroeconomic are flexible analytic instruments whose framework and sector priorities. In addition, it structure and content are largely dependent on identifies the reforms needed in budget country circumstances. Their aim is to provide processes and administration to improve the an integrated view of a country's development efficiency of public spending. PER plays a priorities and a framework for designing central role in shaping World Bank lending development strategies. decisions that have an impact on the public resource envelope. Country Financial Accountability Assessment The Country Financial Accountability Assess- Integrative Fiduciary Assessment ment (CFAA) evaluates the overall quality of a The Integrative Fiduciary Assessment (IFA) country's public financial management system, provides integrated analysis of a country's covering budgeting, accounting, reporting and public expenditure, procurement, and financial auditing, and external scrutiny of public management systems. IFAs integrate the work finances. The CFAA has twin objectives: (a) normally carried out through PERs, CPARs, and improve a country's public financial manage- CRAAs. 7 9 ANNEX D: OUTCOMES AND BANK CONTRIBUTIONS TO COUNTRY CASES LISTED IN FIGURE 2.2 8 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Country/ Economic and Social Indicators 1998 1999 2000 2001 2002 Armenia GDP growth (annual %) 7 3 6 10 13 GNI per capita, Atlas method (current US$) 560 590 650 700 790 Exports of goods and services (annual % growth) 5 7 17 23 27 Total debt service (% of exports of goods and services) 13 12 8 8 Inflation, GDP deflator (annual %) 11 0 ­1 4 2 Poverty headcount, national (% of population) 55 47 Births attended by health staff (% of total) 97 97 Mortality rate, infant (per 1,000 live births) 35 31 Life expectancy at birth, total (years) 74 73 74 74 75 Illiteracy rate, adult total (% of people ages 15 and above) 2 2 2 2 1 School enrollment, primary (% net) 69 Brazil GDP growth (annual %) 0 1 4 1 2 GNI per capita, Atlas method (current US$) 4,610 3,860 3,610 3,060 2,850 Exports of goods and services (annual % growth) 4 9 11 12 Total debt service (% of exports of goods and services) 79 118 94 75 Inflation, GDP deflator (annual %) 5 6 8 7 8 Poverty headcount, national (% of population) Births attended by health staff (% of total) Mortality rate, infant (per 1,000 live births) 32 31 Life expectancy at birth, total (years) 68 68 69 Illiteracy rate, adult total (% of people ages 15 and above) 14 14 13 13 12 School enrollment, primary (% net) 97 97 Bulgaria GDP growth (annual %) 4 2 5 4 4 GNI per capita, Atlas method (current US$) 1,270 1,450 1,590 1,670 1,790 Exports of goods and services (annual % growth) ­5 ­5 17 8 4 Total debt service (% of exports of goods and services) 21 19 16 17 Inflation, GDP deflator (annual %) 24 4 7 6 5 Poverty headcount, national (% of population) Births attended by health staff (% of total) Mortality rate, infant (per 1,000 live births) 14 15 13 14 Life expectancy at birth, total (years) 71 71 72 72 72 Illiteracy rate, adult total (% of people ages 15 and above) 2 2 2 2 1 School enrollment, primary (% net) 97 95 94 China GDP growth (annual %) 8 7 8 7 8 GNI per capita, Atlas method (current US$) 740 780 840 890 940 Exports of goods and services (annual % growth) 7 15 31 10 7 Total debt service (% of exports of goods and services) 9 12 9 8 Inflation, GDP deflator (annual %) ­2 ­2 1 0 ­1 8 2 A N N E X D : O U T C O M E S A N D B A N K C O N T R I B U T I O N S T O C O U N T R Y C A S E S L I S T E D I N F I G U R E 2 . 2 The Bank's financial and analytic assistance has contributed to the maintenance of macroeconomic stability, with significant progress in structural reform programs, especially in large-scale privatization and energy reform, leading to strong economic growth, averaging over 10% annually since 2001. However, high growth rates may be difficult to sustain beacuse growth is focused in a few capital-intesive sectors. The Bank maintained a significant poverty focus through analytic work and conditionality under adjustment projects. Poverty and income inequality were high throughout the decade, but with a lag have begun to respond to growth. Extreme poverty has fallen from 27% to 20%. Income inequality also declined, with an estimated Gini coefficient of income of 0.52 in 2001, down from 0.57 in 1998/99. The Bank's assistance program designed to improve efficiency and growth and to stimulate private investment produced mixed results. The actions and reforms supported by the program were all necessary and underpinned by sound analytical work; however, it did not succeed in removing some critical structural bottlenecks that continue affecting economic performance. Stabilization had a strong impact on poverty reduction because high inflation affected disproportionately the poor. The share of the popula- tion below the poverty and extreme poverty lines declined by 7 and 5 percentage points over the last decade. Brazil is likely to meet all the MDG targets at the national level by 2015. The Bank contributed to many of these outcomes, primarily through its assistance to education, health, and the environment. Two financial crises in the early 1990s were followed by financial discipline, economic stability, and credible reform program during 1997-2000. However, the period of reforms is too short to enable regaining ground lost in the initial transition years. Foundations for sustained growth are not yet fully in place, and the business climate continues to suffer from significant shortcomings. Poverty rates jumped dramatically with the extreme economic instability of the early 1990s, but the poverty rate declined to 11% in 2001, compared to 36% during the crisis. Poverty has a strong regional, ethnic, and rural dimension. Social indicators show mixed result: infant mortality has fallen slightly, but remains high relative to countries in the region, and while enrollment in basic education has increased slightly, it has fallen ten percentage points for poor children. China's achievements in growth and stabilization over the past decade have been outstanding. The Bank helped identify the critical nexus of enterprise/financial sector/fiscal reforms needed to underpin high growth with stability, but intermittent and incomplete reform of the inter- government fiscal system, SOEs, and the financial sector pose risks for continued growth and stability. (Table continues on the following page.) 8 3 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Country/ Economic and Social Indicators 1998 1999 2000 2001 2002 China (continued) Poverty headcount, national (% of population) 5 Births attended by health staff (% of total) 67 70 Mortality rate, infant (per 1,000 live births) 32 31 Life expectancy at birth, total (years) 70 70 71 Illiteracy rate, adult total (% of people ages 15 and above) 16 15 15 14 14 School enrollment, primary (% net) 95 93 Croatia GDP growth (annual %) 3 ­1 3 4 5 GNI per capita, Atlas method (current US$) 4,730 4,560 4,500 4,410 4,640 Exports of goods and services (annual % growth) 4 1 9 9 Total debt service (% of exports of goods and services) 19 20 23 28 Inflation, GDP deflator (annual %) 8 4 5 3 3 Poverty headcount, national (% of population) Births attended by health staff (% of total) 100 Mortality rate, infant (per 1,000 live births) 8 8 8 7 Life expectancy at birth, total (years) 73 74 74 Illiteracy rate, adult total (% of people ages 15 and above) 2 2 2 2 2 School enrollment, primary (% net) Lithuania GDP growth (annual %) 7 -2 4 7 7 GNI per capita, Atlas method (current US$) 2,700 2,850 3,110 3,340 3,660 Exports of goods and services (annual % growth) 1 ­15 11 21 3 Total debt service (% of exports of goods and services) 17 20 22 31 Inflation, GDP deflator (annual %) 5 0 1 0 0 Poverty headcount, national (% of population) Births attended by health staff (% of total) Mortality rate, infant (per 1,000 live births) 9 9 9 8 Life expectancy at birth, total (years) 72 72 73 73 73 Life expectancy at birth, total (years) 72 72 73 73 73 Illiteracy rate, adult total (% of people ages 15 and above) 0 0 0 0 0 School enrollment, primary (% net) 92 94 95 Mongolia GDP growth (annual %) 3 3 1 1 4 GNI per capita, Atlas method (current US$) 410 390 390 410 440 Exports of goods and services (annual % growth) 5 13 2 Total debt service (% of exports of goods and services) 6 4 6 8 Inflation, GDP deflator (annual %) 12 10 12 6 21 Poverty headcount, national (% of population) Births attended by health staff (% of total) 93 Mortality rate, infant (per 1,000 live births) 62 61 Life expectancy at birth, total (years) 65 65 65 Illiteracy rate, adult total (% of people ages 15 and above) 2 2 2 2 1 School enrollment, primary (% net) 88 90 89 8 4 A N N E X D : O U T C O M E S A N D B A N K C O N T R I B U T I O N S T O C O U N T R Y C A S E S L I S T E D I N F I G U R E 2 . 2 China has made significant gains in poverty reduction, but progress has slowed since the mid-1990s despite high growth rates. Income inequality and access to social services has grown, and vulnerability has increased with rise in unemployment and slow development of social protection. The Bank made significant contributions to poverty reduction through its AAA and poverty monitoring, and many proj- ects in agriculture, health, and transport have contributed to poverty reduction. Croatia launched a remarkably successful stabilization program in 1993 with support from the IMF and the Bank. Annual inflation fell below 4%, the currency was successfully pegged to the DM, there were large inflows of capital, and fiscal accounts were put under control. The challenge now is to build on the stabilization program and implement an adjustment program to transform the economy. In line with the government's lack of interest, a proposed Bank poverty reduction program was never implemented, and a Poverty Assess- ment (PA) was also not pursued. With hindsight, and given the experience of other transition economies, the Bank could have pushed for the preparation of a PA even without much government ownership. A recent Client Survey concluded that the Bank is not strongly perceived as assigning a central role to poverty reduction. Bank supported structural reforms have contributed to prudent macroeconomic and fiscal management, leading to strong GDP and export growth. Enterprise privatization is almost complete, all banks are privately owned and have foreign participation, the energy sector operates on a largely commercial basis, and infrastructure bottlenecks are being addressed. Extreme poverty is not substantial, but pockets of poverty, especially in rural areas, must be addressed by improving markets and factor mobility. Health, education, and agriculture reforms have lagged, and human capital and labor mobility need enhancement. Spending on so- cial security and welfare was 10.5% of GDP, below the accession country average of 13.5%. Overall, it's been more difficult to implement social sector projects in line with agreed schedules than projects in other sectors. The Bank made positive contributions to progress in the 1990s, but its influence in several areas was limited. Mongolia stabilized its econ- omy despite external shocks and pursued reforms including removal of price controls, privatization of SMEs, and changes in laws to enable private sector activity. However, efforts to remove some key policy and institutional impediments to growth have met with less success. Following a decline in the early 1990s, delivery of basic health and education services stabilized, although there have been signs of strain. Health and education services expenditure remain relatively high at 4.3% and 5.7% of GDP respectively in 1999. However, primary educa- tion enrollment rates dropped by 8% during 1989-1998, partly attributed to increased demand for boys' labor in livestock herding. The de- clining quality and access to rural health services is of increasing concern. (Table continues on the following page.) 8 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Country/ Economic and Social Indicators 1998 1999 2000 2001 2002 Russia GDP growth (annual %) ­5 5 9 5 4 GNI per capita, Atlas method (current US$) 2,270 1,750 1,690 1,750 2,140 Exports of goods and services (annual % growth) ­2 ­2 9 1 3 Total debt service (% of exports of goods and services) 12 14 10 14 Inflation, GDP deflator (annual %) 16 65 41 18 15 Poverty headcount, national (% of population) Births attended by health staff (% of total) 99 99 Mortality rate, infant (per 1,000 live births) 16 17 16 18 Life expectancy at birth, total (years) 67 66 65 66 66 Illiteracy rate, adult total (% of people ages 15 and above) 1 0 0 0 0 School enrollment, primary (% net) Rwanda GDP growth (annual %) 9 8 6 7 9 GNI per capita, Atlas method (current US$) 250 270 260 240 230 Exports of goods and services (annual % growth) 2 19 9 40 ­7 Total debt service (% of exports of goods and services) 17 26 27 11 Inflation, GDP deflator (annual %) 2 ­4 3 0 0 Poverty headcount, national (% of population) Births attended by health staff (% of total) 31 Mortality rate, infant (per 1,000 live births) 100 96 Life expectancy at birth, total (years) 40 40 40 40 Illiteracy rate, adult total (% of people ages 15 and above) 36 34 33 32 31 School enrollment, primary (% net) 96 97 Tunisia GDP growth (annual %) 5 6 5 5 2 GNI per capita, Atlas method (current US$) 2,050 2,090 2,100 2,070 2,000 Exports of goods and services (annual % growth) 4 5 7 14 ­2 Total debt service (% of exports of goods and services) 15 16 20 13 Inflation, GDP deflator (annual %) 3 4 2 3 3 Poverty headcount, national (% of population) Births attended by health staff (% of total) 90 Mortality rate, infant (per 1,000 live births) 27 26 26 21 Life expectancy at birth, total (years) 72 72 72 72 73 Illiteracy rate, adult total (% of people ages 15 and above) 32 30 29 28 27 School enrollment, primary (% net) 97 98 99 Vietnam GDP growth (annual %) 6 5 7 7 7 GNI per capita, Atlas method (current US$) 350 360 390 410 430 Exports of goods and services (annual % growth) Total debt service (% of exports of goods and services) 9 10 7 7 Inflation, GDP deflator (annual %) 9 6 3 3 3 8 6 A N N E X D : O U T C O M E S A N D B A N K C O N T R I B U T I O N S T O C O U N T R Y C A S E S L I S T E D I N F I G U R E 2 . 2 The government has made significant progress in fiscal adjustment, the incentive regime, legislation approvals for structural reforms, the strength- ening of public institutions, and the restoration of public trust in its ability to conduct policies. Aided by a positive terms of trade shock and the effects of the devaluation, growth has recovered and inflation has been reduced. Country performance on poverty, equity, and human and social development has been disappointing, both in absolute terms and relative to other transition economies. Human development indicators show sharp deterioration between the mid-1980s and the first half of the 1990s, and then either stagnation or only modest recovery. While recovering from their worst levels, infant and maternal mortality remain high, life expectancy low, and access to health services inadequate. It is difficult to have much "development" impact in circumstances as those prevailing in Rwanda over the past decade. The Bank's greatest contribution in the post-genocide period has been its assistance to macroeconomic stabilization and greater fiscal transparency, removal of labor market controls, adoption of a legal framework conducive to private sector growth, and access to the enhanced HIPC initiative. Rwanda's social indicators show a rebound from the immediate post-genocide period, but remain of significant concern. Social service de- livery is weak, with shortages of health personnel and equipment, and lack of adequately trained teachers and teaching materials. The poverty headcount stands at 60%, life expectancy has stagnated, and the HIV/AIDS crisis has worsened. The country remains at the beginning of a long, ardous socio-economic transformation process. With effective support from the Bank, Tunisia averted a balance of payments crisis in the 1980s and has since maintained a solid record of macroeconomic stability and socioeconomic progress. Tunisia has successfully shifted from resource-based exports to manufactures and services. Attributes of Tunisia's success include ownership and broad political consensus, and a well-developed human resource base. The Bank's human resource development strategy has emphasized achieving universal primary education, expanding post-primary enrollment, reducing infant mortality and fertility, increasing health coverage, and enhancing the quality and efficiency of education and health services. The Bank's focus on the social sectors is in line with the country's emphasis on social achievements and the MDGs, which Tunisia is expected to meet by 2015. Vietnam's economic growth since 1990 has been strong, underpinned by prudent monetary and fiscal management. Structural reforms initi- ated in the 1980s without Bank assistance has continued into the 1990s, with the Bank contributing to continued growth, poverty reduction, and institutional developments through sectoral lending, technical assistance, and facilitation of grant facilities. (Table continues on the following page.) 8 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Country/ Economic and Social Indicators 1998 1999 2000 2001 2002 Vietnam (continued) Poverty headcount, national (% of population) Births attended by health staff (% of total) 70 Mortality rate, infant (per 1,000 live births) 28 30 Life expectancy at birth, total (years) 69 69 70 Illiteracy rate, adult total (% of people ages 15 and above) 8 8 7 7 7 School enrollment, primary (% net) 97 96 95 Haiti GDP growth (annual %) 3 2 1 ­2 ­1 GNI per capita, Atlas method (current US$) 400 440 500 480 440 Exports of goods and services (annual % growth) 22 5 2 ­3 Total debt service (% of exports of goods and services) 9 9 8 5 Inflation, GDP deflator (annual %) 13 8 11 17 10 Poverty headcount, national (% of population) Births attended by health staff (% of total) 27 Mortality rate, infant (per 1,000 live births) 81 79 Life expectancy at birth, total (years) 53 52 52 Illiteracy rate, adult total (% of people ages 15 and above) 52 51 50 49 48 School enrollment, primary (% net) Zambia GDP growth (annual %) ­2 2 4 5 3 GNI per capita, Atlas method (current US$) 330 320 310 320 330 Exports of goods and services (annual % growth) 5 5 -14 29 11 Total debt service (% of exports of goods and services) 21 16 20 12 Inflation, GDP deflator (annual %) 19 21 30 24 20 Poverty headcount, national (% of population) 73 Births attended by health staff (% of total) Mortality rate, infant (per 1,000 live births) 112 112 Life expectancy at birth, total (years) 38 38 37 37 Illiteracy rate, adult total (% of people ages 15 and above) 24 23 22 21 20 School enrollment, primary (% net) 69 66 66 Zimbabwe GDP growth (annual %) 3 ­1 ­5 ­8 ­6 GNI per capita, Atlas method (current US$) 570 470 Exports of goods and services (annual % growth) 20 5 ­17 ­4 ­1 Total debt service (% of exports of goods and services) 38 27 22 7 Inflation, GDP deflator (annual %) 29 56 60 70 108 Poverty headcount, national (% of population) Births attended by health staff (% of total) 73 Mortality rate, infant (per 1,000 live births) 73 76 Life expectancy at birth, total (years) 40 40 39 39 Illiteracy rate, adult total (% of people ages 15 and above) 13 12 11 11 10 School enrollment, primary (% net) 80 80 8 8 A N N E X D : O U T C O M E S A N D B A N K C O N T R I B U T I O N S T O C O U N T R Y C A S E S L I S T E D I N F I G U R E 2 . 2 Poverty decreased dramatically from 58% to 37% between 1993 and 1998. Most social indicators (infant, child, and maternal mortality; life ex- pectancy; literacy) continue to improve from already high levels, and gender gaps have narrowed further. In health, there has been a sharp drop in the proportion of deaths from communicable diseases due to successful public health programs. However, many households still re- main close to the poverty line and child malnutrition remains stubbornly high. The Bank's assistance over the last 15 years has had negligible impact on Haiti's development challenges. Governance has not improved, and, in fact, remains in a state of crisis. The capacity of the public sector and public accountability remain very low; budget transparency has ac- tually deteriorated, and there has been little improvement in infrastructure, social services, or the well-being of Haiti's poor. Roughly two-thirds of the population live in poverty, half of adults are illiterate, and health services are inadequate to address high infant and maternal mortality and major public health crises, including TB, HIV/AIDS, and, most recently, polio epidemics. The Bank's most recent (1998) poverty report has been of good quality, but its impact has been limited by the unresolved political situation. Zambia's growth record has been dismal, with a twenty-five year decline in per capita income ($300 in 2000). Investment and economic growth in the 1990s were at or below levels of the 1980s, below Bank projections, which were overly optimistic, and below what was necessary to increase per capita income and reduce poverty. Zambia's incidence of poverty, above 70%, is higher than most countries in the world. The combined impact of widespread poverty, declining per capita incomes and high HIV/AIDS prevalence has led to deterioration in most social welfare indicators. Life expectancy has dropped by more than a decade, malnutrition is rising, and public health efforts such as immunizations are faltering, even in a context of rising literacy and primary school enrollment rates that are relatively high. Progress was made in the 1990s in increasing growth and liberalizing trade and foreign exchange regimes. But, financial liberalization and tax reduction were wrongly sequenced relative to expenditure reductions, proving fiscally costly and leading to a domestic debt trap. Since 2000, the country has been in serious economic crisis fueled by mounting fiscal imbalances, farm invasions and economic policy uncertainties. Extreme poverty rose during the 1990s and remains more widespread, deeper and severe in rural areas, although urban poverty has risen dra- matically since the mid-1990s. Gains in health in the 1980s were largely eroded in the 1990s by the economic crisis, the AIDS epidemic, which now affects a third of the adult population, and recourse to user fees with ineffective exemption systems. Life expectancy has been reduced by more than a decade, and infant mortality has risen sharply. 8 9 ANNEX E: EFFECTIVENESS OF BANK SUPPORT FOR POLICY REFORM: AN OVERVIEW FROM THE CODE CHAIRPERSON The Committee on Development Effectiveness in making cross-country comparisons. OED met on April 28, 2004, to discuss the 2003 responded that the four indicators were Annual Review of Development Effectiveness selected on the basis of their fairly widespread (ARDE 2003) of the Operations Evaluation use and recent updating for a large number of Department (OED). The very rich discussion at developing countries, and also that they the committee carried over to the full board encompassed a fairly broad range of the discussion (box). The 2003 ARDE examines the development policy agenda. OED also noted effectiveness of Bank support to borrower that all indicators indicate recent modest countries to help them put in place policies improvement in the majority of developing conducive to policy reform and sustainable countries' policies. Management informed the poverty reduction. Outcomes of Bank country committee that a report of an expert panel on assistance, as measured by Country Assistance the Bank's policy indicator--the Country Policy Evaluations, have been moderately satisfactory and Institutional Assessment (CPIA)--will soon or better about 70 percent of the time, and be available. policy reform paid off in terms of higher growth and higher Millennium Development Goals Track Record and Turnaround Countries indicators. The report suggests that the The Committee agreed that adopting a effectiveness of Bank support for policy reform cautious stance on lending and relying could be improved by linking lending more primarily on AAA is prudent in non-emergency closely to country and sector knowledge and to non-crisis-situation countries with either no performance, through better adaptation of the track record or poor track records. Several knowledge content of Bank products to country members, however, also agreed with manage- circumstances, and through further piloting and ment's view that opportunities to support experimentation with outcomes-based lending turnaround countries (i.e., those that are and other approaches to strengthening country improving their policies and fiduciary capacity) leadership, ownership, and results orientation. should not be missed and suggested that The committee discussions highlighted six lessons learned from "turnaround" countries issues: (a) analytical and methodological issues; would be helpful in shaping Bank future (b) track record and turnaround countries; (c) assistance to such borrowers. Management said debt sustainability analysis; (d) triggers and that there are cases where the Bank has been conditionality; (e) Poverty Reduction Strategy ahead of the curve, notably in Africa, where Papers (PRSPs) and country ownership; and (f) they managed to "catch" and provide timely analytical and advisory (AAA) services and support to several turnaround countries. economic and sector work (ESW). Management indicated that programs such as "Low-Income Countries Under Stress" (LICUS), Analytical and Methodological Issues with its emphasis on ESW, help position the Several members asked about the choice of Bank to be ready to support countries in policy indicators examined and their usefulness turnaround situations. 9 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Debt Sustainability Analysis to concessional finance from the Bank and The Committee agreed with OED's finding that Fund. OED responded that this was a finding of the Bank's debt sustainability analysis needs to the multi-partner evaluation "Toward Country be improved. Several members commented on Led Development" and that a full PRSP evalua- the unrealistic optimism of many Bank tion would be sent to the board by the end of analyses, particularly concerning prospects for FY 2004. Management stated that this particu- growth of gross domestic product and exports. lar finding differs from the views of the Bank's The committee welcomed management's country directors, who see the PRSP as a new statement that an initiative to strengthen the and useful tool which promotes country Bank's debt sustainability analysis is being ownership and serves a beneficial role in the developed. alignment and harmonization process. Triggers and Conditionality Analytical and Advisory Work and The finding of the report that Bank instru- Economic and Sector Work ments which are used to link lending to policy Committee members recognized the importance performance--Country Assistance Strategy of and broadly commended the Bank's analytic triggers and conditionality--have not resulted work. Most speakers agreed with OED's finding in satisfactory outcomes in highly uncertain and recommendations, especially the need for situations or when conflicting incentives made ESW to be timely and relevant in low-income, low- conditionality difficult to enforce resonated Country Policy and Institutional Assessment, and with several members. OED indicated that the LICUS countries, and further underscored the range of uncertainty that surrounds policy need for the knowledge content of and advice reform makes it very difficult in some situations contained in Bank products to reflect the to specify meaningful triggers or conditions. specificities of client countries. OED remarked Management informed the Committee that a that in certain countries, it is best in the long run report on recent experience with Country to moderate lending and focus on analytical work Assistance Strategies (notably covering the until an adequate policy framework for develop- pilot results-based country strategies) will be ment effectiveness is in place. Management made available early next fiscal year. stated further with regard to ESW that there are many issues pertaining to value added and PRSPs and Country Ownership customization, and that the Bank is increasingly Some members asked for clarification of the improving its understanding about how to do report's finding that the PRSP's potential as a ESW in a participatory manner. vehicle for fostering country ownership is undermined by its role as a condition for access Pietro Veglio, Acting Chairperson CODE 9 2 A N N E X E : E F F E C T I V E N E S S O F B A N K S U P P O R T F O R P O L I C Y R E F O R M V I E W S O F T H E B O A R D O F E X E C U T I V E D I R E C T O R S The Executive Directors of the Bank and IDA discussed the 2003 debted Poor Country (HIPC) programs. Given that work was on- Annual Review of Development Effectiveness on May 20, 2004. A going to develop guidelines within the Bank on debt sustain- large number of speakers welcomed the report for its valuable in- ability analysis, other speakers underscored the need for caution sights into the Bank's performance in the area of policy reform and with lending decisions until resulting guidelines were in place for assembling a wealth of interesting observations based on ac- and highlighted the need for clear and transparent debt sus- tual country cases that should be considered in future Bank sup- tainability analyses to be included in all documentation to the port for policy reform. They noted that a central message of the board on loans to HIPC Initiative countries. report was that policy reform paid off, with countries with improved policies growing at more than twice the rate as those without. The GenericBestPracticesandChoiceofInstruments.Several challenge for the board and management was to ensure that these speakers discussed the theme in the report that the Bank tended insights were adopted and incorporated into future operations. to be too narrowly focused on generic best practices. They agreed that the Bank should intensify efforts to customize and Linking Bank Support to Countries' Policy Reform Track adapt knowledge to specific local problems, noting that use of Records. Several speakers did not subscribe to OED's conclu- local experts could be cost effective. They said the Bank needs sion that the Bank should wait to lend to a country until it had to change the unfavorable perception that it is too narrowly fo- demonstrated a solid record with policy performance. They agreed cused on best practices and tends to provide textbook advice. with management that opportunities to support turnaround cases Some speakers believed that the Bank should support and re- should not be missed and that the Bank should therefore remain ward piloting of new approaches to facilitate country ownership engaged in LICUS countries. Several other speakers pointed to and avoid "one-size fits all" frameworks. faster growth in reforming countries and empirical evidence that the Bank was lending more to better performers with relatively Knowledge Management and ESW. Several speakers con- good policy environments and agreed with the report's argument tinued to advocate improvement in the Bank's interregional and for a cautious approach to lending in countries with weak reform inter-temporal sharing of knowledge. Lessons from certain re- track records while maintaining an active policy dialogue. Ex- gions did not appear to be taken into account by other regions, ercises like the LICUS were therefore useful to position the Bank and one speaker was surprised that only a third of networks and to support countries in turnaround situations. regional units reported having routine practices for validating A number of speakers pointed to the finding that conditional- lessons learned. Some speakers were concerned that insuffi- ity had in general been ineffective in linking aid with reform, and cient knowledge sometimes was transferred to new staff and that Country Assistance Strategy triggers were less useful in recommended that staff consider alternative means to improve cases where future developments and appropriate Bank responses the dissemination of the Bank's knowledge and to address "gov- were uncertain. These findings raised a number of questions ernment bias" at the country level. which need to be carefully examined as the Bank moves forward Some speakers discussed the conclusion concerning the in developing its policies and methods to support policy reform. importance of ESW to the effectiveness of the Bank's operations, Some speakers said that this result supports calls for main- and they encouraged the Bank to continue promoting this work, streaming outcomes-based conditionality and adding flexibility in including increasing the strategic focus, operational relevance, Country Assistance Strategies. Some speakers commented that and developmental impact of this tool. Some other speakers measurement indicators were open to interpretation. If used me- observed that ESW appeared to have had less of an impact in chanically, they could create perverse incentives and do more harm low- income countries and in countries with low Country Pol- than good. They said that judgmental and judicious use of indicators icy and Institutional Assessment scores, and they urged man- has to be stressed at all times and that sometimes the Bank needs agement to make increasing the effectiveness of ESW in these to content itself with qualitative indicators rather than quantita- counties a top priority. One speaker agreed that timely, relevant, tive ones that would lead the Bank in the wrong direction. country-specific information was needed, even in the most prob- lematic country situations, to allow the Bank to be ready when Debt Sustainability. A number of speakers stated that the re- opportunities to move ahead emerged. Another speaker agreed port made some useful observations regarding debt sustain- with OED that the Bank should not normally engage in lending ability, especially problems created by systemic over-optimism before ESW was available providing necessary country and with the debt sustainability analyses underpinning Highly In- sector knowledge. Note: This summary by OED is based on the Corporate Secretariat's record of the ARDE board discussion. 9 3 ENDNOTES Chapter 1 6 are statistically significant at the 5 percent level or 1. ARDE 2003 does not examine Bank initiatives better. See Annex B, Technical Note 1.1. with regard to developed-country policy reform, which 6. Annex B, Technical Note 1.2. includes the Bank's efforts to promote more access 7. Countries shown as improving their policies in to developed-country markets and increases in de- Figure 1.3 are those in the top quintile of increases on velopment finance, because of the lack of evaluation (a) at least 3 of the 4 indicators, with at most 1 bot- evidence. One exception relates to reform efforts tom quintile ranking, or (b) 2 of the 4 indicators with evaluated in connection with the Operations Evalua- no bottom quintile rankings. Countries shown as hav- tion Department's (OED's) recent evaluation of the ing deteriorating policies are in the bottom quintile implementation of Comprehensive Development of increases on (a) at least 3 of the 4 indicators, with Framework principles (OED 2003j). at most 1 top quintile ranking, or (b) 2 of the 4 indi- 2. In his analysis of a set of the Bank's early in- cators with no top quintile ranking. vestment projects in infrastructure and industry, 8. Twenty-one (21) of the 134 countries with CPIA Hirschman (1967) argued that even simple invest- ratings in both 1999 and 2003 either ended conflicts, ment projects frequently have both unexpected prob- experienced one or more serious natural disasters, lems and--in successful projects--unexpected and/or experienced a serious financial crisis during the solutions to those problems. period 1994-2000. Of the 21, 13 registered an increase 3. The Bank has issued guidelines to assist Bank in their CPIA ratings over the four-year period, 5 reg- staff in identifying appropriate tools for analyzing the istered a decrease, and 3 did not change. For this poverty and social impacts of development interven- group of 21 countries, the ratio of countries with in- tions (World Bank 2003a). creasing CPIAs to those with decreasing CPIAs was 2.5- 4. OED, in the context of its evaluation of the to-1, which is higher (but not significantly so at the 15 Bank's implementation of IDA, evaluated the CPIA percent level) than the ratio of increases to decreases and the role it plays in IDA's Performance Based Al- in the 113 non-crisis countries. location system (OED 2001e). Although the number 9. Of the 26 transition countries in Europe and Cen- of criteria (20) and the general coverage of most cri- tral Asia (ECA), CPIA ratings increased for 22 and de- teria have remained the same since the 1999 CPIA rat- creased for 4, a ratio of increases to decreases in this ings, there have been changes in the details of some group of countries of over 6-to-1. This is significantly criteria that could affect the inter-temporal compara- greater than the ratio for non-ECA transition countries bility of CPIA ratings. These changes are not consid- of approximately 1.5-to-1.0. ered significant enough to affect the validity of the 10. Annex A provides a brief explanation of each conclusions presented here for two reasons. First, of these policy areas. examination of detailed year-to-year changes for each 11. Strictly speaking, the CPIA and each of its com- of the 20 CPIA criteria does not show any systematic ponent indicators is best interpreted as reflecting rel- patterns that might be reflective of these changes in ative rankings against the pertinent CPIA criteria, criteria. Second, at the aggregate level, the other rather than measures of some quantity. That is, one three indicators examined, which were compiled ac- cannot conclude meaningfully that a country rated cording to unchanging sets of criteria over the period "4.0" on some indicator has policies that are twice as examined, show similar patterns of improvement and good as a country rated "2.0" on the same indicator. reasonable agreement with the CPIA. The comparison of changes across indicators pre- 5. Rank correlations of changes in ratings (i.e., sented in Figure 1.4 is thus best interpreted as an in- correlations of the ranks of changes) vary depending dicator of the number of countries that have improved upon the particular pair of indicators, but 6 of the 6 their policies relative to the criteria for each CPIA di- possible rank correlations are positive, and 3 of the mension. 9 5 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S 12. This decrease does not show in Figure 1.4 due of Bank assistance change. Each year evaluated con- to the scaling of the figure. stitutes a "country assistance year." 13. Annex B, Technical Note 1.3. 6. Regressions of the rating for the latest year of as- 14. Annex B, Technical Note 1.4. sistance evaluated in a CAE--expressed on a scale of 15. The Dollar-Burnside findings on aid-policy in- 1 (highly unsatisfactory) to 6 (highly satisfactory) on teractions are not robust with respect to data or spec- the CPIA corresponding to that year--explain about ification and are the subject of continuing debate in 40 percent of the variation in CAE ratings and are sig- the development research literature. nificant at the 1 percent level. 16. A number of researchers have subjected em- 7. ARDE 2002, pp. 28-29. pirical models of growth to formal tests of robustness, 8. Annex B, Technical Note 2.1. including Sala-I-Martin (1997); Hoover and Perez 9. The CAE was completed before the 2003 crisis (2000); Ley and Steel (1999); Florax, De Groot, and hit the country. Heijungs (2003). While the results obtained depend 10. Factors other than perceived risk--particularly on the specific methods employed, these studies have reorganization and turnover within the Bank--may generally found that relatively few variables are robust have contributed to the Bank's very cautious stance and that, for the most part, these variables are not af- during this period (OED 2003b). fected by policies. 11. The risks and uncertainties addressed through 17. Rodrik (2003b) argues this point and provides the use of triggers are country specific, but triggers fre- numerous case studies. quently address creditworthiness and/or debt sus- tainability risks, uncertainties concerning the direction Chapter 2 and pace of policy reform, or uncertainties concern- 1. As a share of commitments to countries rated in ing the pace of progress in other dimensions (e.g., the CPIA exercise in both 1999 and 2003. portfolio management) of implementation of the 2. The fact that Bank lending is also concentrated country program. on countries that are relatively highly rated by other 12. Deverajan, Dollar, and Holmgren (2001) note indicators suggests that possible feedback effects that political leaders and technocrats may welcome from lending decisions to ratings--i.e., that the Bank conditionality in the early stages of reform as an in- increases the ratings of countries it is lending to--is strument for binding themselves to policy change. not an important factor in this association. They also note, however, that in order for condition- 3. The correlation coefficient between the average ality to play this role, the conditions must be ones that CPIA over the period 1998-2000 with average Bank the borrowing country truly supports. lending per capita during the period FY99-01 is 0.24, 13. Experience with output-based aid is also rele- which is significant at the 1 percent level. The corre- vant in this regard. lation coefficient between the average CPIA over the 14. The Inter-American Development Bank has re- period 2000-02 and average Bank lending per capita cently adopted an investment lending instrument-- over the period FY01-03 is 0.17, which is significant the "performance driven loan"--that links at the 2 percent level. The correlation coefficient be- disbursements to outcomes. This product will be pi- tween the change in the average CPIA between the pe- loted in Costa Rica, at the request of the government riod 2000-02 and 1998-00 and the change in average (IDB 2003). Bank lending per capita between FY01-03 and FY99- 15. Data in the Bank's conditionality data base 01 is 0.09. This would be significant only at the 15 per- (ALCID) show the average number of legally binding cent level. conditions per adjustment operation fluctuating be- 4. The correlation coefficient between the average tween 24 and 34 between FY98 and FY01, and de- CPIA over the period 1998-2000 and average per capita clining to 21 and 18 in FY02 and FY03, respectively. ODA over the period 1999-2001 is 0.05. This would be significant only at about the 30 percent level. Chapter 3 5. CAEs examine the Bank's assistance over a spec- 1. There is no significant statistical difference be- ified period of time. CAEs distinguish among sub-pe- tween the shares of lending accounted for by adjust- riods when the Bank's strategy shifts and/or outcomes ment between the countries showing policy 9 6 E N D N O T E S improvement and those showing policy deteriora- nances. The ICR is prepared at the time of project tion. completion by the staff of the responsible regional of- 2. As of December 31, 2003, the cutoff date for eval- fice (within six months of the final disbursement of the uations reflected in this report. Business Warehouse. Bank loan). It assesses: (a) the degree to which the 3. Active countries include those in which the project achieved its development objectives and out- Bank has an ongoing policy dialogue and a substan- puts as set out in the project documents; (b) other sig- tial lending program. Inactive countries include con- nificant outcomes and impacts; (c) prospects for the flict-affected and inactive borrowers. Members of the project's sustainability; and (d) Bank and borrower per- Organization of Eastern Caribbean States countries are formance, including compliance with relevant Bank exempted from the requirement to have all five di- safeguard and business policies. It also provides the agnostic reports because of their small size. data and analysis to substantiate these assessments, and 4. Most of the increase in the Policy Notes/Other it identifies the lessons learned from implementation. Products category reflects the recent recognition of The borrower prepares and provides to the Bank additional ESW output types (e.g., workshops/con- its own evaluation report on the project's execution ferences and country dialogue/consultations) as prod- and initial operation, its cost and benefits, the Bank's uct deliveries. In FY98 and previously, the outputs and borrower's performance, and the extent to which were generally coded as process tasks in the Bank's the purposes of the loan were achieved. The bor- management information system. rower's report is attached unedited to the ICR. 5. This observation is not unique to China Once sent to the Board of Executive Directors, (OED2003h). each ICR is evaluated by OED, which validates or ad- 6. The Bank's ESW has multiple purposes, as well justs the ratings based on the information provided as other audiences than Chinese officials and re- in the ICR and other operational documents. OED searchers, of course. ESW informs the Board, other summarizes its findings in an ICR Review (formerly donors, and the broader development community of called "Evaluation Summary"). This review conveys the the Bank's assessment of developments in China, and OED ratings, comments on the lessons to be drawn it provides the analytical underpinning for Bank strat- and on the quality of the ICR, and suggests whether egy and lending. the project is a candidate for a Project Performance 7. The fiscal policy component of the CPIA as- Assessment Report (PPAR). sesses the size of the fiscal balance and the composi- The purpose of the PPAR is to validate the findings tion of government revenue and spending to and augment the information in the ICR, and to ex- determine their compatibility with adequate provision amine issues and lessons of broad applicability. Some of public services for economic growth, favorable PPARs are intended to serve as building blocks for macroeconomic outcomes, and a sustainable path of broad sector studies or Country Assistance Evaluations. public debt. The debt management component as- They provide independent, field-based post-comple- sesses whether the country can manage its public tion verification of a project's implementation and debt, external and domestic, and service it now and results. They incorporate the views of the borrower sustainably into the future. See Annex A. and main stakeholders and analyze the operation in 8. Although it was envisaged that SSPs would con- its sectoral and country context. The operational staff tain such operational guidance, few in fact do. and borrower representatives have an opportunity to comment on the draft report. The final report is sub- Chapter 4 mitted to the Bank's Board and is widely distributed 1. See the discussion in chapter 2 concerning con- within the Bank and the borrowing country. ditionality. 3. Seven projects had both an ICR Review and a PPAR completed during this time period. Appendix 4. OED evaluates outcomes by considering three 1. This cohort excludes one FY04 exit project which factors. Relevance refers to the intervention's objec- has been evaluated during this time period. tives in relation to country needs and institutional 2. The Bank prepares an Implementation Com- priorities. Efficacy refers to the extent to which the pletion Report (ICR) for each lending operation it fi- developmental objectives have been (or are expected 9 7 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S to be) achieved. Efficiency measures the extent to Annex A which the objectives have been (or are expected to 1. From World Bank 2003b. be) achieved without using more resources than nec- 2. The other points of the rating scale are "3" (Mod- essary. Combining these three factors, overall out- erately Unsatisfactory) and "4" (Moderately Satisfac- come is rated on a 6-point scale, ranging from highly tory). satisfactory to highly unsatisfactory. 3. From www.eiu.com. 5. The data for FY03 exits represent a partial sam- 4. Heritage Foundation/Wall Street Journal (2003), ple of lending exits (130 out of 293) and reflect all OED 2003 Index of Economic Freedom (Washington, DC: project evaluations through December 31, 2003; this The Heritage Foundation & The Wall Street Journal), partial coverage is noted with dashed lines in all the pp 49-69; figures in this Appendix. 5. The specific rating ranges in the Heritage system 6. OED's sustainability measure assesses the re- are 1-1.95--Free; 2.00-2.95--Mostly Free; 3.00-3.95-- silience of risk of net benefit flows over time by an- Mostly Unfree; and 4.00 and above--Repressed. swering the following questions: At the time of 6. From www.prsgroup.com. evaluation, what is the resilience to risks of future net benefit flows? How sensitive is the intervention to Annex B changes in the operating environment? Will the in- 1. The correlation coefficient is a measure of the tervention continue to produce net benefits as long strength of the linear relationship between two vari- as intended, or even longer? How well will the inter- ables. It may be that Heritage has a non-linear rela- vention weather shocks and changing circumstances? tionship with the other indices, in which case the 7. OED's institutional development impact meas- rank correlation coefficient would be a more ap- ure evaluates the extent to which an intervention im- propriate indicator. Rank correlations for changes in proves the ability of a country or region to make more indicators are reported in Technical Note 1.1. How- efficient, equitable, and sustainable use of its human, ever, investigation shows the conclusions reported financial, and natural resources. Such improvements in the text are robust to the choice of correlation co- can derive from changes in values, customs, laws and efficient. regulations, and organizational mandates. Account- 2. Using the point estimates from the full sample, ability, good governance, the rule of law, and the par- CPIA regression, three policy scenarios are examined: ticipation of the civil society and the private sector are poor policy (CPIA=2.5), good policy (CPIA=4), and prominent characteristics of an effective institutional the case of a country that has switched to good pol- environment. icy (moving from a CPIA of 2.5 to 4.0) and so also ben- 8. These operations exited the Bank's portfolio efited from the associated growth boost of policy during the FY00-03 period, amounted to $5.2 billion reform. For each scenario, the resulting growth is in disbursements, and accounted for 4.3 percent of all shown assuming zero aid and an aid flow of 10 per- the projects and 7.5 percent of all the disbursements cent of national income. The mean value of all other that exited during the same time period. regressors is used for the calculation. 9 8 BIBLIOGRAPHY Adam, Christopher S., and David L. Bevan. 2002. Deininger, Klaus, Lyn Squire, and Swati Basu. "Fiscal Deficits and Growth in Developing 1998. "Does Economic Analysis Improve the Countries." Mimeo. Department of Econom- Quality of Foreign Assistance?" World Bank ics, University of Oxford. Economic Review 12: 385-418. Alesina, Alberto, and David Dollar. 2000 "Who Dicks-Mireaux, Louis, Mauro Mecagni, and Susan Gives Aid to Whom and Why?" Journal of Schadler. 2000. "Evaluating the Effect of IMF Economic Growth 5: 33-64. Lending to Low-Income Countries." Journal Bird, G., and D. Rowlands. 1997. "The Catalytic Ef- of Development Economics 61(2): 495-526. fect of Lending by the International Financial Devarajan, Shantayanan, David Dollar, and Torgny Institutions." World Economy 20 (7): 81-97. Holmgren. 2001. "Overview." In Devarajan, Boone, Peter. 1995. "Politics and the Effectiveness S., D. Dollar, and T. Holmgren (eds.), Aid and of Foreign Aid." National Bureau of Economic Reform in Africa: Lessons from Ten Case Stud- Research, Working Paper Series (US) 5308: ies. Washington, DC: World Bank. 1-56. Dollar, David, and Jakob Svensson. 1998. "What Branson, William, and Nagy Hanna. 2002. "Own- Explains the Success or Failure of Structural ership and Conditionality." OED Working Adjustment Programs?" Policy Research Work- Paper Series. Washington, DC: World Bank. ing Paper 1938. Washington, DC: World Bank. Bredenkamp, Hugh, and Susan Schadler (eds.). Easterly, William. 2001. The Elusive Quest for 1998. Economic Adjustment and Reform in Growth: Economists' Adventures and Mis- Low-Income Countries. Washington, DC: In- adventures in the Tropics. Cambridge, MA: ternational Monetary Fund. MIT Press. Burnside, Craig, and David Dollar. 2000. "Aid, Easterly, William, Ross Levine, and David Rood- Policies, and Growth." American Economic man. 2003. "New Data, New Doubts: A Com- Review 90(4): 847-68. ment on Burnside and Dollar's `Aid, Policies, ------. 2004. "Aid, Policies, and Growth: Revis- and Growth' (2000)." National Bureau of Eco- iting the Evidence." Working Paper 3251. nomic Research. Working Paper Series (US) Washington, DC: World Bank. 9846:1-18. Collier, Paul, and David Dollar. 2002. "Aid Allo- Easterly, William, Michael Kremer, Lant Pritchett, cation and Poverty Reduction." European and Lawrence Summers. 1993. "Good Policy Economic Review 45(1): 1-26. or Good Luck? Country Growth Performance Collier, Paul. 1997. "The Failure of Conditional- and Temporary Shocks." Journal of Mone- ity." In C. Gwin and J. Nelson (eds.), Per- tary Economics 32(3): 458-83. spectives on Aid and Development. Easterly, William, and Sergio T. Rebelo. 1993. Washington, DC: Overseas Development "Fiscal Policy and Economic Growth: An Em- Council. pirical Investigation." Journal of Monetary Collier, Paul et. al. 1997. "Redesigning Condi- Economics 32(3): 417-58. tionality." World Development (UK) 25: 1399- Feyzioglu, T., V. Swaroop, and M. Zhu. 1998. "A 1407. Panel Data Analysis of the Fungibility of For- Cordella, Tito, and Giovanni Dell'Ariccia. 2003. eign Aid." World Bank Economic Review "Budget Support Versus Project Aid." IMF 12(1): 29-58. Working Paper WP/03/88. Washington, DC: Florax, Raymond J. G. M., Henri L. F. de Groot, International Monetary Fund. and Reinout Heijungs. 2003. "The Empirical Dalgaard, Carl-Johan, and Henrik Hansen. 2001. Growth Literature: Robustness, Significance "On Aid, Growth and Good Policies." Journal and Size." Tinbergen Institute Discussion Paper of Development Studies 37(6): 17-41. 02-040/3. Amsterdam: Tinbergen Institute. 9 9 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S Fox, James W. 2003. "Drawing Lessons from World Jayarajah, Carl, and William Branson. 1995. "Struc- Bank PSD Experience." Background paper tural and Sectoral Adjustment: World Bank prepared for ARDE 2003. Washington, DC: Experience, 1980-92." No. 14691. Operations World Bank. Evaluation Department. Washington, DC: Freeland, Chrystia. 2000. Sale of the Century: Rus- World Bank. sia's Wild Ride from Communism to Capi- Kanbur, Ravi. 2000. "Aid, Conditionality and Debt talism. New York: Crown Business. in Africa." In Finn Tarp (ed.), Foreign Aid and Garuda, Gopal. 2000. "The Distributional Effects Development: Lessons Learnt and Directions of IMF Programs: A Cross-Country Analysis." for the Future. New York: Routledge. World Development (UK) 28(6): 1031-51. ------. 2001. "Economics Policy, Distribution Guillaumont, Patrick, and Lisa Chauvet. 2001. and Poverty: The Nature of Disagreements." "Aid and Performance: A Reassessment." Jour- World Development (UK) 29(6): 1083-94. nal of Development Studies 37(6): 66-92. Killick, Tony. 1998. Aid and the Political Econ- Hansen, Henrik, and Finn Tarp. 2000. "Aid Ef- omy of Policy Change. London: Routledge. fectiveness Disputed." Journal of Interna- ------. 2003. Macro-level evaluations and aid ef- tional Development 12(3): 375-98. fectiveness. Mimeo. ------. 2001. "Aid and Growth Regressions." Koeberle, Stefan. 2003. "Should Policy Based Journal of Development Economics 64(2): Lending Still Involve Conditionality?" World 547-70. Bank Research Observer 18(2): 249-73. Helbling, Thomas, Ashoka Mody, and Ratna Kuczynski, Pedro-Pablo, and John Williamson Sahay. "Debt Accumulation in the CIS-7 Coun- (eds.). 2003. After the Washington Consensus: tries: Bad Luck, Bad Policies, or Bad Advice." Restarting Growth and Reform in Latin Amer- Paper presented at the Lucerne conference of ica. Washington, DC: Institute for Interna- the CIS-7 Initiative, Switzerland, January 20- tional Economics. 22, 2003. Lensink, Robert, and Howard White. 2000. "As- Hirschman, Albert O. 1967. Development Projects sessing Aid: A Manifesto for the 21st Century." Observed. Washington DC: The Brookings In- Oxford Development Studies 28(1): 5-17. stitution. Ley, Eduardo, and Mark F. J. Steel. 1999. "We Hoover, Kevin D., and Stephen J. Perez. 2000. Just Averaged Over Two Trillion Cross-Coun- "Truth and Robustness in Cross-Country try Growth Regressions." IMF Working Paper Growth Regressions." UC Davis Economics WP/99/101. Washington, DC: International Working Paper No. 01-01. University of Cali- Monetary Fund. fornia, Davis, California, US. Mody, Ashoka, and Diego Saravia. 2003. "Cat- International Monetary Fund. 2003. World Eco- alyzing Capital Flows: Do IMF-Supported Pro- nomic Outlook: Public Debt in Emerging grams Work as Commitment Devices?" IMF Markets. Washington, DC: International Mon- Working Paper WP/03/100. Washington, DC: etary Fund. International Monetary Fund. IEO (Independent Evaluation Office). 2003. Fis- Mosley, Paul, John Hudson and Sara Horrell. cal Adjustment In IMF Programs. Washington, 2001. "Aid, the Public Sector and the Market DC: International Monetary Fund. in Less Developed Countries." Economic Jour- IDB (Inter-American Development Bank). 2003. nal 97(387): 616-41. IDB Strategy with Costa Rica. Washington, Mosley, F., J. Harrigan, and J. Toye. 1995. Aid DC: Inter-American Development Bank. and Power: The World Bank and Policy- Ivanova, Anna, Wolfgang Mayer, Alex Mour- based Lending. New York: Routledge. mouras, and George Anayiotos. 2003. "What Nelson, Joan M. 2000. "Reforming Social Sector Determines the Implementation of IMF-Sup- Governance." In Joseph S. Tulchin and Allison ported Programs?" IMF Working Paper M. Garland, Social Development in Latin WP/03/8. Washington, DC: International Mon- America. Washington, DC: Woodrow Wilson etary Fund. Center. 1 0 0 B I B L I O G R A P H Y OED (Operations Evaluation Department). ------. 2002k. Zambia Country Assistance Eval- 2000a. Costa Rica Country Assistance Eval- uation. Washington, DC: World Bank. uation. Washington, DC: World Bank. ------. 2003a. Debt Relief for the Poorest: An ------. 2000b. Uruguay Country Assistance OED Review of the HIPC Initiative. Washing- Evaluation. Washington, DC: World Bank. ton, DC: World Bank. ------. 2001a. Annual Review of Development ------. 2003b. Dominican Republic Country As- Effectiveness ­ Making Choices. Washington, sistance Evaluation. Washington, DC: World DC: World Bank. Bank. ------. 2001b. Integrating Gender in World ------. 2003c. Efficient, Sustainable Service for Bank Evaluation. Washington, DC: World All: An OED Review of the World Bank's As- Bank. sistance to Water Supply and Sanitation. ------. 2001c. Kyrgyz Republic Country Assis- Washington, DC: World Bank. tance Evaluation. Washington, DC: World ------. 2003d. Extractive Industries and Sus- Bank. tainable Development: An Evaluation of ------. 2001d. Mexico Country Assistance Eval- World Bank Group Experience, Vol. 2. Wash- uation. Washington, DC: World Bank. ington, DC: World Bank. ------. 2001e. Review of the Performance Based ------. 2003e. Guinea Project Performance Allocation System, IDA 10-12. Washington, Assessment Report: Second, Third, Fourth DC: World Bank. Adjustment Credit; Public Enterprise Sector ------. 2001f. Vietnam Country Assistance Eval- Rationalization and Privatization Technical uation. Washington, DC: World Bank. Assistance Credit; Financial Sector Adjust- ------. 2001g. Yemen Country Assistance Eval- ment Credit. Washington, DC: World Bank. uation. Washington, DC: World Bank. ------. 2003f. Kenya Country Assistance Eval- ------. 2002a. Annual Review of Development uation Update. Washington, DC: World Bank. Effectiveness - Achieving Development Out- ------. 2003g. Mainstreaming Anti-Corruption comes: The Millennium Challenge. Wash- Activities in World Bank Assistance: A Re- ington, DC: World Bank. view of Progress Since 1997. Washington, DC: ------. 2002b. Bulgaria Country Assistance World Bank. Evaluation. Washington, DC: World Bank. ------. 2003h. Power for Development: A Re- ------. 2002c. Chile Country Assistance Eval- view of the World Bank's Experience With uation. Washington, DC: World Bank. Private Participation in the Electricity Sector. ------. 2002d. El Salvador Country Assistance Washington, DC: World Bank. Evaluation. Washington, DC: World Bank. ------. 2003i. Sharing Knowledge: Innovations ------. 2002e. Guatemala Country Assistance and Remaining Challenges. Washington, DC: Evaluation. Washington, DC: World Bank. World Bank. ------. 2002f. Haiti Country Assistance Evalu- ------. 2003j. Toward Country-Led Develop- ation. Washington, DC: World Bank. ment: A Multi-Partner Evaluation of the Com- ------. 2002g. Peru Country Assistance Evalu- prehensive Development Framework. ation. Washington, DC: World Bank. Washington, DC: World Bank. ------. 2002h. Promoting Environmental Sus- ------. 2003k. Zimbabwe Country Assistance tainability in Development: An Evaluation Evaluation. Washington, DC: World Bank. of the World Bank's Performance. Washing- ------. 2004a. Economies in Transition: An ton, DC: World Bank. OED Evaluation of World Bank Assistance. ------. 2002i. The Gender Dimension of Bank Washington, DC: World Bank. Assistance: An Evaluation of Results. Wash- ------. 2004b. Putting Social Development to ington, DC: World Bank. Work for the Poor: An OED Review of World ------. 2002j. Russian Federation: Country As- Bank Activities. Washington, DC: World Bank. sistance Evaluation. Washington, DC: World ------. 2004c. Armenia Country Assistance Bank. Evaluation. Washington, DC: World Bank. 1 0 1 2 0 0 3 A N N U A L R E V I E W O F D E V E L O P M E N T E F F E C T I V E N E S S ------. 2004d. Books, Buildings and Learning sues and Findings." IMF Occasional Paper 128. Outcomes: An Impact Evaluation of World Washington, DC: International Monetary Fund. Bank Support to Basic Education in Ghana. ------. 1995b. "Experience Under Stand-By and Washington, DC: World Bank. Extended Arrangements, Part II: Background ------. 2004e. Brazil Country Assistance Eval- Papers." IMF Occasional Paper 129. Washing- uation. Washington, DC: World Bank. ton, DC: International Monetary Fund. ------. 2004f. China Country Assistance Eval- Sala-I-Martin, Xavier. 1997. "I Just Ran Four Mil- uation. Washington, DC: World Bank. lion Regressions." National Bureau of Eco- ------. 2004g. Croatia Country Assistance Eval- nomic Research, Working Paper Series (US) uation. Washington, DC: World Bank. 6252: 1-21. ------. 2004h. Lithuania Country Assistance Smith, R. Jeffrey. 2003. "GOP's Pressing Question Evaluation. Washington, DC: World Bank. on Medicare Vote: Did Some Go Too Far To ------. 2004i. Mongolia Country Assistance Change a No to a Yes?" The Washington Post, Evaluation. Washington, DC: World Bank. Tuesday, December 23, pg. A01. ------. 2004j. Rwanda Country Assistance Eval- Stiglitz, Joseph E. 2002. Globalization and Its uation. Washington, DC: World Bank. Discontents. New York: W.W. Norton. ------. 2004k. Tunisia Country Assistance Eval- Svensson, Jakob. 2003. "Why Conditional Aid uation. Washington, DC: World Bank. Does Not Work and What Can Be Done About ------. 2004l. Uganda Project Performance As- It?" Journal of Development Economics 70(2): sessment Report: Primary Education and 381-402. Teacher Development Project; Education Sec- Thomas, Alun. Forthcoming. Prior Actions - True tor Adjustment Credit. Washington, DC: World Repentance? An Evaluation Based on IMF Bank. Programs Over the 1992-1999 Period. IMF Pack, Howard. 1993. "Foreign Aid and the Ques- Working Paper. Washington, DC: International tion of Fungibility." Review of Economics and Monetary Fund. Statistics 75: 258-65. Thomas, M.A. 2003. Can the World Bank Enforce QAG (Quality Assurance Group). 2003. Quality its Own Conditions? Working Paper Series. of ESW in FY02: A QAG Assessment. Washing- Washington, DC: World Learning. ton, DC: World Bank. UNDP (United Nations Development Program). Rodrik, Dani. 2002. "After Neoliberalism, What?" 2003. Development Effectiveness Report 2003: Notes for a presentation at the BNDES sem- Partnership for Results. UNDP Evaluation Of- inar on "New Paths of Development," Rio de fice. New York, NY. Janeiro, Brazil, September 12-13, 2002. White, Howard N. 2001. "Will the New Aid Journal of International Development (UK) Rodrik, Dani. 2003a. "Economic Reform Without 13(7): 1057-70. Rules of Thumb." Comments made at a Wolfensohn, James. 2002. "PRSP Process ­ Cre- Festschrift conference for Joseph Stiglitz, Co- ating Better Lives for Poor People." Opening lumbia University, New York, NY, October 24- Remark at the International Conference on 25, 2003. Poverty Reduction Strategies, Washington, Rodrik, Dani (ed.). 2003b. In Search of Pros- DC, January 14-17, 2002. perity: Analytic Narratives on Economic World Bank. 1991. World Development Report Growth. New Jersey: Princeton University 1991: The Challenge of Development. Wash- Press. ington, DC: World Bank. Schadler, Susan, Adam Bennett, Maria Carkovic, ------. 1997. World Development Report 1997: Louis Dicks-Mireaux, Mauro Mecagni, James The State in a Changing World. Washington, H.J. Morsink, and Miguel A. Savastano. 1995a. DC: World Bank. "IMF Conditionality: Experience Under Stand- ------. 2000. Supporting Country Development: By and Extended Arrangements, Part I: Key Is- World Bank Role and Instruments in Low- 1 0 2 B I B L I O G R A P H Y and Middle-Income Countries. Washington, ------. 2003c. Progress on Economic and Sec- DC: World Bank. tor Work. IDA/R2003-0193. Report to IDA 13 ------. 2001. Adjustment Lending Retrospec- Deputies. Washington, DC: World Bank. tive. Washington, DC: World Bank. ------. 2003d. Supporting Sound Policies with ------. 2002. Transition ­ The First Ten Years: Adequate and Appropriate Financing. Wash- Analysis and Lessons for Eastern Europe and ington, DC: World Bank. the Former Soviet Union. Washington, DC: ------. 2004. Doing Business in 2004: Under- World Bank. standing Regulation. Private Sector Devel- ------. 2003a. A User's Guide to Poverty and So- opment Group. Washington, DC: World Bank. cial Impact Analysis. Poverty Reduction Zongo, Tersius, Siaka Coulibaly, Gilles Hervio, Group and Social Development Department. Javier Nino Perez, and Stefan Emblad. 2000. Washington, DC: World Bank. Conditionality Reform: The Burkina Faso ------. 2003b. Country Policy and Institutional Pilot Case. Brussels: SPA Task Team on Con- Assessment 2003: Assessment Questionnaire. tractual Relationships and Selectivity. Washington, D.C: World Bank. 1 0 3 OPERATIONS EVALUATION DEPARTMENT ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. TMxHSKIMBy359389zv":&:*:.:> THE WORLD BANK ISBN 0-8213-5938-X