56456 The World Bank's Country Policy and Institutional Assessment An IEG Evaluation THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions--the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their environment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP IMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on assessment of IFC's work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to the Bank's Board of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group's work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. The World Bank's Country Policy and Institutional Assessment An IEG Evaluation 2010 The World Bank http://www.worldbank.org/ieg Washington, D.C. © The International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org E-mail: feedback@worldbank.org All rights reserved 1 2 3 4 13 12 11 10 This volume is a product of the staff of the International Bank for Reconstruction and Development / The World Bank Group. The findings, interpre- tations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the govern- ments they represent. The World Bank does not guarantee the accuracy of the data in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / The World Bank encourages a dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyrights.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H , Street, NW Washington, DC 20433, USA: fax:202-522-2422; email:pubrights@worldbank.org. Cover image: Unisphere at Flushing Meadows ­ Corona Park. New York, New York. © Rudy Sulgan/Corbis. The Unisphere was built as the theme symbol for the 1964­65 World's Fair. ISBN: 978-0-8213-8427-5 e-ISBN: 978-0-8213-8429-9 DOI: 1596/978-0-8213-8427-5 Library of Congress Cataloging-in-Publication data have been applied for. World Bank InfoShop Independent Evaluation Group E-mail: pic@worldbank.org Communications, Strategy, and Learning Telephone: 202-458-5454 E-mail: ieg@worldbank.org Facsimile: 202-522-1500 Telephone: 202-458-4497 Facsimile: 202-522-3125 Printed on Recycled Paper Contents vii Abbreviations ix Acknowledgments xi Foreword xiii Executive Summary xix Management Response xxv Chairman's Summary: Committee on Development Effectiveness (CODE) xxvii Advisory Panel Statement 1 1 Introduction and Evolution of the CPIA 4 Evolution in the Content of the CPIA 8 Other Changes in the CPIA 9 Role of the CPIA in IDA Allocation 13 2 Relevance of the CPIA for Growth, Poverty Reduction, and Effective Use of Development Assistance 15 The CPIA and Determinants of Sustained Growth 25 The CPIA and Determinants of Poverty Reduction 29 The CPIA and the Effective Use of Development Assistance 31 Findings and Recommendations 39 3 Reliability of the CPIA Ratings 41 Comparability with Other Indicators 48 CPIA Ratings Generation Process 57 4 Findings and Recommendations 59 Overview 59 Main Findings 62 Recommendations 65 Appendixes 67 A: 2008 Country Policy and Institutional Assessment Criteria 71 B: Public Sector Literature Review 81 C: 2007 CPIA Criteria on Economic Management, Structural Policies, and Policies for Social Inclusion/Equity and Evidence in the Literature 91 D: Literature Review on Aid Effectiveness 93 E: Examples of Positive Impacts of Aid Projects from Randomized Evaluations in Education, Health, Infrastructure, and Agriculture 95 F: Review of Financial Sector Criterion 99 G: Country Policy and Institutional Assessment (CPIA) and Loan Performance iii THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT 103 H: Comparing Country Policy and Institutional Assessment Ratings by the World Bank, African Development Bank, and Asian Development Bank 105 I: Comparator Indices of the CPIA 109 J: Number of IDA and IBRD Countries for Which External Data are Available 111 K: Comments by Network on Regional CPIA Rating Proposals, 2007 113 Endnotes 119 References Boxes 4 1.1 Changes to the CPIA Criteria in 1998 That Reflect the Emphasis on Institutions 11 1.2 The Governance Adjustment in IDA's Country Performance Ratings, FY1998­2008 19 2.1 Channels through Which Inequality Affects Growth 44 3.1 Comparator Indicators 50 3.2 The Process of Preparing CPIA Ratings 51 3.3 The Network Reviews of CPIA Ratings Figure 25 2.1 Relationship between Changes in PBA and the Ratio of Cluster D Ratings to Ratings on Other Clusters for "Core IDA" Countries Tables 5 1.1 CPIA Criteria 1998­2008 10 1.2 Evolution of IDA's Performance-Based Allocation Formula and the Adjustment for Governance 16 2.1 Mapping of the "Consensus" Determinants for Sustained Growth and the CPIA Criteria 22 2.2 Correlations between Ratings of CPIA Clusters, 2007 22 2.3 Correlations between CPIA with Different Cluster Weights and CPIA with Equal Cluster Weights, 2007 23 2.4 Simulation Results: Effects on Performance-Based Allocations for "Core IDA" Countries Arising from a Larger Weight on the "Governance" Cluster Compared to Equal Weights on All Clusters 33 2.5 Simulation Results: Changes in PBA from Dropping q8 34 2.6 Simulation Results: Changes in Trade Ratings Arising from Changes in Weights of Trade Subcriteria by Numbers and Shares of IBRD and IDA Countries 35 2.7 Simulation Results: IBRD and IDA Countries That Would Experience Changes in Trade Ratings due to Changes in Weights for Trade Subcriteria 37 2.8 Simulation Results: Changes in PBA Arising from Changes in Weights for Trade Subcriteria 42 3.1 Rank Correlation Coefficients between CPIA Ratings and Comparator Ratings for 2007 45 3.2 Other Indicators--Expert Judgment or Hard Data? 46 3.3 Are Other Comparator Indicators Closer to the Bank or to AfDB and ADB? 47 3.4 Comparison of Changes in CPIA Ratings 2006­07 between the Bank, AfDB, and ADB iv CONTENTS 49 3.5 Rank Correlations between the CPIA and Other Indicators: IBRD versus IDA Countries 52 3.6 Numbers and Shares of Initial Regional Rating Proposals on Which the Networks Disagreed with the Regions, by Criteria, for 2007 53 3.7 Number and Share of Initial Regional Rating Proposals on Which the Networks Disagreed with the Regions, by Region and IBRD and IDA Countries for 2007 53 3.8 Network Disagreements with Initial Regional Rating Proposals 54 3.9 Share of Instances Where Networks Prevailed When Networks Disagreed with Regions over Proposed Increases in Ratings from 2006 54 3.10 Share of Instances Where Networks Prevailed When Networks Disagreed with Regions, 2007 55 3.11 Shares of Countries on Which Networks Commented in 2007 When Regions Proposed the Same Ratings as in 2006 v Energy transmission and power lines, Tajikistan. Photo by Gennadiy Ratushenko/World Bank vi Abbreviations ADB Asian Development Bank AfDB African Development Bank AML-CFT Anti-money laundering and combating the financing of terrorists ARPP Annual review of project performance BCP Basel core principles CGAP Consultative Group to Assist the Poor CPIA Country policy and institutional assessment CPR Country performance ratings DAC Development Assistance Committee of the Organisation for Economic Co-operation and Development DB Doing Business ESI Environmental Sustainability Index ETI Enabling Trade Index GCI Global Competitiveness Index GDP Gross domestic product GNIpc Gross national income per capita GNPpc Gross national product per capita HIV-AIDS Human immunodeficiency virus-acquired immunodeficiency syndrome IBRD International Bank for Reconstruction and Development ICP IDA country performance ICRG International Country Risk Guide IDA International Development Association IEG Independent Evaluation Group IMF International Monetary Fund IP Implementation progress ISR Implementation Status and Results Report IT Information technology MDGs Millennium Development Goals NGOs Non-governmental organizations NPLs Non-performing loans NTB Non-tariff barrier OECD Organisation for Economic Co-operation and Development OED Operations Evaluation Department (the previous name of IEG) OLS Ordinary least squares OPCS Operations Policy and Country Services PBA Performance-based allocation PRMED Economic Policy and Debt Department QAG Quality Assurance Group SMEs Small and medium enterprises UNDP United Nations Development Programme VAT Value-added tax WGI Worldwide Governance Indicators vii Harvesting rice fields, Vietnam. Photo by Tran Thi Hoa/World Bank Acknowledgments This evaluation of the World Bank's Country by Agnes Santos. Barbara Balaj copy edited the Policy and Institutional Assessment was study for publication. prepared by the Country Evaluation and Regional Relations division of the Independent The peer reviewers were Victoria Elliott, Aart Evaluation Group. Kraay, and Steven Webb. The team received guidance and support from the IEG management The evaluation was led by Helena Tang, the team, and also benefited from the comments author of the report. Saubhik Deb carried out received from the various IEG units. the statistical and econometric analyses for the report. Background papers were prepared by The evaluation team extends its thanks to the Julia Cage, Shailaja Fennel, Patrick Honohan, and staff from all six Regions, the various networks, Biaoyun Qiao. Inputs were provided by Carla Operations Policy and Country Services, and Pazce, Rupa Ranganathan, and Dusan Vujovic. Resource Mobilization who were interviewed for Production and logistical support was provided the report. Director-General, Evaluation: Vinod Thomas Director, Independent Evaluation Group ­ World Bank: Cheryl Gray Senior Manager, IEGCR: Ali M. Khadr Task Manager: Helena Tang ix Woman in India. Photo by Curt Carnemark/World Bank Foreword The Bank's Country Policy and Institutional narrower sense--specifically, the performance of Assessment (CPIA) assesses the conduciveness Bank loans. of a country's policy and institutional framework to poverty reduction, sustainable growth, and The CPIA's 16 criteria are grouped into four the effective use of development assistance. The clusters--economic management, structural CPIA enters the calculation of country perfor- policies, policies for social inclusion and equity, mance ratings that, since 1980, have been used and public sector management and institutions. to allocate International Development Associa- These clusters are weighted equally to derive the tion (IDA) resources to eligible client countries. overall CPIA rating. In contrast, the IDA allocation This evaluation was undertaken at the request of formula weights the clusters unevenly--the first Board members to assess the appropriateness three clusters are each given a weight of 8 percent, of the CPIA as a broad indicator of development and the last cluster (the governance cluster) a effectiveness and as a determinant of the alloca- weight of 68 percent and portfolio performance tion of IDA funds. As indicated in the Approach the remaining weight of 8 percent. The literature Paper, this evaluation reviews the effects of the offers no evidence to justify any particular set of CPIA ratings on IDA allocations but does not weights on the four clusters, whether in deriving review the IDA allocation formula itself. the overall CPIA rating or in calculating the IDA allocation. Neither is there justification for why The evaluation finds that the CPIA content the clustering is as it is--having all social sectors broadly reflects the determinants of economic combined with the environment in one cluster, growth and poverty reduction identified in the for example. There is also insufficient evidence economics literature, but some criteria need to conclude that the governance cluster associ- to be revised (such as the trade criterion that ates better with Bank loan performance than the places much greater emphasis on imports than other three clusters. exports) and streamlined, and one criterion (assessment of disadvantaged socioeconomic The report lays out four recommendations: categories other than gender) added. The CPIA disclose International Bank for Reconstruction ratings also correlate well with ratings of similar and Development ratings; discontinue the "stage indicators, and more so for International Bank of development" adjustment to the ratings; for Reconstruction and Development than for review and revise the content and clustering of IDA countries. In part, this could be caused by the criteria; and discontinue the current aggrega- the CPIA exercise's practice over the past several tion of the criteria into an overall index. years of taking into account a country's stage of development, which also means that the CPIA is no longer an index in the true sense of the word. It is difficult to establish an empirical link between the CPIA and economic growth outcomes, although CPIA ratings are found to be Vinod Thomas positively associated with aid effectiveness in the Director-General, Evaluation xi Girl in classroom, Mexico. Photo by Curt Carnemark/World Bank Executive Summary T he World Bank's Country Policy and Institutional Assessment (CPIA) assesses the conduciveness of a country's policy and institutional framework to poverty reduction, sustainable growth, and the effec- tive use of development assistance. It plays an important role in the country performance ratings that have been used for allocating resources from the International Development Association (IDA) to eligible countries since 1980. The CPIA consists of 16 criteria grouped into four regarding the relevance of the content of the clusters--economic management, structural CPIA for aid effectiveness broadly defined--that policies, policies for social inclusion and equity, is, that it represents the policies and institutions and public sector management and institutions-- important for aid to lead to growth. However, the weighted equally to derive the overall CPIA CPIA is associated with aid effectiveness defined rating. Since the beginning of fiscal 2009, IDA more narrowly as the better performance of has made transparent the weights of the clusters Bank loans. But there is insufficient evidence to used in the IDA allocation formula--24 percent conclude that the most heavily weighted CPIA on the first three CPIA clusters combined and 68 cluster associates better with loan performance percent on the fourth (governance) cluster, with than the other three clusters. the remaining 8 percent weighted on portfolio performance. In other words, the governance The CPIA ratings are generally reliable and cluster has eight and a half times the weight of correlate well with similar indicators, but it is each of the other three clusters in the formula. difficult to establish an empirical link between the This has also made transparent the weak link CPIA and growth outcomes. Network reviewers' between the overall CPIA index and IDA alloca- validation of ratings helps guard against potential tions, with a country's governance performance biases in having Bank staff rate countries on (particularly relative to its performance in the which their work programs depend. The CPIA other clusters) being more important in the ratings correlate better with similar indicators latter. for International Bank for Reconstruction and Development (IBRD) than for IDA countries. The content of the CPIA broadly reflects the This correlation could in part be because more determinants of growth and poverty reduction information is available on IBRD countries, identified in the economics literature. However, and in part because the CPIA ratings are meant some criteria need to be revised and streamlined to take into account the stage of development and one criterion added. The literature offers no (which is more pertinent for IDA countries and evidence to justify any particular set of weights which means ratings for these countries are on the four clusters used for IDA allocation or more subject to judgment than those for IBRD the way the criteria are clustered (for example, countries). This tendency is exacerbated by the having social sectors and environment in one different practices with respect to accounting for cluster). The literature offers only mixed evidence the stage of development, as none of the regional xiii THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT reviewers of the CPIA do this, whereas network indicates there is little consensus on the impact reviewers vary in their practices. of aid on growth itself and on the conditions under which aid can have a positive impact on The International Evaluation Group (IEG) makes growth. four recommendations. First, disclose the ratings for IBRD countries in the interest of accountabil- However, the CPIA is associated with aid ity and transparency. Second, remove account- effectiveness in a narrower sense--that is, ing for the stage of development in the rating with respect to the performance of World exercise to reduce subjectivity. Third, undertake Bank loans. Empirical analysis finds that the a thorough review of the adequacy of each overall CPIA ratings are negatively associated criterion, including a review of experience and with the share of problem loans that in turn is the literature, and revise as necessary, based on, correlated with loan outcomes. among other things, the findings of this evalua- tion. Fourth, consider not producing an overall Empirical analysis indicates that there CPIA index although continue to produce and is insufficient evidence to conclude that publish the separate CPIA components. the governance cluster associates better with loan performance than with the Overview other clusters. Based on this finding, as well This evaluation takes the premise that beyond as the lack of consensus in the literature on the informing IDA allocation, the CPIA is useful as conditions under which aid has an impact on a broad indicator of development effectiveness. growth, it can be surmised that the way in which It reviews the appropriateness of the CPIA as the CPIA enters the formula for the allocation an indicator that assesses the conduciveness of of IDA funds is driven much more by fiduciary a country's policies and institutions to foster- and possibly other concerns of donors than by ing poverty reduction, sustainable growth, and the objectives of achieving sustained growth and the effective use of development assistance. It poverty reduction. assesses the relevance of the content of the CPIA through a review of the economics literature. It The CPIA strives to allow for country also assesses the reliability of CPIA ratings in two specificity, that is, that different sets of ways-- through comparing CPIA ratings with policies and institutions can achieve similar indicators, and through reviewing the similar outcomes. However, there are CPIA ratings generation process. Based on these some pitfalls. The CPIA instructions to staff assessments, the evaluation derives recommen- indicate that outcomes need to be taken into dations for enhancing the CPIA. account when assessing policies and institu- tions, which helps to account for country Relevance of CPIA specificity. Indeed, outcome indicators are The contents of the CPIA are largely included in the assessment of some criteria. relevant for growth and poverty reduction. They could also be added to other criteria, in The CPIA criteria map well with the determi- particular trade. nants--policies and institutions--of growth and poverty reduction identified in the literature, The trade criterion does not adequately although some criteria can usefully be revised allow for country specificity. The specifica- and streamlined and one can be added (see tion of particular tariff rates for different ratings recommendations). reflects a one-size-fits-all approach to trade liberalization that is not supported by country The evidence is mixed regarding the experience. Export performance (an outcome relevance of the content of the CPIA for indicator) needs to be included in the assess- aid effectiveness as broadly defined in the ment and would help to allow for country literature. Indeed, the review of the literature specificity. xiv EXECUTIVE SUMMARY The trade criterion also does not reflect those of the African Development Bank and the the importance of complementary institu- Asian Development Bank, the closest compara- tions for successful liberalization. The tors to the Bank, as they use almost exactly the two-thirds weight on trade restrictiveness and same CPIA guidelines. one-third weight on trade facilitation is not supported by country experience that shows CPIA ratings correlate better with similar that at moderate tariff levels (which practically indicators for IBRD than for IDA countries. all countries currently have), complementary This could be due in part to the greater amount factors (macroeconomic stability and trade facili- of information available on IBRD than on IDA tation) are more important than further tariff countries, which increases the likelihood of reduction to promote integration into the global different institutions having similar assessments economy. on IBRD countries. It could also be due partly to the fact that the CPIA rating exercise takes into The CPIA is missing an assessment of account the stage of development (introduced disadvantaged socioeconomic groups since 2004). This is more pertinent for IDA other than gender. Currently, only gender countries and hence would subject ratings of is being assessed with respect to equality, yet those countries to more judgment in an exercise country evidence indicates that social exclusion that is already centered on staff judgment. of other marginalized groups could have severe poverty and growth implications. Accounting for the stage of development in the CPIA ratings is problematic. In Important linkages among certain criteria addition to the judgment involved, accounting are not reflected in the CPIA. Except for the for the stage of development is also problematic three economic management criteria, all of the because of the different practices adopted across CPIA criteria are assessed independently. This the Bank. Regional reviewers do not take this could be problematic in two instances. First, the into account, whereas network reviewers vary in assessment of trade liberalization needs to take their practices. Further, accounting for the stage into account the extent of intersectoral labor of development means that the CPIA is no longer mobility because liberalization in the absence of an index in the true sense of the word. labor mobility could exacerbate poverty. Second, fiscal policy needs to be assessed in conjunc- The review process for the CPIA, which gives tion with the quality of budgetary and financial the networks responsibility for validat- management to ensure that the fiscal condition of ing the ratings, helps to guard against the country in its entirety is realistically captured. potential biases in ratings, although there are exceptions. A major advantage of the CPIA Reliability of CPIA Ratings exercise is having well-informed professional The Bank has made efforts over time judgment of staff as the central determinant of to improve the definition of the CPIA the ratings. At the same time, however, having rating scale to enhance the reliability of staff rate the countries on which their work the ratings. These efforts have been aimed at programs depend could lead to rating biases. reducing staff discretion in providing ratings. Analysis of the 2007 review process indicates that The CPIA ratings correlate well with similar for instances where the networks challenged indicators in terms of relative rankings of the regions' initial proposals of a rating increase countries and direction of change. For each from 2006, the networks prevailed 73 percent of the 16 criteria, the rank correlation coefficients of the time for IDA countries. They prevailed of CPIA ratings with similar indicators average more often--86 percent of the time--for IBRD between 0.7 and 0.8. Other indicators correlate countries. However, these instances made up better with the Bank's CPIA ratings than with only 6 percent of the ratings for IDA countries xv THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT and 5 percent of the ratings for IBRD countries; environment, which limit the emphasis accorded hence, there does not seem to be a strong upward to these aspects. Guideposts for assessing the bias in ratings for either group of countries. criteria need to be reviewed at the same time. The following points also need to be taken into Recommendations account in the review and revisions: Based on its findings, IEG has derived recommen- dations to enhance the CPIA as an indicator of · Revising the trade criterion to include a sub- policies and institutions that are important for component on exports that evaluates perfor- growth, poverty reduction (or welfare more mance as well as policies and institutions to broadly), and the effective use of development reduce anti-export bias. This subcomponent assistance. and those on trade restrictiveness and trade facilitation need to all get equal weights. The Adoption of these recommendations could result trade restrictiveness subcomponent needs to in a discontinuity in the CPIA ratings, which be revised to reflect country experience that Bank management has been trying to avoid. at moderate levels of tariffs (which almost all However, it is important that the CPIA reflect the countries have), any further reduction would latest thinking in development as well as lessons be less important than complementary factors learned--both of which are stated intentions of for global integration. the Bank. It would also provide the opportunity · Dropping or reformulating the criterion on to address an issue that some network reviewers equity of public resource use, as much of its have raised regarding the quality of the ratings content is already covered by other CPIA cri- for some criteria because of what they perceive teria (specifically, property rights, access to as inflated baseline ratings from a few years ago. education and to credit, income transfers) or The recommendations are as follows. information is lacking for an adequate assess- ment (specifically, the progressivity or regres- First, disclose the ratings for IBRD sivity of taxes). countries. Disclosure is important for account- · Adding an assessment of other disadvantaged ability and transparency and would further socioeconomic groups to the CPIA. This could enhance the quality of the ratings. either replace the criterion on equity of public resource use or be added to that criterion if it Second, remove accounting for the stage of were to be reformulated. development from the CPIA exercise. If this · Revising the financial sector criterion. This cannot be done, at the very least it is important to needs to entail (i) revising of the weights for clarify and justify in the guidelines which criteria the three subcomponents--stability, depth need to take into account the stage of develop- and efficiency, and access--in light of the im- ment and how such adjustments need to be portance of financial stability as reflected by re- made. cent global evidence, and the mixed evidence on the importance of microfinance; (ii) adding Third, undertake a thorough review of the assessment of policies, regulations, and insti- CPIA and revise the criteria as necessary. tutions for fostering an enabling environment IEG recommends that the review entail an for the financial sector taking into account in-depth literature review for each criterion and lessons learned, notably from the current cri- reflect the latest thinking on development and sis; and (iii) strengthening the assessment of lessons learned. The review needs to reflect an financial stability. appropriate balance between liberalization and · Combining the assessment of tax policy with regulation. The review needs to also examine fiscal policy. whether the clustering of criteria is appropriate. · Streamlining the assessment of judicial inde- In particular, it needs to examine the appropri- pendence and the assessment of corruption in ateness of combining the social sectors with the the public sector management and institutions xvi EXECUTIVE SUMMARY cluster, as they are currently assessed in more publish the separate CPIA components. than one criterion in this cluster. The overall CPIA index is not used as such for · Strengthening the assessment of the environ- the allocation of IDA funds. With respect to the ment criterion and making the process more broader use of the CPIA as an index of policies and efficient. Currently, staff need to answer 85 institutions, country specificity implies that the questions for only one rating. appropriate weights of the different clusters could · Reporting only one consolidated rating for differ depending on a country's initial conditions the three economic management criteria to and stage of development. Producing the differ- avoid confusion. ent components of the CPIA without assigning weights to them to arrive at an aggregate index Fourth, consider not producing an overall would allow for different weights to be applied CPIA index, and continue to produce and according to country contexts and use. xvii Students building houses in Kahyelitsha township, South Africa. Photo by Trevor Samson/World Bank Management Response M anagement welcomes the Independent Evaluation Group (IEG) report on the World Bank's Country Policy and Institutional As- sessment (CPIA). In management's view the findings of the review include several useful insights that will contribute to further strengthen the CPIA. The report makes four recommendations: the conditions, including the role of policies and institutions, under which aid can influence · Disclose the ratings for International Bank growth. The IEG evaluation finds, however, that for Reconstruction and Development (IBRD) the CPIA is associated with aid effectiveness in the countries. narrower context of the performance of World · Remove accounting for the stage of develop- Bank loans. Poor CPIA scores are correlated ment from the CPIA exercise, or, if this cannot with the share of problem loans, which in turn is be done, clarify and justify in the guidelines correlated with loan outcomes. which criteria should be subject to the ad- justments and how the adjustments should CPIA criteria. The evaluation also contains be made. recommendations on a few CPIA criteria, such · Undertake a thorough review of the CPIA and as the criteria covering trade, the financial sector, revise the criteria as needed (the evaluation and the equity of public resource use, which contains recommendations regarding a few IEG finds could be streamlined and revisited. specific criteria, such as trade and financial Management considers these recommenda- sector). tions useful and intends to use them to inform · Consider not producing an overall CPIA index the next review of the CPIA. After assessing although continuing to produce and publish gaps in coverage, the IEG evaluation notes that the separate CPIA components. Except for the CPIA is missing an assessment of disadvan- the recommendation on disclosing the CPIA taged socioeconomic groups other than gender. ratings for the IBRD countries, management Management intends to address this issue in the broadly concurs with the recommendations context of the CPIA review. emanating from this evaluation. Reliability. The IEG evaluation notes the efforts Relevance of the CPIA. The evaluation finds the Bank has made over time to strengthen the that the contents of the CPIA are largely relevant CPIA and enhance the reliability of the scores. It for growth and poverty reduction and that they finds that in terms of relative ranking and directions map well with the policies and institutions that are of change, the CPIA scores are correlated well identified in the literature as relevant for growth with existing indicators, but it notes that the and poverty reduction. On the basis of a review correlations are higher for IBRD than for Interna- of the literature, the IEG evaluation concludes tional Development Association (IDA) countries. that there is little consensus on the impact of The report also analyzes the process used by aid on growth and poverty reduction and on the World Bank to generate the CPIA scores--a xix THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT process in which the regions put forward a set of Management disagrees with the recommenda- proposals for country scores that are then subject tion to disclose the IBRD scores and prefers to review by the networks and central depart- restricting the coverage of the CPIA exercise to ments. IEG finds that this internal review process the IDA-only countries. In the context of the gives the networks an important role in validat- forthcoming CPIA review, management will ing the scores, helping to prevent potential bias analyze in more depth the value added and the in the scores and to address possible conflicts of costs of preparing CPIA scores for IBRD countries interest. The review concluded that there is no for internal Bank uses, as well as other relevant strong evidence of upward bias for either the IDA aspects. The conclusions of this work will inform or IBRD country scores. Management welcomes management's decision on how to go forward, these findings and views them as useful inputs for namely regarding the coverage of the CPIA, and, further strengthening the CPIA process. if warranted, the consideration of alternative approaches to disclosure. In the meantime the General Comments CPIA exercise will continue to cover the IBRD Disclosure of IBRD scores. The objective of countries. the CPIA exercise is first and foremost to provide an assessment of country performance that will be Accounting for development stage . The used to determine IDA allocations. To underscore CPIA guidelines state that staff may need to take this point, by the suggestion of the Board, these into account the size of the economy and its scores are disclosed as the IDA Resource Alloca- degree of sophistication in their assessments. tion Index. IEG argues that disclosure of the The criteria were developed so that higher IBRD scores is important from an accountability scores could be attained by a country that, and transparency standpoint and will strengthen given its stage of development, has a policy and the ratings. The report neither elaborates on the institutional framework that fosters growth and argument nor discusses trade-offs. Accountabil- poverty reduction. This approach recognizes ity and transparency are important, but there are that in many areas, countries cannot be judged other issues to consider. by the same yardstick if they are at very differ- ent stages of development. Some of the policy A major reason not to disclose the IBRD scores objectives may be considered to be invariant to is the possible effect on market perceptions income--for example, the desirability of having and credit ratings and associated financial a well-managed budget. But others depend, for consequences for the countries concerned. example, on the sophistication of the financial Moreover, the Bank would not want to be seen as system (expectations regarding regulatory a credit rating agency. Unlike the scores for IDA capacity would be different for a high-income countries, the scores are not discussed or shared country than for a low-income country) or on by Bank staff with their IBRD counterparts; the the degree of urbanization. Social protection in a IBRD country scores do not play a role in lending largely urban, formal economy (unemployment decisions, and their confidentiality limits their insurance, pensions, and so on) is fundamentally use. They have been used internally in analytic different from the problem of protecting a poor work and by the Quality Assurance Group and rural subsistence economy from weather-related IEG on portfolio-related issues. When the 2004 harvest shocks. external panel reviewed the CPIA and discussed these issues, it leaned toward dropping the IBRD The report raises a number of concerns regard- countries from the exercise. IEG notes (chapter ing the CPIA treatment of the stage of develop- 4) that the report recommendations are aimed at ment. At the same time, the evaluation and the enhancing the CPIA beyond its use for IDA alloca- recommendations (including those concerning tions, and that if the CPIA is viewed only in an the revision of the financial sector criterion) IDA-allocation context, the need to include IBRD recognize that stage of development consid- countries can be questioned. erations are important (appendix box F.1). xx MANAGEMENT RESPONSE Unless this dimension is considered, some of the time IDA 16 is launched. Management wishes the criteria scores may be linearly correlated to point out, however, several important consid- with income--which is not the objective of the erations to take into account in planning the exercise. Controlling for a country's stage of timing of the review. First, in revising the CPIA, development seems necessary, as what consti- it is important to balance making the instrument tutes good policy in many of the areas covered flexible enough to reflect new developments with by the CPIA is linked to stage of development maintaining some stability in the criteria that as well as to country-specific characteristics. The will allow for comparisons of scores over time. report points out that accounting for the stage Revisions will create another break in the CPIA of development in the CPIA exercise may not series, and, as in 2004, there will be substantial cost always have been uniformly applied. It suggests in reworking the country scores and in explain- that, if the approach continues to be used, ing to the governments and external audiences the guidelines should clarify and provide the the new criteria, the differences in relation to rationale for its use in specific criteria, showing the previous criteria, and the rationale for the how the adjustment should be carried out in changes. Second, following the introduction of determining the final scores. Management a new set of criteria, changes in some scores do agrees with this recommendation and in the not necessarily reflect a deterioration or improve- context of the review of the CPIA will revise the ment in performance, but result from the changes guidelines accordingly. in the criteria. Because the scores are used for IDA allocations, the revisions of the criteria could Review of the CPIA criteria. Periodic reviews result in aid volatility. And third, the CPIA criteria of the CPIA to update and refine the content of are used by other multilateral development banks, the criteria and the conduct of the exercise have and management also intends to consult them been a mainstay of the CPIA's history, and they throughout the process of revising the instru- should continue to remain so going forward. But ment. Management would add, however, with these reviews should also be done at sufficient respect to the IEG report's detailed recommenda- intervals so that the CPIA scores have some tions on how some criteria could be revised, that it validity over time. Consensus on development finds these suggestions useful and intends to use thinking moves slowly. As the IEG report notes, them to inform the next revision. these periodic reviews resulted in several breaks in the CPIA series, as some criteria were dropped, Caveats. Although management broadly agrees some were added, and some were revised. As the with the thrust of the findings of the IEG evalua- report notes, the last major revision took place tion, it would like to point out that the report in 2004, informed by the recommendations of an contains a few examples of statements--specifi- external panel that undertook an in-depth review cally, regarding the interpretation of some of of the CPIA. the findings--that would have benefited from further elaboration or qualification. Overall, The IEG report suggests that perhaps the time management agrees with most of the IEG has come for Bank management to undertake a findings and, with the exception noted above, thorough review and revision of the CPIA. Manage- accepts its recommendations. Management's ment generally concurs with this suggestion and specific responses to the IEG recommendations plans a revision of the CPIA, to be completed by are given in the Management Action Record. xxi THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Management Action Record Independent Evaluation Group (IEG) Recommendations Requiring a Response Management Response Disclose ratings for Disagree. The objective of the Country Policy and Institutional Assessment (CPIA) exercise International Bank for is first and foremost to provide an assessment of country performance that will be used Reconstruction and in determining International Development Association (IDA) allocations. IEG argues Development (IBRD) without elaboration that disclosure of the IBRD ratings is important for accountability and countries. transparency and would further enhance the quality of the ratings. Whether "disclosure" will further the quality of the ratings is not self-evident. Accountability and transparency are important in their own right, but there are other issues to consider. A major reason not to disclose the ratings is the possible effect on market perceptions and credit ratings and the associated financial consequences for the countries concerned. IEG notes (chapter 4) that the report recommendations are aimed at enhancing the CPIA beyond its use for IDA allocations. It suggests that if the CPIA is viewed only in an IDA-allocation context, the need to rate IBRD countries can be questioned. Management disagrees with the recommendation to disclose the IBRD scores and prefers to limit the coverage of the CPIA to the IDA-eligible countries only. Given that the IBRD scores are used internally by the Bank, the forthcoming CPIA review will include a more in-depth analysis of the value added and the costs of preparing for internal uses CPIA scores for IBRD countries. The conclusions of this work will inform management's decision on next steps. In the meantime the CPIA exercise will continue to cover the IBRD countries. Remove accounting for the Partially agreed. As the report notes (for example, chapter 2), there is relative consensus stage of development from in the literature that there is no single recipe for growth and that country specificities, the CPIA rating exercise. including the stage of development, need to be taken into account. Some of the policy objectives may be invariant to income (for example, desirability of well-managed budgets), but others are not (for example, expectations regarding regulatory capacity in low- income countries versus middle-income countries; social protection in a largely urban formal economy versus a poor rural subsistence economy). The IEG report suggests (the recommendations in the executive summary and chapter 4) that if accounting for the stage of development stage cannot be removed, then it is important to clarify in the guidelines which criteria should take into account the stage of development, what the rationale is for doing so, and how the adjustments should be made. Management agrees with this suggestion. Therefore, as part of the broad review of the CPIA (see below), the guidelines will be revised to clarify which criteria should be adjusted to account for stage of development and how the adjustment should be made. Undertake a thorough Agreed. Periodic reviews of the content and methodology have been a fixture of the review of the CPIA and evolution of the CPIA, and going forward they should continue to be. As the IEG evaluation revise the criteria as recognizes, these reviews create discontinuities, as some criteria are added, dropped, or necessary. This needs to revised. The last major revision took place in 2004, informed by the recommendations of an entail a detailed review of the external panel that undertook an in-depth review of the CPIA. Consensus on development literature for each criterion thinking moves slowly, and revisions should be undertaken with sufficient intervals so and needs to reflect the latest that the CPIA scores have some consistency over time. From the standpoint of country thinking on development relations and aid volatility, it is also important to avoid situations where changes in scores and lessons learned. It also result from modifications in the criteria rather than from a deterioration or improvement needs to take into account the in country performance. The CPIA is used by other multilateral development banks and an recommendations of IEG on extensive consultation process would be necessary. The IEG evaluation found that "perhaps specific changes to the criteria the time has come... for a thorough review of the CPIA" (chapter 2). Management broadly that were derived from the agrees but underscores that such a review needs to be carefully planned and done in the evaluation. context of IDA 16. The specific suggestions provided in the IEG evaluation will inform this review, to be completed by the time IDA 16 is launched. (continued on next page) xxii MANAGEMENT RESPONSE Management Action Record Independent Evaluation Group (IEG) Recommendations Requiring a Response Management Response Consider not producing an Agreed. Management will take this IEG recommendation into consideration in the context overall CPIA index although of the review of the CPIA mentioned above. IEG's rationale for this recommendation is that continue to produce and producing the different components of the CPIA without assigning weights to them in order publish the separate CPIA to arrive at an aggregate index would allow different weights to be applied according to components. country context and uses. In management's view, in the absence of robust evidence as to what these weights should be, there is value in applying a uniform weighting scheme across all countries and producing an overall index that summarizes the information contained in the different criteria and provides a clear reference point. Moreover, because the scores for all the criteria are disclosed, nothing prevents the users from creating an alternative index based on their preferred set of weights. As part of the review of the CPIA, management will consider whether or not to produce an overall index. xxiii Young boy sitting by centuries-old cistern, Hababa, Republic of Yemen. Photo by Bill Lyons/World Bank xxiv Chairman's Summary: Committee on Development Effectiveness (CODE) T he Committee on Development Effectiveness (CODE) considered the report The World Bank's Country Policy and Institutional Assessment (CPIA) ­ An Evaluation, prepared by the Independent Evaluation Group (IEG), and the draft Management Response. A statement by the external advisory panel on the IEG report was distributed as background document for the meeting. Summary before further considering the matter. Others The Committee welcomed the timely discussion endorsed the recommendation and the benefits of the IEG report, which confirms the useful- of disclosure for accountability and transparency, ness of CPIA as a broad indicator of develop- although they recognized the complexity of this ment effectiveness. The Committee noted that issue. There was general consensus that further the CPIA is not only being used for allocation of review and consultations would be needed International Development Association (IDA) with a view to consider improving transparency resources, but also for other purposes such as over time. Regarding "accounting for the stage the debt sustainability framework, for which an of development" in the CPIA exercise, some assessment on the impact of the CPIA review was members believed that this dimension should requested. In this vein, there was an agreement still be incorporated in CPIA and supported that the purpose of the CODE discussion was not management's proposal to clarify the relevant to address the use of CPIA in the performance- staff guidelines. based allocation formula for IDA. Recommendations and Next Steps The discussion focused on the four recommen- The Committee recommended to management dations in the evaluation. Members and manage- the following: ment broadly agreed with IEG's findings on the content of the CPIA and the recommendation The review of the CPIA should take into to review the individual CPIA criteria. There was account the comments and suggestions raised extensive discussion about IEG recommendation at the meeting to enhance its quality. This to disclose International Bank for Reconstruc- would include reviewing the CPIA criteria tion and Development (IBRD) ratings. Some as called for in the evaluation--for example, members questioned the value added of disclos- with respect to trade and finance, social and ing CPIA for IBRD and stressed the importance environmental components, and incorpo- of consultations with the countries being rated ration of criteria on disadvantaged groups xxv THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT in addition to gender, and engaging client development" on IDA allocations before endors- countries. The next steps are: ing the recommendation. One speaker stressed the need for CPIA to guide allocations in a fair, · Management will undertake a thorough review transparent, and effective manner. In this regard, of the CPIA in the context of IDA 16. members raised questions on how to synthesize · IEG will disclose its report together with the effectively or prioritize specific issues such as Management Response and the summary of governance, and how to strike a balance on "soft" CODE discussion. versus "hard" macro issues. In particular, there was support for strengthening the "soft" indica- Main Issues Discussed tors in the CPIA. Disclosure of IBRD ratings. Differing views were expressed on this recommendation. Some Review of CPIA. Members broadly encouraged speakers disagreed and recommended a more management to undertake a thorough review of prudent and cautious approach to consider the CPIA and revise the content and criteria as the value added of CPIA for IBRD countries. It recommended by IEG. In this regard, there were was noted that the disclosure of CPIA for IDA comments on the lack of agreement in the litera- countries was related to its use for the allocation ture on the impact of aid assistance on growth of IDA resources, that the CPIA did not play a and on the evidence to justify the large emphasis role in determining IBRD lending envelopes, and on governance; the need to avoid overlaps and that IBRD countries were not consulted on their further enhance the reliability of CPIA ratings; CPIA. Others supported the IEG recommenda- the linkage with Country Assistance Strategies tion to extend disclosure to IBRD countries in and single country exposure framework; and the spirit of transparency and accountability, disclosure of the CPIA methodology. Manage- suggesting that this may be done on a voluntary ment indicated that the review of the CPIA will basis or for selected clusters of indicators, and also analyze the issues of the value added and always consulting the concerned countries cost of preparing a CPIA for IBRD countries. before moving to disclosure. There was also a The conclusion of this work will inform manage- proposal to extend the indicator to industrial- ment's decision on how to go forward. ized countries. One speaker underscored that the CPIA is an indicator that tries to measure very Overall CPIA Index. There were different views different countries against a single benchmark. expressed on the need to produce an overall CPIA index although continue producing and Stage of development. Some members agreed publishing the separate CPIA component. Some with management on the importance of clarify- speakers noted that it was inevitable to have one ing the staff guidelines rather than removing overall index. the "accounting for the stage of development" in the CPIA exercise as recommended by IEG. Others pointed out the need to know the effect of removing the "accounting for the stage of Giovanni Majnoni, Chairperson xxvi Advisory Panel Statement Comments by Jürgen Zattler on some of · Criterion 8 can be dropped or reformulated the recommendations possibly measuring policies aimed at poverty Deputy Director General, Federal Ministry reduction such as agriculture (as proposed in of Economic Cooperation and Development, the review) or even infrastructure. Germany · Assessment of other marginalized socio- economic groups besides gender should be The Independent Evaluation Group (IEG) definitely integrated. In general, participation suggests removing accounting for the stages and minority protection could be integrated of development from the Country Policy and (possibly in the governance cluster). Institutional Assessment (CPIA) exercise: · There should be a separation of social sectors and environment, possibly creating a separate · It is much more difficult for a small fragile state environmental cluster with more differenti- to account for all standards that the CPIA de- ated criteria, but with a reduced number of mands than for India. Hence, there would be questions for the reviewers. an unfair treatment for less-developed coun- tries to receive a fair allocation. IEG suggests that there is no proof that the high · Alternatively to accounting for stages of devel- weight of the governance cluster increases loan opment by regional and network reviewers, performance. there could be a more differentiated weighting of the various criteria. The most important · Establishing good governance is one of the criteria to fulfill for a least-developed country core and most difficult tasks for a fragile state in fragility should be weighed higher. Hence, or a least-developed country and managing to fragile states can achieve a higher rating do so could be especially rewarded by weigh- quickly if they concentrate on the most ur- ing the governance cluster higher. gent criteria first. This measure also provides an incentive system to sequence measures for IEG suggests the disclosure of CPIA ratings development. for International Bank for Reconstruction and Development (IBRD) countries to increase IEG recommends that it should be considered transparency. I support this. not to produce an overall CPIA index although continue producing and publishing the separate · However, there might be many further issues CPIA components. where transparency and accountability can be better addressed (such as publishing the · If the separate clusters should be weighed in- margin of error, and increased use of exter- dividually according to the individual country nal sources for double-checking). The review situation, then this would be in line with my could have touched upon more issues. proposal above to weigh criteria according to their importance for development. We would like to emphasize our support for the contents of the articles in chapter 2 regard- IEG recommends a thorough review of CPIA and ing revising the trade and financial sector revise criteria if necessary. This I can fully support. criteria. xxvii THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Comments by K.Y.Amoako particularly since governance challenges tend Executive Secretary, to be country specific. Furthermore, reliable Economic Commission for Africa information may not exist to make objective United Nations Under-Secretary General assessments, and staff of the World Bank may not possess the required skills/competencies to make The IEG report should provide a sound basis for the right calls on these issues that require deep streamlining the structure, criteria and indica- appreciation of the political economies. There is tors of the CPIA to enhance its alignment to the a need for more work on the governance criteria goals of economic growth, poverty reduction to strengthen the relevance of governance indica- and development effectiveness. The report also tors, identify gaps in information and take steps provides the basis to discuss where to position to close those gaps. Work in this area would gain the CPIA's process and results in the World from the use on national and regional governance Bank's toolkit for improving the effectiveness experts that are close to the scene. of its support for economic growth and poverty reduction. The recommendation that the Bank not produce an overall CPIA index, although continuing The recommendations for changes in the criteria to produce and publish the separate CPIA for trade to include exports and reduce the weight component, is a good one. An aggregate index is given to trade protection, and for the inclusion of not likely to be a basis for informed policy discus- agriculture as a criteria in the CPIA, are welcome. sions and probably takes away the focus on the These are particularly germane for growth and component ratings, where debate and analysis poverty in Sub-Saharan Africa. On trade, indica- would be most useful. tors of export diversification and compliance with regional integration obligations would be useful. The report notes that "the strength of the CPIA Indicators for the agriculture criterion should ratings is Bank staff professional judgment." Thus not only focus on public expenditures on agricul- the process through which the Bank harvests its ture, but should also seek to reflect progress in considerable expertise for the CPIA is important. research and extension services, adoption of The evaluation report assumes that the process new technologies, strengthening land tenure, is fine. Nonetheless one may question whether provision of credit to farmers, as well as market- the existing process, which could be viewed ing, distribution and pricing issues. as overly bureaucratic, is best for tapping the expertise in the World Bank. Other related issues Expanding microcredit and developing microcredit include the nature of consultations with govern- institutions can help to enhance financial interme- ments and other informed stakeholders, support diation and to develop financial services and for economic and sector work and the quality contribute to the deepening of the financial sector of the statistical information base. For low- in general. Thus, the inconclusive evidence on the income countries, the Bank is the main source growth impact of microfinance notwithstanding, of economic and sector analysis and support for its place in the CPIA should be retained. statistics development invariably depends on external assistance. Countries with a combina- The overarching nature of governance would tion of relevant World Bank staff with limited justify the large overweighting. Besides, for experience, limited recent economic and sector those countries with long periods of poor work and lack of good statistics, may end up with governance, the potential impact of improve- unreliable CPIA ratings. ments in governance may be large compared to other clusters. However, the indicators in the The CPIA is carried out every year. This could governance cluster, particularly in q15 and q16, be too frequent as the policies, institutions and may not be the most relevant indicators to assess performance do not change that rapidly. Further- progress in governance in low-income countries, more, the annual revisions of the International xxviii A D V I S O R Y PA N E L S TAT E M E N T Development Association (IDA) allocations CPIA components that this report calls for (see cannot be helpful to country programming by below). I think the report should make this an the World Bank and budget planning by the explicit recommendation (or subrecommenda- governments. Although the CPIA does stimulate tion) to get management's response to it. thinking about a range of development issues, it is not a substitute for detailed policy and institu- A second suggestion is that when the ratings are tional analysis that would help the countries disclosed each year, the Bank should engage in make policy and build institutions. Is the CPIA a debate and discussion with local scholars and crowding essential country work in the environ- analysts on a country's ratings. A group of us did ment of constrained administrative budgets? In this a couple of years ago in Ghana, with some particular, there is the question of value addition surprising results--some local scholars thought of the CPIA for non-IDA countries and thus the the Bank was being too soft on some scores. need for CPIA for non-IDA countries. A more radical option is to bring in local expertise Comments by Ravi Kanbur at the time of rating--perhaps in the form of a T.H. Lee Professor of World Affairs and standing panel of distinguished country experts Economics who can provide their inputs to the Bank country Cornell University team. I welcome this report on the CPIA. It is a Second, remove accounting for the stage of thorough assessment and it raises a number of development from the CPIA exercise. important issues that Bank management needs to address. Moreover, given the key role played The central issue here is country specificity (see by the CPIA in the IDA allocation process, and also my comments on the third recommenda- in many analytical contributions to the develop- tion below). The conceptual foundation of the ment literature, the report's assessments are of CPIA is a cross-country econometric regression interest to the broader development community of a development outcome (usually growth but as well. By and large, I support the analysis and it could be a social indicator as well) against a the recommendations of the report. However, in number of "right hand side" (RHS) variables. my comments I will highlight where I think the It is these RHS variables that the CPIA clusters conclusions could have been much sharper. and categories are meant to capture. But in any regression there are points above and below I will structure my comments around the four the line. The question is, do these deviations principal recommendations of the report. contain information, or are the deviations purely random, with no information content whatso- First, disclose the ratings for IBRD ever? The difficulty for a CPIA type exercise arises countries. because we think that there is indeed informa- tion content in the deviations--that the "Bangla- I agree. But the report could call for more desh paradox" (why does a country with such transparency all around. poor governance ratings does so well on social indicators?) is indeed a paradox. One suggestion is that all previous ratings, IBRD and IDA, in all previous years, should be made As noted in the report, the "stage of develop- public. There is no reason why this cannot be ment" accounting is a way of trying to put back done. This will allow analysts in general, and not country specificity. The intention is good but, as just Bank researchers, to analyze the relation- documented by the report, the way it is done is ships between the different components of not. I support the recommendation to remove the CPIA and development performance. The accounting for the stage of development as it debate will serve to strengthen the review of is currently done, but this still leaves open the xxix THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT question of how country specificity is to be My basic point is that the major review of the brought in to the assessment (see below). CPIA that is recommended in the report must explicitly address the question of systematic Third, undertake a thorough review of the inclusion of outcome variables in the assessment CPIA and revise the criteria as necessary. as part of an overall investigation of how country specificity is to be brought into the assessment, I support this recommendation strongly. Indeed, which itself is part of the fundamental question after this major review I would suggest something which the review must start with--"What observ- like a cycle of three-year reviews. An alternative is able variables are good predictors of develop- to have a standing committee of external experts ment performance along the dimensions we are keep a watch on the CPIA process, with a major interested in?" review every three-to-five years to incorporate new knowledge of the development process. Fourth, consider not producing an overall CPIA index although continuing to produce By and large I support the specific subrecom- and publish the separate CPIA components. mendations under this category. However, I would like to highlight a point which, although I support this recommendation. It will then it is present in the report, is not emphasized render transparent how different uses, for enough. This is the importance of bringing in example the IDA allocation process, weight the actual outcome variables in the CPIA. I have different components. It will allow researchers to argued elsewhere (Kanbur 2005) --that bringing try out different weights for different purposes in the evolution of outcome variables is one and advance the development debate in that way. way of factoring in country specificity that, for But (see my comments on the first recommenda- whatever reason, is not easily captured by the tion), in order for the research and the debate to CPIA variables (think again of the Bangladesh be comprehensive, the Bank should release all paradox). As noted in the report, some outcome previous ratings, component by component, for variables are already brought in to the CPIA all previous years. assessment. The report itself argues for some more outcome variables, for example when it To conclude, let me say again that I welcome this recommends "Revision of the trade criterion report and I trust Bank management will respond to include a subcomponent on exports that to it positively. evaluates performance as well as policies and institutions." xxx Chapter 1 Evaluation Highlights · This evaluation assesses the rel- evance of the CPIA criteria and the reliability of the ratings. · The CPIA has evolved since its in- ception to cover 16 criteria in four clusters. · Since IDA 12 the CPIA has been used to allocate IDA funds with a larger weight on the governance criteria--specifically, the gover- nance cluster has 8.5 times the weight of each of the other three clusters. Students in classroom, Turkey. Photo by Scott Wallace/World Bank Introduction and Evolution of the CPIA T he Country Policy and Institutional Assessment (CPIA) assesses the quality of a country's present policy and institutional framework, with "quality" referring to the conduciveness of the framework to fostering poverty reduction, sustainable growth, and the effective use of development assistance (World Bank 2008b, p.1). It plays an important role in the country performance ratings (CPRs) that have been established annually since 1980 as a basis for the allocation of resources from the International Development Association (IDA) to eligible countries. Although CPIA ratings were initiated and used for of the CPIA in the context of a "Review of the IDA allocation purposes, they can and are being Performance-Based Allocation System" for its used for wider purposes. For example, the Bank IDA 10­12 Review in 2001.1 uses CPIA ratings for other corporate activities including the Global Monitoring Report. This Since the 2001 IEG review, there have been evaluation takes the premise that beyond inform- several developments and changes pertaining ing IDA allocations, the CPIA is useful as a broad to the CPIA. These include two restructurings of indicator of development effectiveness. the CPIA: in 2001 following the IEG2 review, and in 2004 following an external panel review.3 The Currently the CPIA consists of 16 criteria grouped external panel review of CPIA ratings and method- into four clusters, with each cluster having equal ology was instituted by Bank management in the weight in the overall CPIA rating. The four clusters context of the discussions about broadening are: economic management (cluster A); structural the disclosure of CPIA ratings for IDA-eligible policies (cluster B); policies for social inclusion countries. Other developments pertaining to the and equity (cluster C); and public sector manage- CPIA include IDA negotiations and the result- ment and institutions (cluster D) (see appendix A ing changes in the country performance ratings for a summary of the contents of each criterion). used in the performance-based allocation (PBA) system for allocating IDA resources. This is the first self-standing evaluation of the CPIA by the Independent Evaluation Group This evaluation will address the following (IEG). Prior to this, IEG had undertaken a review questions: 3 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT 1. What is the relevance of the CPIA criteria The remainder of this chapter presents a brief with respect to the policies and institutions discussion on the following: (i) the evolution in that are important for sustained growth, the content of the CPIA, including its relationship poverty reduction, and the effective use of with the underlying development paradigm and development assistance? IDA negotiations outcomes; (ii) other changes in 2. How reliable are the CPIA the CPIA; and (iii) the role of the CPIA in the IDA ratings (focusing on the most allocation formula. The evaluation assesses recently available ratings at the time the relevance of CPIA of this writing, the 2007 ratings, in Evolution in the Content of the CPIA criteria and the reliability reflecting such policies and institu- The CPIA has been evolving since its introduc- of the CPIA ratings. tions in the countries concerned? tion due either to changes in the Bank's thinking with respect to the development paradigm or Chapter 2 of this report will evaluate the to IDA negotiations, or to both. As stated by the relevance of the CPIA criteria with respect to Bank, "The [CPIA] methodology has evolved growth, poverty reduction, and the effective use over time, reflecting lessons learned and of development assistance based on a review of mirroring the development paradigm"(World the economics literature. Bank 2004d). Changes to the CPIA in Chapter 3 will address the second Among the most prominent changes made to the 1998 increased emphasis question--the reliability of the CPIA CPIA were those introduced in 1998, the spirit on institutions. ratings in two ways. First, it will of which has remained in place to date. The compare the ratings of the various most important of the changes was the greater CPIA criteria with those of other indicators that emphasis placed on institutions. Criteria were measure similar criteria. Second, it will review the added to the CPIA on the capacity to manage CPIA ratings generation process within the Bank. and implement policies, and existing criteria were revised to include/emphasize institutional Chapter 4 will summarize the findings presented aspects (box 1.1). Greater weight was given to in the previous chapters. Recommendations the public sector management cluster, which was will be drawn aimed at strengthening the CPIA raised from 14 percent of the CPIA in 1997 to 20 as an indicator that represents the factors in the percent in 1998 (table 1.1).4 country important for sustaining growth, foster- ing poverty reduction, and the effectiveness of In addition, the Bank started emphasizing development assistance. that the CPIA assess countries' policies and Box 1.1: Changes to the CPIA Criteria in 1998 That Reflect the Emphasis on Institutions Two criteria were added to the macroeconomic cluster, "mac- decisions, time spent by businessmen negotiating with bureau- roeconomic management capacity" and "sustainability of crats, and theft and crime that raise the cost of doing business. structural reforms." The latter evaluates the commitment of the A specific reference to environmental regulations was added authorities to reforms and the support of such reforms from the to the environment criterion. society at large. The civil administration criterion was replaced by the criterion The criterion legal and regulatory framework was renamed on accountability of the public service, with specific references property rights and rule-based governance, and specific refer- added regarding accountability mechanisms, and the voice and ences were added on contract enforcement, impartial judicial participation of the general public in public activities. Source: IEG, based on World Bank documents. 4 Table 1.1: CPIA Criteria 1998­2008 1998 2000 2004­08 General Macroeconomic Management of Inflation and Macroeconomic Performance (0.05) Current Account (0.05) Management (0.08) (q1) Fiscal Policy (0.05) Fiscal Policy (0.05) Fiscal Policy (0.08) (q2) Macroeconomic Management of External Debt Management of External Debt Debt Policy (0.08) (q3) Management and Economic Management Economic Management (0.05) (0.05) Sustainability of Reforms (0.20) (0.25) (0.25) Macroeconomic Management Management and Capacity (0.05) Sustainability of the Development Program (0.05) Sustainability of Structural Reforms (0.05) Trade Policy (0.05) Trade Policy and Foreign Trade (0.08) (q4) Exchange Regime (0.05) Foreign Exchange Regime (0.05) Financial Stability and Depth Financial Stability and Depth Financial Sector (0.08) (q5) (0.05) (0.05) Banking Sector Efficiency and Banking Sector Efficiency and INTRODUCTION AND EVOLUTION OF THE CPIA Resource Mobilization (0.05) Resource Mobilization (0.05) Policies for Sustainable Competitive Environment for Structural Policies Competitive Environment for Structural Policies Business Regulatory and Equitable Growth the Private Sector (0.05) (0.30) the Private Sector (0.05) (0.25) Environment (0.40) (0.08) (q6) Property Rights and Rule- Based Governance (0.05) Factor and Product Markets Factor and Product Markets (0.05) (0.05) Environmental Policies and Policies and Institutions for Regulations (0.05) Environmental Sustainability (0.05) (continued on next page) 5 6 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table 1.1: CPIA Criteria 1998­2008 1998 2000 2004­08 Pro-Poor Targeting of Gender (0.05) Gender Equality (0.05) (q7) Programs (0.05) Safety Nets (0.05) Equity of Public Resource Use Equity of Public Resource (0.05) Use (0.05) (q8) Policies for Reducing Policies for Social Building Human Resources Policies for Social Building Human Resources Inequalities Inclusion/ Equity (0.05) Inclusion/ Equity (0.05) (q9) (0.15) (0.25) (0.25) Social Protection and Labor Social Protection and Labor (0.05) (0.05) (q10) Poverty Monitoring and Poverty Monitoring and Policies and Institutions Analysis (0.05) Analysis (0.05) for Environmental Sustainability (0.05) (q11) Quality of Budget and Public Property Rights and Rule- Property Rights and Rule- Investment Process Based Governance (0.05) Based Governance (0.05) (0.05) (q12) Quality of Budgetary and Quality of Budgetary and Financial Management (0.05) Financial Management (0.05) (q13) Public Sector Public Sector Public Sector Management Efficiency and Equity of Efficiency of Revenue Management and Efficiency of Revenue Management and Institutions Revenue Mobilization (0.05) Mobilization (0.05) Institutions Mobilization (0.05) (q14) (0.20) (0.25) (0.25) Efficiency and Equity of Public Efficiency of Public Quality of Public Expenditures (0.05) Expenditures (0.05) Administration (0.05) (q15) Accountability of the Public Transparency, Accountability, Transparency, Service (0.05) and Corruption in the Public Accountability, and Sector (0.05) Corruption in the Public Sector (0.05) (q16) Source: IEG, based on World Bank documents. Note: Weight of the criterion in parenthesis. INTRODUCTION AND EVOLUTION OF THE CPIA the institutions that implement these policies, and the weights for the four clusters remained rather than development outcomes. Finally, the unchanged. The few prominent changes during ratings were to be given based on the "level" this period included the replacement of the of the countries' policies and institutions at the criterion on equality of economic opportunity time, rather than the changes in these policies by a criterion on gender in 2000. This effectively and institutions compared to the previous year, excluded discriminatory effects by socioeco- as had been done in the past. This in turn was nomic group (for example, by race, caste, and predicated on the assumption that the levels ethnic group) from the assessment. Another of such policies and institutions were the main change was the replacement of the criterion on determinants of aid effectiveness. social safety net by a criterion on social protec- tion and labor. This broadened the The focus on the public sector continued in 1999. assessment of the protection of the In 1999, emphasis on This reflected the interest of IDA deputies during poor beyond safety nets, and reflected social policies was the IDA 12 replenishment exercise. They noted the new social protection strategy that increased in the CPIA. that "accountability, transparency, the rule of law was launched by the Bank at the time and participation represent four major pillars of (World Bank 2000b). governance that are critical to the development process and the effective use of IDA resources." The changes in 2001 included the addition of This was also indicative of the new thinking by the an explicit reference to economic growth in the Bank on public sector effectiveness. The weight assessment of fiscal policy. In 2002, domestic of the public sector cluster was raised by another debt was included in the public debt criterion 5 percent to 25 percent with the transference (formerly only external debt was covered), of the criterion property rights and rule-based and other communicable diseases (in addition governance from the policies for sustainable to human immunodeficiency virus/acquired and equitable growth cluster to the public sector immunodeficiency syndrome [HIV/AIDS], which cluster. was there already), was added to the building human resources criterion. There was also a large increase in emphasis on social policies in 1999, reflecting the interest of Finally, the most recent and major restructuring IDA deputies during the IDA 12 Replenishment of the CPIA occurred in 2004. It was based on the exercise. Two criteria were added to the social recommendations of an external panel review policy cluster: equality of economic opportunity of the CPIA noted above. Several changes were and building human resources. The addition of introduced. The number of criteria was reduced these two criteria increased the weight of the from 20 to 16. This entailed collapsing four social policy cluster from 15 percent in 1998 to criteria into two5 and dropping two that were 25 percent in 1999. covered by other CPIA criteria.6 In sum, the changes in 1998 and 1999 resulted The content of virtually all the criteria All the CPIA criteria in much greater emphasis on social policies and was revised. For example, more detailed were revised in 2004 on the public sector in the CPIA. The weights for specification was provided on what and equal weights were these two clusters rose from 15 and 14 percent, was being assessed under each of the given to each cluster. respectively, to 25 percent each, and the weight three criteria of the economic manage- for the economic management cluster fell from 25 ment cluster. Greater emphasis was also placed to 20 percent, and that for the structural policies on customs and trade facilitation for the trade cluster fell similarly, from 40 to 30 percent. criterion. Assessment of gender disparities in politi- cal participation was added to the gender criterion. Adjustments continued to be made to the CPIA Tuberculosis and malaria were added to HIV/AIDS after 1999 through 2003, although these were in the assessment of building human resources less extensive than those made in 1998 and 1999, criterion, among other changes. 7 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Equal weights for each criterion were replaced The Bank's six regions, the networks, and the by equal weights for each cluster. This raised the central departments are all involved in the weight of the economic management cluster from selection of a representative sample of countries 20 to 25 percent (bringing it back to the 1997­98 that cover all the regions and include IBRD- and weight). The additional 5 percent weight given to IDA-eligible borrowers. Both good and poor the economic management cluster was effected performers are included in the sample, and the by dropping the monitoring and analysis of ratings of these countries have a similar distribu- poverty outcomes and impacts criterion from tion as the overall distribution of CPIA ratings. the social polices cluster. Although, ostensibly, The set of benchmark countries is reviewed the weight of the social policy cluster every year, taking into account the need to both In 2004, the weight on the remained at 25 percent, it was only maintain some continuity in the sample and to economic management because the environment criterion refresh it. The number of benchmark countries cluster was raised was transferred to this cluster (from has increased from 11 in 1998 to 19 in 2008, with from 20 to 25 percent, the structural policy cluster). In effect, the inclusion of additional IDA countries compris- whereas the weight on therefore, the weight is lower for ing all of this increase. Over the same period, the social policy cluster the non-environment social criteria. the share of IDA countries in the benchmarking (excluding environment) Between 2004 and 2008, the CPIA group has risen from 45 to 68 percent. was reduced from criteria and their weights remained 25 to 20 percent. unchanged.7 Policies/institutions versus outcomes. As mentioned earlier regarding the evolution in the In sum, over the past decade adjustments content of the CPIA, as part of the 1998 restruc- have been made to the CPIA with respect to turing of the CPIA, the Bank emphasized that the content, the number of criteria, and the countries' policies and institutions are being weights of the criteria. Notwithstanding these assessed rather than outcomes. Nonetheless, adjustments, the coverage of the CPIA has in the instructions to staff (the CPIA question- remained largely unchanged since the changes naire) with respect to preparing the ratings, it in 1998­2000, which introduced was clearly stated that Bank staff need to take Although the CPIA is the emphasis on the public sector into account country outcomes when assigning intended to assess policies and social policies (with the latter ratings, a statement that has remained in each and institutions, outcomes including the gender criterion) of the CPIA questionnaires since then. Further, also affect ratings. (table 1.1). although most of the metrics and indicators specified for the assessment of various CPIA Other Changes in the CPIA criteria are policies and institutions, for a few Many changes have been made over time in the criteria outcome indicators were also included, preparation process of the CPIA ratings, some in particular for the financial sector and for resulting from the findings of the IEG review of gender. 2001 and the external panel review of 2004. The changes are as follows. Rating scale and definition. In 1998, the rating scale was changed from five points to six points. Benc hmarking . Beginning in 1998, a In 2001, explicit definitions were provided for benchmarking step has been introduced to the the rating levels of 2, 3, 4, and 5 for each of the ratings process to strengthen the comparability CPIA criteria (previously only the 2 and 5 rating of country scores. This entails the introduction of levels were defined). In 2004, the definition of an initial phase in the CPIA preparation process rating levels was extended to rating levels 1 and of selecting and rating benchmark countries 6. Prior to 2004, a "6" rating was given for criteria (IDA and IBRD), against which ratings for other that had received a "5" rating for three or more countries would be compared during the CPIA years, and a "1" rating was given for criteria that preparation process. had received a "2" rating for three or more years (World Bank 2003a). 8 INTRODUCTION AND EVOLUTION OF THE CPIA Guideposts. These were introduced in 2001 sector management and institutions (referred and by 2005 were provided for each criterion. to as the governance criteria)--have played a There have been both additions to and removal greater role in IDA allocation than others, and of guideposts since their introduction. their role in IDA allocation has received increased attention over the last decade. Country context. Since 2004, following the recommendations of the external panel review, Beginning in 1998 with IDA 12, and in response to a specific instruction has been added in the CPIA IDA Deputies' suggestions, IDA allocations have questionnaire that staff may need to take into been adjusted by the country's perfor- account "the size of the economy and its degree of mance in the governance-related CPIA Since 2004, the rating sophistication in implementing the guidelines." criteria and in procurement practices. exercise is meant to Specific references regarding this point have According to IDA, "The stress on take into account the been added to the criteria on the financial governance has evolved over the past size of the country sector and social protection and labor. decade and was put in place by donors and sophistication because of its importance for improv- of its economy. Written record . This requirement was ing the development performance of introduced in 2001 for staff to provide written partner countries and for mitigating fiduciary justification to accompany their rating proposals. risks to aid funds" (World Bank 2006b, p. i.). The practice has been maintained since then. The governance adjustment was first introduced Disclosure. At the start of IDA 12 in fiscal 2000, IDA in 1998 for IDA allocations in the form of a initiated the disclosure, in quintile format, of the governance discount. This was replaced by CPIA and IDA Country Performance (ICP) relative the governance factor in 2001 to address ratings for IDA eligible-countries. Management the problem of discontinuity in allocations instructed country teams of IDA-eligible countries under the governance discount. For both the to discuss with each country's authorities their governance discount and the governance factor, country's CPIA and ICP ratings and the resulting the adjustment took into account the ratings of country IDA allocation. The quintile-based rating the CPIA governance criteria and that results for the CPIA, its four clusters, the country of the procurement criterion of the Since IDA 12, greater portfolio, and the quintile-based ICP rating were Annual Review of Project Performance weight has been given to then posted on the external World Bank Web site. (ARPP). the governance criteria for allocating IDA funds. On September 7, 2005, following the recommen- In 2004, the number of CPIA dation of the external review panel, the Board governance criteria included in the governance approved the disclosure of CPIA ratings for IDA factor fell from six to five, as the 2004 restructur- eligible countries. For the first time in June 2006, ing of CPIA had removed one of the governance IDA disclosed the numerical scores for all CPIA criteria. This reduced the effective weight of the criteria and the overall score for all IDA-eligible governance criteria in the country performance countries as the "IDA Resource Allocation rating from 68 to 66 percent. (See box 1.2 for a Index." This index is a misnomer, though, as IDA more detailed discussion of the evolution of the resources are actually not allocated according to governance adjustment.) this index, as will be discussed in the next section. Adjusting the country performance rating by Role of the CPIA in IDA Allocation the governance factor rendered the allocation The role of the CPIA in country performance formula more complex. The CPIA governance assessments, and in turn in the allocation of IDA cluster and the procurement flag from the funds through the PBA formula, has evolved over ARPP were double counted (see table 1.2). The time (see table 1.2). In particular, certain CPIA exponential multiplier (of 1.5) on the governance criteria--specifically those in the cluster on public rating (to arrive at the governance factor) made 9 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table 1.2: Evolution of IDA's Performance-Based Allocation Formula and the Adjustment for Governance Country Performance Rating Elements in the performance-based Portfolio allocation formula performance Governance adjustments in IDA Period (per capita) CPIA rating allocation formula 1991­96 GNPpc­0.25, CPR1.8 100%a 0% n.a. 1997 GNPpc­0.125, CPR0.5 CPR < 2 100%b -- n.a. GNPpc­0.125 , CPR 1.6 2 < CPR < 2.9 GNPpc­0.125, CPR1.95 CPR >2.9 1998­2000 GNPpc­0.125, (CPR/3)1.75 CPR < 3 80% 20% Governance discount (introduced fiscal 2000): For countries with 3 or more ratings GNPpc­0.125, CPR2 CPR > 3 of 2 or below out of the 6 CPIA governance criteria, and over 30% of projects with deficient procurement practices (according to the ARPP rating), the CPR is cut by one-third. 2001­08 GNPpc­0.125, CPR2 CPR = (0.8*CPIA + 0.2*ARPP) * 2001­03: governance rating = average governance factor of 7 governance criteria (6 CPIA criteria plus procurement criterion of the ARPP where governance factor = (governance portfolio rating). rating/3.5)1.5 2004­07: governance rating = average of 6 governance criteria (5 CPIA criteria plus a 3-year moving average of the procurement flag of the ARPP portfolio rating). Fiscal 2008: governance rating = average of 5 governance criteria From fiscal GNPpc­0.125, CPR5 CPR = 0.24*CPIAA-C + 0.68*CPIAD Governance rating = average of 5 2009 + 0.08*Portfolio performance rating c governance criteria Source: IEG, based on various IDA reports. Note: ARPP = Annual Review of Project Performance; CPR = Country Performance Rating; GNPpc = Gross National Product Per Capita. a. From 1998 through fiscal 2007, this was represented by the projects at risk rating in the ARPP. Projects at risk consist of actual and potential problem projects. Ratings of actual problem projects are done by task managers and reported in the implementation supervision reports. Ratings of potential problem projects are done by the Quality Assurance Group which looks at a number of criteria including the country's history of failure rate, defined as over 50 percent unsatisfactory outcome ratings by IEG. Beginning in 2001, the measurement of procurement enhancement was improved to capture not only the timeliness of the procurement process but also its quality. Beginning in fiscal 2008, only actual problem projects were included in the portfolio performance rating for the CPR. Hence, from then, the procurement flag has been dropped from the governance factor. b. The CPIA included a portfolio performance element which made up 20 percent of the weight in 1994, 10 percent in 1995­96, and 7 percent in 1997. c. CPIAA-C refers to CPIA clusters A (economic management), B (structural policies), and C (policies for social inclusion/equity), and CPIAD refers to CPIA cluster D (public sector manage- ment and institutions). The formula used the calculation and interpretation of to be transparent about how its resources were was very complex. the performance rating more complex. allocated, which led to the decision to disclose CPIA and country performance ratings beginning The complexity of the allocation formula was in June 2006 (see chapter 1, section on disclo- especially problematic as IDA was taking steps sure). In this light, at the Mid-Term Review of 10 INTRODUCTION AND EVOLUTION OF THE CPIA Box 1.2: The Governance Adjustment in IDA's Country Performance Ratings, FY1998­2008 IDA introduced adjustments to the country performance rat- which is equivalent to (governance rating/3.5).1.5 The governance ings used in the performance-based allocation system in 1998, rating is derived from the country's average rating for the seven under IDA 12. These adjustments initially took the form of a governance criteria mentioned above, with 3.5 being the mid-point governance discount. Specifically, IDA reduced the country of the rating. This governance factor is applied to the overall performance rating in the allocation formula by one-third for country performance rating. Under this new design, governance those countries with three or more highly unsatisfactory rat- performance at all levels is taken into account in IDA allocations: ings out of seven governance factors. Effectively, this reduced countries that score above the mid-point on governance-related IDA allocations for those countries affected by the discount criteria receive a premium, and those that score below receive on average by half. a discount. The 7 governance factors were 6 CPIA criteria plus the coun- There was still a discontinuity in allocation despite the replace- try's performance on procurement practices, according to the ment of the governance discount with the governance factor. ARPP. The 6 CPIA criteria were: (i) management and sustainability Specifically, a one point drop in just one of the seven governance of structural reforms; (ii) property rights and rule-based gover- criteria results in a 7.5 percent drop in the overall IDA rating, and nance; (iii) quality of budget and public investment process; (iv) in turn a 15 percent drop in the country's allocation (World Bank efficiency and equity of revenue mobilization; (v) efficiency and 2003b, p. 2). equity of public expenditures; and (vi) accountability of the public The seven governance criteria together had an effective weight service. Ratings were considered to be highly unsatisfactory if they of 68 percent in the IDA CPR. The effective weight fell slightly to were "2" or below for the CPIA criteria and in the case of ARPP 66 percent (World Bank 2004a, p. 3) in 2004 with the restructuring procurement criterion, if over 30 percent of projects had deficient of the CPIA criteria that removed one of the governance criteria procurement practices. (the one on management and sustainability of the development The governance discount produced a discontinuity effect at the program). point where the discount was triggered, with allocations dropping Adjusting the country performance ratings by the governance by one-half when only one criterion dropped from 2.5 to 2.0 (World criteria raised the dispersion of these ratings, as was intended by Bank 2001b, p. 4). Perhaps because of this, there were upward IDA to better differentiate the allocated resources depending on pressures on the ratings at the cut-off point (World Bank 2002b, the country's quality of governance. The governance adjustment p. 4). IDA deputies were also concerned about the punitive bias also raised the volatility of the country performance ratings. The of the governance discount, and the fact that it was not affecting procurement ratings, in particular, were more volatile than ratings all countries with weak governance (World Bank, 2002b, p. 4). of the CPIA governance criteria. To address this issue, in 2004, IDA To address these drawbacks, the governance discount was 14 introduced a three-year moving average for the procurement replaced by a governance factor in 2001 (World Bank 2002b, p. 4), ratings (World Bank 2006a, p. 13). Source: IEG, based on IDA documents. IDA 14, IDA Deputies requested Bank manage- CPR = (0.24 * CPIAA­C + 0.68 * CPIAD + 0.08* ment to "...simplify the allocation formula and portfolio performance), reduce unwarranted ratings volatility" (World Bank 2006a). where CPIAA­C refers to the average of the ratings of CPIA clusters A (economic management), B IDA deputies decided that, beginning with IDA 15 (structural policies) and C (policies for social (fiscal 2009), the country performance rating will be inclusion/equity), and CPIAD refers to ratings of simplified to make the weights of the components CPIA cluster D (public sector management and more explicit. Specifically, the country perfor- policies). Correspondingly, the PBA formula was mance rating (CPR) will be changed to: changed from: 11 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT The formula has been PBA = f(CPR2, population, These ch anges in the CPR (and associated simplified for IDA 15, GNIpc­0.125) change in the PBA) have made the IDA allocation making transparent to PBA = f(CPR5, population, formula more transparent--specifically that CPIA the relative weights GNIpc­0.125), cluster D has 8.5 times the weight of each of the of the different CPIA clusters A­C in the CPR. Yet, at the same time, by clusters in the allocation where GNIpc is gross national income breaking up the CPIA into the different constitu- formula--that is, that the per capita. The exponent on the CPR ent parts that are used in the CPR, the changes governance cluster has was changed from 2 to 5 to maintain have also made transparent the weakness of the 8.5 times the weight of the same dispersion of ratings and link between the overall CPIA index and IDA each of the other clusters. therefore of allocations as before.8 allocations. 12 Chapter 2 Evaluation Highlights · The CPIA covers the main deter- minants of sustained growth and poverty reduction, although some criteria can be usefully revised and streamlined and one added. · The evidence is less clear regarding the relevance of the content of the CPIA for aid effectiveness broadly, that is, that it represents the policies and institutions important for aid to lead to growth. · CPIA ratings are associated with a narrow definition of aid effective- ness, specifically the better perfor- mance of Bank loans. · There is insufficient evidence to conclude that the governance clus- ter associates better with loan per- formance than the other clusters. · The effects of a larger weight on governance in the IDA allocation formula (compared to equal weights on each cluster) are not due just to the governance rating but to how different that rating is compared to ratings on the other clusters. Hong Kong, China Relevance of the CPIA for Growth, Poverty Reduction, and Effective Use of Development Assistance A ccording to the Bank, the CPIA assesses the quality of a country's pres- ent policy and institutional framework, where quality refers to how conducive that framework is to fostering poverty reduction, sustainable growth, and the effective use of development assistance (World Bank 2008b). The review of economic literature (theoretical/ limited impact in the absence of more fundamen- conceptual as well as empirical) indicates that, tal institutional reforms (World Bank 1998). by and large, the CPIA criteria pertain to policies and institutions that are found to be important Today, there is relative consensus in the literature for sustained growth and poverty reduction (and around the idea that there is no single recipe for welfare more generally). The evidence is more growth and that country specificities--includ- mixed as to the criteria's importance for aid ing the country's stage of development--need effectiveness. to be taken into account.2 Of course, countries can learn from each other, but no simple recipe The CPIA and Determinants of can be pulled off the shelf to stimulate growth. Sustained Growth Each country needs to learn through trial and The literature on the determinants of sustained error what works for it (World Bank 2004e). growth has undergone a significant evolution This does not mean, however, that there are no during the last 50 years. In the 1950s and 1960s, growth determinants. What it does point to is it was widely argued that long-run economic the need "...to identify the exact set of policies performance depends on capital investment and institutional changes needed to address and that raising savings through a "big push" binding constraints on growth, based on first (Rosenstein-Rodan 1943) would launch countries principles in each instance" (World Bank 2005). into self-sustaining growth or "take-off " (Rostow 1960). In the 1980s, the literature begins to By and large, this evaluation finds that the CPIA emphasize the importance of a good economic covers those growth determinants over which there policy environment (Williamson 1990; World is relative consensus in the literature. Bank 1993) characterized by reduced tariffs, These are: institutions and governance; The CPIA covers appropriate foreign exchange rates, and low education; productivity and technologi- determinants of growth inflation.1 Then, in the 1990s, the literature cal innovation; and equity and equality for which there is emphasizes that these policies would have only of opportunity (table 2.1). relative consensus. 15 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table 2.1: Mapping of the "Consensus" Determinants for Sustained Growth and the CPIA Criteria Institutions and Governance Security of Property rights q12 Property Rights and Rule-Based Governance Rule of Law q12 Property Rights and Rule-Based Governance Government Credibility, q16 Transparency, Accountability, and Corruption in the Public Corruption Sector Quality of the Bureaucracy q15 Quality of Public Administration Human Capital Education q9 Building Human Resources Investment, Productivity and Private investment Technological Innovation · Stable fiscal policy q2 Fiscal policy · Stable monetary policy q3 Debt policy · Sound financial systems q13 Budgetary and financial management · Stable investment q14 Revenue mobilization regimes q1 Macroeconomic management · Clear and transparent q5 Financial sector business environment q6 Business regulatory environment · Labor mobility q6 Business regulatory environment · Rule of law q6 Business regulatory environment · Fighting corruption q12 Property rights and rule-based governance q16 Transparency, accountability, and corruption in the public sector Public investment and q2 Fiscal policy infrastructure Equity and Equality of · Property rights q12 Property rights and rule-based governance Opportunity · Access to credit q5 Financial sector · Access to education q9 Building human resources · Gender equality q7 Gender equality · Income transfers q10 Social protection and labor Sources: IEG, based on Cage (2009), background paper for this evaluation, and the CPIA 2008 questionnaire. Three CPIA criteria do not appear in the above determinants and how they are treated in the table: trade (q4), equity of public resource use CPIA will also be addressed. (q8), and environment (q11). This does not mean, however, that they are not important for growth, Institutions and governance only that there is less consensus in the literature Institutions and governance are among the on their impact on growth (specifically pertaining main growth determinants around which there to trade and environment), or what is important is relative consensus, and on which there is a for the criteria already covered by other criteria in sizeable literature (appendix B). In this literature, the CPIA (in the case of equity of public institutions refer to, variously, private property The trade and resource use). The evidence on the rights protection, contract enforceability, environment criteria impact of health (part of q9) on growth operation of the rule of law (including effective- have less consensus is also inconclusive, although health is ness and predictability of the judiciary, percep- in the literature. clearly important for welfare. tion of the incidence of crime), the quality of the bureaucracy, accountability of the government The rest of this section provides a brief summary (including independence of the media), and the of the literature on each of these determinants extent of corruption. of growth--both those for which there is more consensus and those for which there is continu- The existing evidence on the impact of virtually ing controversy. The relationship between these all of these indicators on growth is positive. The 16 R E L E VA N C E O F T H E C P I A one exception is corruption, where some earlier This is particularly pertinent for low The evidence on literature (from the mid-1960s to the mid-1990s) income countries which may perform the impact of most posits that corruption can have a positive well on the macro/fiscal stability front, institutional indicators impact on growth in instances where there are yet have weak fiscal management on growth is positive. pre-existing policy distortions such as pervasive capacity. In such cases, a good rating and cumbersome regulations, in which case for q2 needs to be tempered by an corruption can help efficiency and growth. But appropriate rating for q13 in order that the fiscal all of the literature from the mid-1990s onward aspect of the country in its entirety is realistically has found that corruption has a negative impact captured. on growth (appendix B). Similarly, finance depends on institutions, includ- The institution and governance indicators identi- ing informational and regulatory institutions, fied in this literature are covered in three of the institutions that strengthen creditor rights, five governance indicators under the public contract enforcement, and accounting practices sector management and institutions cluster of and the legal and judicial framework (Levine, the CPIA. These are the criteria on property Loayza, and Beck 2000; World Bank 2005). These rights and rule-based governance (q12), the institutions are covered in the CPIA criteria on quality of the bureaucracy (q15), and transpar- the financial sector (q5) and on property rights ency, accountability, and corruption in the and rule-based governance (q12). public sector (q16). Human capital The importance of institutions goes beyond Human capital--and in particular education-- these indicators. In particular, the institutional is one of the main determinants of sustained context within which policies are formulated growth around which there is consensus in is also important. For example, regarding the literature. In particular, the link between macroeconomic policies, it is not just low and primary enrollment and subsequent growth is stable inflation that is important, but the convic- well established in the literature (see appendix tion of the private sector that low and stable C). In addition, improvements in secondary and inflation is a permanent feature of the economic tertiary education systems are also important, environment. The latter requires an appropri- depending on the stage of development of the ate institutional underpinning for price stability country.3 Education is adequately covered in the (World Bank 2005; Montiel and Servén 2006). building human resources criterion (q9) of the CPIA, which includes assessment of In the fiscal arena, an appropriate institutional both basic and post-basic education. The institutional context setting needs to also ensure transparency, in which economic sustainable solvency, flexibility, and a pro-growth Although the importance of education policies are formulated structure of the budget. The institutional aspects on growth is clearly and strongly is important. of macroeconomic (and fiscal) policy are covered supported by evidence, the evidence in the macroeconomic management criterion on the impact of health on both the level of (q1), the fiscal policy criterion (q2), the quality economic development (per capita incomes) and of budgetary and financial management economic growth is less conclusive, criterion (q13), and the efficiency of revenue mainly because population increases Although education is mobilization criterion (q14) of the CPIA. that result from better health have a clearly important for negative effect on per capita income economic growth, the Although there is adequate coverage of the policy (see appendix C). Nonetheless, it evidence of health on (q2) and institutional (q13) aspects of fiscal is clear that health is important for economic growth is management, an issue arises over the coordina- welfare (the non-income dimension more mixed-- although tion of the assessment of these two criteria (in of poverty--see discussion later in this health is clearly the Bank they are assessed by different groups). chapter). important for welfare. 17 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Investment, productivity, and technological three macroeconomic criteria can be usefully innovation assessed as one. It is widely acknowledged among economists that strong, enduring growth requires high rates Public investment of investment (World Bank 2008a; Aghion and Private investment needs to be complemented by Howitt 2009). All of the different growth theories public investment to enhance competitiveness and have investment of one type or another driving create new market opportunities. Complementary growth.4 Both private and public investments are public investment in expanding infrastructure and important. Further, savings is equally important; communications and in upgrading the skills of the indeed, the evidence shows that there is no case labor force is particularly important (Easterly and of a sustained high investment (and high growth) Rebelo 1993; World Bank 1994; Sachs 2005, 2008; path that is not backed up by high savings (with Collier 2007; World Bank 2008a). the latter aided by fiscal prudence) (World Bank 2008a). In fast-growing Asia, for example, public invest- ment in infrastructure accounts for 5­7 percent The CPIA covers the Private investment of gross domestic product (GDP) or more. In multiple elements Fostering private investment requires China, Thailand and Vietnam, total infrastruc- necessary to foster reducing risks for private investors, ture investment exceeds 7 percent of GDP. private investment. through stable fiscal and monetary History suggests that this is the correct order policy, stable investment regimes, of magnitude for high and sustained growth, sound financial systems, and a clear and transpar- although it is difficult to be precise (World Bank ent business environment including flexibility of 2008a). Finally, public investment can be used as the labor market. It also requires ensuring the a tool to increase equality of opportunity, which rule of law, and measures to fight corruption is another determinant of sustained growth, (World Bank 2001a). discussed next. The CPIA covers public invest- ment in it criterion on fiscal policy (q2), where The CPIA covers all these elements important there is an explicit reference to "the provision for private investment in several of its criteria, of public goods, including infrastructure.... including those on macroeconomic manage- consistent with medium-term growth." ment (q1), fiscal (and debt) policies (q2 and q3), the financial sector (q5), business regula- Equity and equality of opportunity tory environment (q6), property rights and Development economics has seen a major shift in rule-based governance (q12), and transpar- view regarding the role of inequality on growth, ency, accountability, and corruption in the from one that initially saw increases in equality public sector (q16). as a natural accompaniment to development (Kuznets 1955) or actually facilitating develop- Public investment, All of the above criteria are concep- ment (through the incentives it provides) including for tually distinct except for macroeco- (Lewis 1954) to the current view that inequality infrastructure, is nomic management, fiscal, and debt is detrimental to growth (Todaro 1997; Aghion, covered in the criterion policies. Debt policies are clearly an Caroli, and García-Peńalosa 1999; Bardhan 2000; for fiscal policy. intrinsic part of fiscal policies, and Hoff and Stiglitz 2001). This view is supported fiscal policies in turn are clearly also by the results of several empirical (Alesina and an intrinsic part of macroeconomic manage- Rodrick 1994; Perotti 1992, 1993, 1996; Persson ment. Indeed, the macroeconomic manage- and Tabellini 1994) and other studies (box 2.1). ment criterion refers to public spending. It also refers to monetary/exchange rate policies aimed Equality of opportunity is an important element, at price stability, which cannot be achieved and indeed the starting point, of equity.5 System- without taking into account fiscal policies at the atic denial of opportunities to a group because same time. Thus, it appears that the existing of its ethnicity, religion, caste, or gender could 18 R E L E VA N C E O F T H E C P I A Box 2.1: Channels Through Which Inequality Affects Growth Inequality can have adverse consequences on efficiency, and Inequality often induces more political instability, as well as hence growth, through various channels. Inequality of wealth crime and insecurity of property rights, all of which depress invest- affects investment in physical and human capital. A better distri- ment and productivity growth (Alesina and Perotti 1996; Bardhan bution of wealth reduces credit constraints; broader availability 2000). Inequality (in the form of unequal access to investment op- of credit has a significant and positive effect on growth (Perotti portunities) can lead to macroeconomic volatility (Aghion, Caroli, 1992; Bardhan 2000; World Bank 2005). Micropanel studies show and Garcia-Penalosa 1999), which in turn has been found to reduce that households with few physical and human assets are often growth (Hausmann and Gavin 1996; Breen and Garcia-Penalosa caught in a poverty trap that sharply reduces their chance of 2005). Finally, too much inequality may also lead to social tension economic advancement and thus harms the overall economic expressed through violent redistribution, which has a negative performance of the economy (Christiaensen, Demery, and Pa- impact on growth (Bourguignon 2004). ternostro 2002; Woolard and Klasen 2005). Source: Cage (2009a) undermine social peace and spark political gnon 2004). There is a debate, however, as to how unrest (World Bank 2008a). There is evidence important microfinance is compared with overall that gender inequality--particularly in access to financial development (as measured by private education--reduces economic growth as it fails sector credit as a share of GDP interme- to make adequate use of female resources (World diated through the formal banking Equality of opportunity Bank 2001a; Klasen 2002; Knowles, Lorgelly, and sector). There are individual success is an important starting Owen 2002; Klasen and Lamanna 2003). stories of microfinance, including from point for equity. impact assessments that show microfi- The literature has provided some measures that nance in general helps the poor, although all partic- can improve equality and equality of opportu- ipants may not benefit equally.8 However, other nity. These include strengthening property rights studies find that overall financial development over land (Besley and Burgess 2003), expanding (measured by financial depth) has had a larger and access to education (Bardhan 2000; Dreze and more certain impact on growth and Although access to credit Sen 2002; Chhibber and Nayyar 2007), and means- poverty reduction than the expansion is clearly important, tested income transfers.6 Strengthening property of microfinance (Honohan 2004b; Beck, the importance of rights over rural land has resulted in higher Demirguc-Kunt, and Levine 2004). microfinance is less clear. agricultural productivity and output in China (Lin 1992) and India (Banerjee, Gertler, and Ghatak The CPIA covers the measures for equity (redistri- 2002); and strengthening land rights in urban bution) and for equality of opportunity identified areas can help poor households gain access to in the literature. These include property rights credit (De Soto 2000; Field 2002). Redistribution (q12); access to credit (q5); access to education of land has played an important role in fostering (q9); gender equality (q7); and income transfers economic growth;7 today, such reforms would (q10) (table 2.1). The financial sector criterion take the form of subsidized transactions in the covers both financial depth and microfinance. land market (Bardhan 2000; Bourguignon 2004). Thus, it covers all relevant ground irrespective of whether microfinance is important. Access to credit is also important for reducing inequality (Aghion, Caroli, and Garcia-Penalosa There are two issues related to the equity and 1999; Bardhan 2000; Besley and Burgess 2003; equality aspect of growth that are important for Beck, Demirguc-Kunt, and Levine 2004; Bourgui- the CPIA. The first is that only gender issues are 19 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT included in the assessment. Concerns related to by other CPIA criteria, although there is not other socioeconomic groups that are discrimi- enough information to meaningfully rate the nated against (due to race, caste, ethnic group) are public revenue subcomponent. At the same time, not included. Yet evidence indicates that poverty some socioeconomic groups that are discrimi- can have a strong ethnic dimension in some nated against are not included in the assess- countries (Bodewig and Sethi 2005). This implies ment on equity and equality of opportunity. that tackling social exclusion of such groups is The recommendation is to replace the criterion important not only for reducing poverty, but also on equity of resource use with a criterion on for raising growth for the country as a whole. equity and equality of opportunity for other socioeconomic groups. Alternatively, there can Although equity is The second issue is whether a be a reformulation of the criterion on equity of important, it is already criterion on equity of public resource resource use by, among other things, incorporat- covered in other CPIA use is needed. The criterion has two ing an assessment of other socioeconomic groups. criteria and in ways subcomponents: public expenditures more amenable to (66.6 percent weight) and revenue Integration into the global economy assessment than the collection (33.3 percent weight) that Integration into the global economy is a criterion on equity of affect the poor. According to the widely accepted determinant of growth public resource use. relevant network's review team, the although there is considerable debate in the assessment of public expenditures literature on how this can be achieved. The focuses on spending on education, experience of the 1990s demonstrates that health, rural infrastructure, and safety nets. Of there are many possible ways to integrate these, education and safety nets have been identi- globally (World Bank 2005). The challenge is fied in the literature as being important and, for policy makers to identify which best suits as mentioned in the previous paragraph, they their country's political economy, institutional are already covered by two other CPIA criteria, constraints, and initial conditions. Some analysts q9 (education) and q10 (safety nets). As for are in favor of granting temporary modest levels the other two items, spending on health is also of import protection to emerging industries covered in q9, and spending on rural infrastruc- where there is a demonstrated need (Williamson ture could be explicitly mentioned in the fiscal 2004). Others have focused on choosing the right policy criterion (which already mentions public form of protection, advocating subsidies to initial spending on infrastructure). entrants rather than the use of import duties. Indeed, when tariffs (the reduction of which is Integration into the global According also to the network review the most common policy prescription for trade economy is important team, the public revenue subcom- openness) are tried as an explanatory variable for growth, but the CPIA ponent, which focuses on whether for growth, they are not found to be statistically trade criterion does not taxes are progressive or regressive, is significant (Rodrik 2000). allow for flexibility in the very difficult to assess. Such an assess- approach or give adequate ment needs to be based on incidence The experience of the 1990s also indicates that attention to exports. analyses, which are typically not trade reforms need to be part of a comprehen- undertaken for many countries (or at sive growth strategy in order to be successful. best undertaken sporadically), so that results are Efforts to promote exports would need to be generally outdated even if existent. In other words, part of such a growth strategy, as the experience Bank staff does not have enough information to also shows that the successful liberalizers either meaningfully rate this subcomponent. The lack of explicitly or implicitly promote export growth information is even more acute for IBRD countries, (World Bank 2005). Many complementary as the issue is not as important for them. factors are needed for export growth, the most important of which are macroeconomic stability In sum, the public expenditure subcomponent of and the building of trade-related infrastructure the equity of resource use criterion is captured and institutions. 20 R E L E VA N C E O F T H E C P I A The CPIA criterion on trade (q4) covers trade According to that IEG evaluation, the costs associ- policy restrictions (tariffs and non-tariff barriers) ated with environmental degradation--such as and custom and trade facilitation, with 75 and public health costs of pollution or soil nutrient loss 25 percent weights, respectively. The CPIA from uncontrolled erosion--often reduce produc- guidelines provide instructions on the specific tivity, resulting in lower rates of economic growth tariff rates for each of the ratings. This is than would otherwise be the case. Beyond this, problematic on at least two fronts. It does not people are frequently impoverished by a declining allow for flexibility in trade reform approaches resource base and forced by their circumstances that have proven to work in different countries. to further degrade the environment. Also, the implicit assumption behind the relative weights--that tariff reduction is much more Hence, it seems reasonable that growth The CPIA takes account of important than complementary institutions for strategies in developing countries country specificity such successful liberalization--is not supported by need to take into account environ- as the size of the economy the evidence.9 In particular, country experience mental concerns from the outset, and stage of development. in the 1990s indicates that at moderate levels of even if they do not immediately adopt tariffs (which practically all countries currently the toughest environmental standards applied in have), further tariff reduction is not as important developed countries (World Bank 2008a). The as complementary factors for successful integra- CPIA has a criterion that assesses environmental tion into the global economy. policy and regulations on pollution and natural resources. Further, the trade criterion in the CPIA does not give adequate attention to exports. Country specificity Granted, a reduction in tariffs should promote The CPIA does take into account The application of equal exports,10 and there is evidence that this was country specificity, and in particular the weights to all four clusters indeed the case in the 1990s (World Bank stage of development. Specifically, the does not allow for country 2005). At the same time, however, tariff CPIA guidelines indicate that "Staff may specificity in the sense that reduction by itself is not enough, especially in need to take into account the size of the different countries may light of the possibility of different approaches economy and its degree of sophistica- face different policy and to trade liberalization. tion in implementing the guidelines." institutional priorities. Specific references are added on this It would be useful if the CPIA trade criterion for the financial sector and social protection could add a subcomponent on exports (with and labor criteria. Yet there are significant issues equal weights, as for trade restrictions and trade pertaining to the implementation of this in the facilitation) that assesses export performance, CPIA exercise, which are discussed in chapter 3. restrictions on exports (such as export taxes), and policies/institutions to reduce anti-export bias, In addition to the stage of develop- Links between growth, such as having a functional export rebate or duty ment, another important aspect of poverty, and the drawback system. The last is one of the indicators country specificity is the notion that environment are covered under the efficiency of revenue mobili- different policies and/or institutions complex and run in zation criterion, which could usefully be shifted can produce similar outcomes. The both directions. to the trade criterion. CPIA instruction to staff to take into account outcomes when assessing policies and Environmental sustainability institutions could help to address this aspect of A recent IEG evaluation (IEG 2008) on the country specificity. Some criteria already assess environment finds that links among growth, outcomes, although outcome variables could be poverty, and environment are complex and run added to other criteria, in particular trade. in both directions. Many, if not all, environmental problems improve as output levels rise, but they Yet another aspect of country specificity is the fact may get worse before they get better.11 that different countries may face different sets of 21 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT institutional or policy priorities. Taking this into Second, it follows from the relatively high account would require larger weights to be applied correlations that the weighting scheme used for to those criteria/clusters that are more important the CPIA does not matter very much in terms (or are the "binding constraints") to of representing the overall policies and institu- The four CPIA clusters growth. The Global Competitiveness tions of a country. This can be seen from the are highly correlated, Index has, as of 2009, taken steps in relatively high correlations between an unequally so the weighting scheme this direction by applying different weighted CPIA (no matter which cluster gets the does not matter much weights (derived based on econometric greater weight) and a CPIA with equal weights on for representing the analysis) to different components of the the four clusters (table 2.3). overall policies and index for countries at different stages of institutions of a country, development (World Economic Forum Third, different weights on the CPIA clusters do, but they do matter for the 2008). Currently, the application of however, and very importantly, matter for the allocation of IDA funds. equal weights to each of the four CPIA allocation of IDA funds, as discussed below. clusters does not allow for this aspect of country specificity. Chapter 3 will discuss the ways As an illustration, this evaluation undertook in which the different aspects of country specificity simulations to compare the PBA under a weight- are addressed in the CPIA rating exercise. ing scheme of equal weights for each cluster (as is done for the CPIA overall country score) Weighting scheme of the CPIA and a scheme of greater weight on governance Country specificity aside, the question of how to (as is done in the PBA formula). The simulation weight the various criteria in the CPIA has also replaced the country performance ratings (CPR) drawn a lot of attention because of the much in the PBA formula with equal weights on the greater weight given to cluster D (the governance four clusters, holding all the other factors that cluster) in the formula for IDA allocation (see affect the PBA constant.12 The simulation was chapter 1). There are three main observations performed on "core" IDA countries--that is, pertaining to weighting. those that are not subject to exceptions to the PBA due to post-conflict or re-engaging status or First, the CPIA ratings for the four clusters are to caps on allocations.13 relatively highly correlated (table 2.2), with the correlation coefficients ranging from 0.65 The simulation results indicate quite substan- between clusters A and B to 0.88 between clusters tial changes to the PBA of countries (table 2.4). C and D. This implies that countries that perform Although the simulations are based on data on well on one cluster generally perform well on the other clusters. Table 2.3: Correlations between CPIA with Different Cluster Weights and CPIA with Equal Cluster Weights, 2007 Table 2.2: Correlations between Ratings of CPIA Clusters, 2007 Cluster that has the Correlation with overall greater weight (as in CPIA (equal-weighted CPIAA CPIAB CPIAC CPIAoverall PBA formula) clusters) CPIAA 1.00 0.65 0.71 0.87 Cluster A 0.93 CPIAB 0.65 1.00 0.77 0.89 Cluster B 0.95 CPIAC 0.71 0.77 1.00 0.91 Cluster C 0.96 CPIAD 0.73 0.84 0.88 0.94 Cluster D 0.97 Source: IEG. Source: IEG. 22 R E L E VA N C E O F T H E C P I A Table 2.4: Simulation Results: Effects on Performance-Based Allocations for "Core IDA" Countries Arising from a Larger Weight on the "Governance" Cluster Compared to Equal Weights on All Clusters Rating of cluster D as a Change in performance- Rating of governance Average rating share of average rating of based allocation (%) cluster (cluster D) of clusters A to C clusters A to C (%) Country 1 31.4 3.4 3.4 99.7 Country 2 31.4 3.4 3.4 99.7 Country 3 27.4 3.9 4.0 98.3 Country 4 20.5 3.2 3.3 96.0 Country 5 19.5 3.3 3.5 95.5 Country 6 16.8 3.5 3.7 94.3 Country 7 15.3 3.7 3.9 94.1 Country 8 15.0 3.3 3.5 94.0 Country 9 14.3 3.5 3.7 93.7 Country 10 13.1 3.5 3.8 93.2 Country 11 11.4 3.5 3.8 92.6 Country 12 9.5 3.5 3.8 92.1 Country 13 6.2 3.2 3.5 90.9 Country 14 5.9 3.0 3.3 90.6 Country 15 4.6 3.3 3.7 90.3 Country 16 4.1 3.5 3.9 90.0 Country 17 2.8 3.2 3.6 89.4 Country 18 1.0 3.3 3.7 88.9 Country 19 0.3 3.3 3.7 88.4 Country 20 ­0.2 3.5 4.0 88.5 Country 21 ­1.5 2.8 3.2 87.8 Country 22 ­1.6 3.4 3.9 87.9 Country 23 ­1.9 3.4 3.9 86.9 Country 24 ­2.1 2.9 3.3 87.0 Country 25 ­4.1 2.7 3.1 86.8 Country 26 ­4.2 3.3 3.8 86.6 Country 27 ­6.2 3.0 3.5 85.7 Country 28 ­8.3 2.9 3.5 83.9 Country 29 ­8.7 3.3 3.9 85.1 Country 30 ­12.7 3.7 4.4 83.3 Country 31 ­13.0 2.7 3.3 82.1 Country 32 ­15.2 3.0 3.6 82.3 (continued on next page) 23 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table 2.4: Simulation Results: Effects on Performance-Based Allocations for "Core IDA" Countries Arising from a Larger Weight on the "Governance" Cluster Compared to Equal Weights on All Clusters (continued) Rating of cluster D as a Change in performance- Rating of governance Average rating share of average rating of based allocation (%) cluster (cluster D) of clusters A to C clusters A to C (%) Country 33 ­16.0 2.9 3.6 81.3 Country 34 ­16.3 2.2 2.7 81.5 Country 35 ­17.1 3.7 4.6 81.0 Country 36 ­18.2 3.3 4.1 80.9 Country 37 ­18.6 3.2 4.0 80.9 Country 38 ­20.4 2.7 3.4 79.9 Country 39 ­27.3 2.5 3.3 74.8 Country 40 ­29.4 2.6 3.5 75.2 Country 41 ­32.5 2.9 3.9 73.9 Source: IEG. Note: "Core IDA" countries refer to those IDA countries that are not subject to exceptions to the PBA due to post-conflict or re-engaging status or to caps on allocations. Small states are also excluded because their base allocations exceed the PBA. actual countries, the names of the countries At the same time, however, the much larger are not presented in the table. It should be weight on the governance cluster (compared noted that the PBA constitutes only part of the to equal weights on each cluster) has also led overall IDA allocation, which also includes a to perhaps unexpected results. Specifically, the base allocation of special drawing rights (SDR) simulation results indicate that the effects of the 1.5 million per country per year. The simulation much larger weight on governance on the PBA results demonstrate that, as intended by the PBA are not due just to the governance rating, but formula, a country that has a higher rating on the to how different the governance rating is from governance cluster but the same average ratings ratings on other clusters. on the other three clusters compared to another country would gain (from the larger weight on Two countries, country 4 and country 30, can be governance compared to equal weights for all used as examples to illustrate this point. Country clusters), whereas the other country would lose. 30 has a better governance rating (3.7) than This can be seen from the simulation results for country 4 (3.2). Country 30 also performs better country 5 and country 27, both of on all the other clusters compared to country 4, The effects of a larger which have the same average rating with the ratings for clusters A to C averaging 4.4 weight on governance of 3.5 for clusters A­C, but country 5 compared to country 4's average rating of 3.3. (compared to equal has a higher rating of 3.3 for cluster Yet, country 30 suffers a loss in PBA of 13 percent weights for each cluster) D compared to country 27's rating of under the current PBA formula (compared to a on the PBA are not due 3.0. Under the current PBA formula, formula with equal weights on all four clusters), just to the governance country 5 would have a PBA nearly whereas country 4 actually gains 20.5 percent. rating, but also to how 20 percent higher than a formula for These results are attributable to the fact that different that rating is which all clusters have equal weights, country 30's governance rating is much worse compared to ratings whereas country 27 would have a PBA than its ratings on other clusters, whereas country on other clusters. of 6 percent lower. 4's ratings on governance are only slightly worse 24 R E L E VA N C E O F T H E C P I A than its ratings on the other clusters. In other change with changes in per capita income, which words, country 30 suffers a loss under the current in turn means that, on average, growth leads PBA formula not because it has poor governance, to reduction in poverty. Indeed, it is but because its governance performance relative well-established in the literature that, For countries that have to performance on other fronts is worse than again on average, economic growth is worse governance ratings that of country 4, even though it performs better associated with a reduction in poverty than ratings on other than country 4 on all fronts. (Ames and others 2001; Besley clusters, some gain and and Burgess 2000; Ravallion 2001; others lose from the larger More generally, table 2.4 shows that all core IDA White and Anderson 2002; Christi- weight on governance countries (excluding small states) have worse aensen, Demergy, and Paternostro in the PBA formula governance ratings than ratings on other clusters, 2002; Dollar and Kraay 2002; Besley depending on how much yet some countries gain and other countries lose and Burgess 2003; Klasen 2002). worse the ratio of their from the larger weight on governance. Whether Hence, the determinants of growth governance ratings to they gain or lose depends on how much worse discussed in the preceding section ratings on the other the ratio of their governance ratings to ratings on are as important as the determinants clusters is compared other clusters is than other countries (figure 2.1). of poverty reduction. to other countries. The CPIA and Determinants of Although, on average, growth leads to poverty Poverty Reduction reduction, this is by no means the case for all It is a straightforward supposition that growth countries or for everyone in a country. Actual will lead to poverty reduction-- if it does not data show considerable variation--there are lead to greater inequality at the same time. A cases where inequality goes up with growth, and large empirical literature on the relationship cases where inequality goes down with growth between growth and changes in inequality finds (Kanbur 2004). One paper finds a huge range no statistical correlation between the two.14 This in the gains to the poor from a given rate of means that, on average, inequality does not growth.15 The reasons behind this wide range of Figure 2.1: Relationship between Changes in PBA and the Ratio of Cluster D Ratings to Ratings on Other Clusters for "Core IDA" Countries 100% Ratio of cluster D rating to rating on 95% 90% other clusters 85% 80% 75% 70% ­40% ­30% ­20% ­10% 0% 10% 20% 30% 40% Change in performance-based allocations Source: IEG. Note: "Core IDA" countries refer to those IDA countries that are not subject to exceptions to the PBA due to post-conflict or re-engaging status or to caps on allocations. Small states are also excluded because their base allocations exceed the PBA. 25 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT effects include differences in initial inequalities disproportionately boosting the incomes of the between countries and between regions within poor (Beck, Demirguc-Kunt, and Levine 2004). countries that create differences in how much Financial depth is addressed in the CPIA criterion the poor share in aggregate growth (or contrac- for the financial sector. tion). Another paper finds that the incomes of the poor do not grow one-for-one with increases As discussed earlier, education improves the in average income (Foster and Svékely 2008). equality of opportunity for the poor and other disadvantaged groups (including women). An Growth that reduces This evidence implies that growth that empirical study on Brazil finds that investments inequality will have a reduces inequality will have a larger in human capital are very important to make larger impact on poverty. impact on poverty. Therefore, policies growth more pro-poor (Menezes-Filho and need to take into account the distribu- Vasconcellos 2004). Education is covered in the tional impact of economic growth (the so-called CPIA criterion on building human resources. "pro-poor growth" policies) (Ghura, Leite, and Tsangarides 2002; Besley and Burgess 2003). In Not only does inequality have a negative impact addition, there are also policies that can have on growth, but higher initial levels of inequal- a direct impact on poverty independent of the ity also lower the poverty reduction impact of growth channel (the so-called "super pro-poor" growth (Ravallion 2001; Bourguignon 2004; policies).16 Chhibber and Nayyar 2007). Hence, changing the initial level of inequality enhances pro-poor The rest of this section will discuss the determi- growth. This requires measures to redistribute nants of pro-poor growth and super pro-poor wealth as well as to improve equality of opportu- policies. This will be followed by nity. (Such measures and how they are addressed Financial depth, human a discussion on two controversial in the CPIA are discussed earlier in this chapter.) capital, equality, determinants of poverty reduction-- and institutions and trade and the environment. Finally, Regarding institutions and governance , governance directly affect other non-income dimensions of establishing property rights can help the poor poverty reduction. poverty will be addressed. access credit (Fleisig 1995; de Soto 2000; Field 2002) and enhance their ability to utilize and Pro-poor growth and super pro-poor policies invest in land they cultivate (Deolalikar and There is a significant overlap between the determi- others 2002). It has been empirically found nants of pro-poor growth and of super pro-poor that increased protection of property rights has policies identified in the literature. Financial strong effects in reducing poverty (Acemoglu, depth, human capital, equality, and institutions Johnson, and Robinson 2001). One estimate and governance are found to be important for finds that increasing the protection of property poverty reduction directly as well as through rights across the globe by half of one standard the growth channel. They are also important for deviation would halve global poverty (Besley and reducing inequality and hence enhancing the Burgess 2003). poverty reduction effects of economic growth. Protection of property rights requires the Quite apart from reducing poverty through presence of rule of law, specifically controls over growth, financial sector development may crime and violence. Studies also find that the benefit the poor directly by facilitating access to victims of crimes are more likely to come from credit and improving risk sharing.17 Empirically, the poorer part of the population (Bourguignon there is evidence that financial (banking) depth 1999; Deolalikar and others 2002; Heinemann and is negatively associated with headcount poverty, Verner 2006). One study finds that police corrup- even after taking into account mean income and tion, especially in slum areas of poorer countries, inequality.18 Further, financial development has may increase the uncertainty of property rights also been found to reduce income inequality by of the very poor (Andvig and Fjeldstad 2008). 26 R E L E VA N C E O F T H E C P I A Corruption also directly affects poverty by increas- Indeed, a review of the East Asian Agriculture is important ing income inequality. One possible reason could experience indicates that "the for pro-poor growth; be that the benefits of corruption are likely to countries that have been most success- it would be useful to accrue to the better-connected individuals in ful in attacking poverty have achieved add a reference on the the society, who belong mostly to high-income rapid agricultural growth and broader provision of public groups (Gupta, Davoodi, and Terme 1998). It is economic growth that makes efficient goods in agriculture also possible that corruption distorts government use of labor and have invested in the to the fiscal policy allocations of goods and services (Tanzi 1998). human capital of the poor"(Rosegrant criterion of the CPIA. One study finds that an increase in the corrup- and Hazell 2000). Another paper finds tion index of a country by one standard deviation that rural growth reduced poverty in both rural (2.52 points on a scale of 1­10) increases the and urban areas, although urban growth had only Gini coefficient by 5.4 points (Gupta, Davoodi, some impact on urban poverty (Ravallion and and Terme 1998, 2002). Another study finds that Datt 2002). corruption decreases the share of government expenditures on health and education (Gupta, Improvements in labor productivity in agriculture Davoodi, and Terme 2002). are found (in both cross-country analyses and country case studies) to have been more pro-poor Finally, government accountability can have a than such improvements in non-agricultural direct impact on poverty. It has been argued that sectors (Eastwood and Lipton 2001). Increases no country with a free press has ever had a major in agricultural yields by 20 percent are found to famine (Dreze and Sen 1989), and that a free flow reduce the numbers of the poor by 18 percent of information pressures (even non-democratic) in a cross-country empirical investigation (Irz and governments into public action (Dreze and Sen others 2001). Agricultural research, in particular, is 1995). important in this respect, as it has led to crop yield gains in the past. Some researchers conclude from All of these different elements of institutions this that "it is unlikely that there are many other and governance that have been identified in the development interventions capable of reducing literature as being important determinants of the numbers in poverty so effectively" (Irz and pro-poor growth or super pro-poor policies are others 2001; Hazell and Haddad 2001). The CPIA covered in various criteria under the CPIA cluster does not explicitly cover agriculture--nor does on public sector management and institutions. it cover any other economic sectors. It does, however, allow for the provision of public goods Aside from these four determinants of pro-poor in the fiscal policy criterion. This is pertinent for growth, which are also determinants of growth, agriculture, in view of its importance identified in agriculture has been identified in the literature the literature. Hence, the CPIA instructions may as being important for pro-poor growth. Pro-poor usefully include a specific reference to public growth needs to take place in sectors where the goods in agriculture (in addition to the current poor are active and draw on the factors of produc- mention of infrastructure) for the fiscal policy tion that the poor possess. The vast majority of criterion. the poor live in rural areas, and a majority of them depend directly or indirectly on agricul- Trade and poverty ture for their livelihood. The factor of production From an analytical point of view, the relationship that the poor possess and use most is labor, and between trade and poverty is ambiguous--that sometimes land as well (World Bank 2000c; Ames is, trade can have a positive or a negative impact and others 2000; World Bank 2000a; Ravallion on poverty, as it does on growth (Agenor 2004; and Datt 2002; Eastwood and Lipton 2001). World Bank 2005). Indeed, there are several Therefore, pro-poor growth must focus on rural channels through which trade can affect poverty, areas, improve incomes in agriculture, and make and they can have opposite effects. These intensive use of labor (Klasen 2002). channels are household production, household 27 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Assessment of the trade consumption, participation in the (specifically flexibility in hiring and firing) in its criterion needs to labor markets, government revenues, criterion on business regulatory environment take into account the and social expenditures.19 Given that (q6). However, it is important that such mobility extent of intersectoral theory is ambiguous with respect to be ensured before trade liberalization proceeds, labor mobility. the impact of trade on poverty, the lest liberalization exacerbate poverty. Therefore, issue becomes an empirical one. Alas, assessment of the trade criterion needs to take the empirical literature--both cross-country and into account the extent of labor mobility. case studies--is equally inconclusive (Winters, McCulloch, and McKay 2004; Ravallion 2004; Environment and poverty Harrison 2006). As in the case of linkages between the environ- ment and growth, the linkages between the What seems to be clear, though, is that impedi- environment and poverty are similarly complex. ments to exports exacerbate poverty (although it On the one hand, there is evidence that environ- is less definitive as to what the effects of import mental regulations have a negative impact on liberalization are). One study finds that informal poverty. One paper finds that although, in general, export barriers to trade (such as transport costs, low-income households appear to bear a dispro- cumbersome customs practices, and costly portionate share of existing environmental risks, regulations and bribes) have significant adverse policies that reduce environmental risks are not effects on poverty in Moldova.20 Another study necessarily progressive (Parry and others 2005). (Balat, Brambilla, and Porto 2007) finds that Another paper studying the distributional effects lower export marketing costs encour- of environmental policy finds that many effects Linkages between age agricultural exports and lower the of such policies are likely regressive (Fullerton the environment and poverty levels of those engaged in 2008). On the other hand, there is also evidence poverty are complex. export cropping compared to others that resource degradation has a negative impact in the rural areas. Export market- on the poor (World Bank 2008f). ing costs could be reduced by investments in infrastructure such as roads, provision of Nonetheless, the impact of the environment marketing information, provision of credit and on welfare is clear--environmental pollution technical assistance to farmers, and promotion is clearly detrimental to health (World Bank of out-grower schemes among others (Balat, 2008d). Given that poverty needs to be viewed Brambilla, and Porto 2007; Otsuka 2002; as a multidimensional concept that includes Anderson 2003; Harrison 2006). welfare, environmental sustainability should clearly be taken into account as an important Labor mobility is another important complemen- factor in poverty reduction. tary factor. The negative impact of trade reforms on poverty in India is found to be related to the Multidimensional poverty extremely limited mobility of labor across regions Over the last decade or so, there has been and industries in the country (Topalova 2004, increasing recognition that the notion of 2005). Similarly, labor market reforms are found poverty encompasses more than just income to be important for minimizing the adverse poverty. Indeed, poverty includes a host of effects of trade reform on the poor in Colombia other dimensions that are central to the Millen- (Goldberg and Pavcnik 2005). nium Development Goals (MDGs). These other dimensions are education, health, gender The CPIA does not adequately take into account the equality, and environmental sustainability, all of importance of complementary factors in the trade which are covered by the CPIA. criterion to avert the potential negative impacts of trade liberalization on poverty. Specifically, The notion has also been advanced that poverty this relates to the complementary factor of labor goes beyond these income and non-income mobility. The CPIA does address labor mobility measures of physiological deprivation (inability to 28 R E L E VA N C E O F T H E C P I A meet basic material needs) to incorporate measures This theoretical foundation has been The notion of poverty of social deprivation (for example, access to the questioned by several researchers, encompasses both components of power such as decision making).21 dating back to the 1960s. Their main income and non-income The World Bank has indicated that "poverty is criticism is the key assumption of the dimensions as well as more than inadequate income or human develop- theory that foreign aid finances invest- social deprivation. ment--it is also vulnerability and a lack of voice, ment instead of financing consump- power, and representation" (World Bank 2000c). tion. Perhaps partly reflecting this less than robust theoretical foundation, no consensus in It is in this context that the concept of empower- the empirical literature can be found through the ment of the poor has emerged. Specifically, it mid-1990s of the impact of aid on growth. The is thought that because the poor are the main various reviewers of this literature come to differ- actors in the fight against poverty, they must be ing conclusions. One review (Hansen and Tarp brought to center stage in designing, implement- 2000) concludes that a majority of the literature ing, and monitoring anti-poverty strategies. up to the mid-1990s finds that aid has a positive This requires, among other things, empowering impact on growth. Two other reviews (Clemens, "pro-poor" coalitions, which can involve parts of Radelet, and Bhavnani 2004; McGillivray and governments, non-governmental organizations, others 2005) find no consensus. donors, and civil society (World Bank 2000a). Such coalitions can be helped by a free press, A watershed in the empirical literature was democratic institutions, and accountable govern- reached in the mid-1990s with the publication of ments-- particularly in countries where the poor a seminal paper in 1994 that empirically tested are the majority. The CPIA covers media freedom the assumption that aid financed investment and accountability in its criterion on transpar- (Boone 1994).24 The paper finds that aid did ency, accountability, and corruption in the not finance investment but financed public and public sector. private consumption instead. Furthermore, the higher consumption did not benefit the poor, as The CPIA and the Effective Use of reflected in the absence of a significant impact of Development Assistance aid on improvements in infant mortality, primary The notion of using the CPIA as an indica- schooling ratios, and life expectancy.25 This paper tor in the allocation of IDA resources is based demarcates the earlier generation of aid impact on two premises. The first premise is that IDA literature based on the financing gap models from resources are important for supporting "...the the latest generation, that is, underpinned by the world's poorest countries in their efforts to boost new growth theory. The latter generation of litera- economic growth, lower poverty and improve the ture specifically takes into account the effect of living conditions of people" (World Bank 2008e). economic policies and institutions on growth.26 The second premise is that such resources could only be used effectively in the presence of sound For many researchers, the Boone 1994 paper policies and institutions that are assessed under confirms the "macro-micro paradox": that many the CPIA. This section reviews the theoretical aid-funded projects report positive microlevel and empirical bases for these two notions. economic returns which are somehow undetect- able at the macrolevel. The literature that emerges The theoretical foundation of the effect of aid after this paper can be classified into three strands: on growth22 is the neoclassical growth model.23 those that deny the existence of the macro-micro Under this model, aid fills the gap in domestic paradox; those that try to explain it; and those savings or foreign savings to finance investment, that, like the 1994 paper, do not find any impact leading to growth. This theoretical foundation of aid at all (see appendix D for the list of papers). underpins a sizeable empirical literature (of well over 100 papers) on the impact of aid on growth Those that belong to the first strand find that from the 1960s through the mid-1990s. aid works on average, without conditions (the 29 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT "unconditional" strand). Then there are those ing the fragility of the empirical results of these that accept the contention that aid does not work papers. on average but seek to identify conditions under which it could be effective (the "conditional" Yet none of the researchers who find no impact strand). Foremost among the conditional strand of aid on growth, or who overturn the findings is the 1997 World Bank working paper (Burnside of aid having an impact on growth, conclude and Dollar 2000) that spearheads this strand of definitively that aid is not effective, or that literature with the finding that aid is effective (that policies, governance, or exogenous conditions is, has a positive impact on growth) only in the do not matter for aid to be effective. What they do Among those researchers presence of good policies (specifically 27 conclude is that cross-country empirics may not who find that aid has fiscal, monetary, and trade policies ), be very useful for analyzing whether and when aid an impact on growth, which are themselves important for works, fraught as they are with data problems.30 one group finds that it growth. Further, the paper finds that works unconditionally aid does not lead to good policies, In the midst of this ongoing controversy, some and the other group finds but that having the right policies in researchers have forged new ground by analyz- that it only works under place matters for aid to be effective in ing the impact of different types of aid on growth. specific conditions. terms of higher growth. One type of disaggregation is by donor objectives, based on the rationale that donors have strategic A number of other empirical papers that as well as developmental objectives for giving followed find other conditions to be important aid; hence, not all aid will lead to higher growth. for aid effectiveness. These conditions range In general, bilateral aid has strategic/geopoliti- from countries emerging from civil war and have cal objectives (although bilateral aid from some good policies, to countries prone countries has developmental goals), whereas Different analyses find to external shocks such as climatic multilateral aid has developmental objectives. By that different conditions and trade shocks (or terms of trade and large, the emerging literature on this finds matter for aid to have shocks), to countries outside of the that multilateral aid leads to higher growth, but an impact on growth. tropics. not bilateral aid.31 The third strand of literature finds that aid Another type of disaggregation focuses on aid has no impact on growth at all. In addition to that could have an impact in the short run. the seminal paper of 1994 mentioned above, One analysis finds that such "short-impact aid" two others find that by and large aid did not (that includes budget and balance of payments increase investment and that investment did support, investments in infrastructure, and aid not raise growth (Easterly 2001, 2003). A recent for productive sectors such as agriculture and paper that takes into account the motivations of industry) causes growth, on average, regard- donors in granting aid28 also finds it less of the recipient's quality of institutions and Some analyses find "... difficult to discern any systematic policies.32 that aid has no effect of aid on growth" (Rajan and impact on growth. Subramaniam 2008). Hence, a decade after the publication of the World Bank working paper that put the conditional aid Not only are the findings of this latest generation effectiveness literature on the map, there is no of literature diverse, they are also not robust. A consensus in the cross-country literature on the paper that tested many of these studies (belong- impact of aid on growth. ing to both the "conditional" and the "uncondi- tional" strands) for robustness finds that none of Findings of this literature range from (i) aid the results of these studies withstood the tests.29 having no impact on growth to (ii) aid having a Perhaps the most striking outcome of the tests is positive impact on growth conditional on policies that modification of the sample period affects the or exogenous factors (but with no consensus on regression results the most, thereby highlight- which policies and institutions matter) to (iii) aid 30 R E L E VA N C E O F T H E C P I A having a positive impact on growth regardless of Econometric analysis finds that the Empirical analysis policies and institutions. policies and institutions that are finds that policies and assessed by the CPIA matter for loan institutions assessed by Yet, despite the ambiguity of the impact of aid performance (see appendix G). Specif- the CPIA matter for loan from the cross-country empirical literature, there ically, overall CPIA ratings are found performance, although are many specific examples of aid being effective. to be negatively associated with the there is insufficient These examples range from the eradication share of problem projects as assessed evidence to conclude of certain diseases (for example, smallpox by Bank staff in loan implementation that the governance globally and polio in the western hemisphere) status reports. (The share of problem cluster matters more or the Green Revolution in India in the 1960s, projects is also found to be positively than the others. to improvements in school attendance and associated with loan outcomes with a health indicators resulting from conditional cash correlation coefficient of 0.63). transfer programs more recently. Further, ratings of each of the four CPIA clusters Yet another strand of aid effectiveness literature are also found to be negatively associated with loan has emerged, whereby efforts are focused on performance. It is not possible, however, to discern narrower evaluations of the impact of specific the relative importance of the four CPIA clusters aid project interventions. Such evaluations have on loan performance because the ratings of the been conducted in the context of so-called four clusters are highly correlated with each other. impact evaluations--or randomized evalua- Hence, there is not sufficient evidence to conclude tions--that evaluate the impact of specific that the governance cluster associates better with interventions by comparing the effects on those loan performance than the other clusters. who received the intervention with a compara- ble group who did not.33 Findings and Recommendations The CPIA covers the By and large, the CPIA criteria cover main determinants of Randomized evaluations over the last 10 years the main determinants of sustained sustained growth and or so have found positive benefits of aid projects growth and poverty reduction identi- poverty reduction. in education, health, physical infrastructure, fied in the literature. The CPIA covers and agriculture, among others (see appendix important determinants for both income and E). This has led some researchers to propose non-income poverty, with the latter includ- that development assistance should be mainly ing many of the MDGs (specifically, education, devoted to such project-specific efforts. health, gender equality, and environmental sustainability). It also covers some of the key Empirical analysis of CPIA and loan aspects of another important notion of poverty-- performance empowerment. The preceding discussion indicates that the evidence is mixed regarding the relevance It would have been useful to analyze empiri- of the content of the CPIA on aid effective- cally the impact of CPIA ratings including when ness in the broad sense--that is, whether the present together with IDA assistance, on the CPIA represents the policies and institutions actual growth performances of the countries important for aid to lead to growth. An empirical rated. However, this was not possible because analysis of the association between CPIA ratings of the major restructuring of the CPIA content and aid effectiveness is fraught with data difficul- as well as the 2004 rating scale. The discontinu- ties (see chapter 2, section on findings and ity in the CPIA ratings implied by the recommendations). Therefore, this evaluation restructuring would invalidate any Data constraints limit takes a different approach and examines the analysis using data that spans 2004. the use of econometric relevance of the CPIA in a narrower sense--that Using only data from 2004 onward analysis to establish a is, whether it is associated with the performance would not allow for a long enough link between CPIA ratings of Bank loans. time period for such analysis. and growth outcomes. 31 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Although data limitations make it difficult to largely covered by other criteria; consideration establish an empirical link between the CPIA and needs to be given to dropping or reformulat- growth outcomes and hence aid effectiveness ing it. This criterion has two subcomponents. broadly, the CPIA is found to be associated with The first subcomponent on public expenditures aid effectiveness in a narrower sense--specifi- is by and large covered by other CPIA criteria, cally in the performance of Bank loans. However, though Bank staff do not have enough informa- there is insufficient evidence to conclude that any tion to meaningfully rate the second one on tax one of the four CPIA clusters is more important revenues. Dropping q8 would lead to only minor for loan performance than the others. changes in the relative rankings of countries-- the rank correlation between the CPIA with q8 Based on the findings in this chapter, the evalua- and one without is 0.999--as well as only small tion derives recommendations regarding the changes to the PBA (table 2.5). A few more following broad issues. countries would gain than would lose. However, the changes on both the upside and downside Tax policy could Weighting of the CPIA would be rather small, with the largest loser reasonably be The findings indicate that when the experiencing a 1.7 percent drop in PBA and the combined with the CPIA is considered broadly as an index largest winner a 2.5 percent gain. fiscal policy criterion. of a country's policies and institu- tions, the weighting scheme is not so Second, currently tax policy is assessed in the important because the various CPIA clusters are criterion on efficiency of revenue mobiliza- highly correlated. However, the weighting of the tion (q14a). Yet tax policy is an intrinsic part different clusters in the PBA formula does matter of fiscal policy, and it would be reasonable to for a country's allocations of IDA funds. combine the assessment of the two in the fiscal policy (q2) criterion. Further, the part of the tax The utility of aggregating These findings raise the question policy subcriterion that deals with trade--specif- the CPIA clusters into of the usefulness of aggregating the ically import taxes and export rebate or duty an overall index is various CPIA clusters into an overall drawback--really belongs to the trade criterion questionable. index according to any predeter- (q4). In fact, q4 already deals with import taxes, mined weighting scheme. In the case so there is an overlap that needs to be removed. of the broad use of the CPIA, it does not allow Export rebate or duty drawback needs to be for country specificity which could imply differ- incorporated into the trade criterion, given the ent weights on the different clusters, depending importance of promoting exports for integration on the initial conditions and stage of develop- into the global economy. ment of the country. In the case of IDA alloca- tion, the overall index is already not used as such Third, there are some overlaps in content (see chapter 1). The recommendation, then, is between various criteria in the public sector for Bank management to consider not produc- management and institutions cluster that could ing an overall CPIA index, although continuing to be usefully streamlined. Judicial independence produce and publish the separate components is covered in both the criterion on property of the CPIA. rights and governance (q12) and the criterion on transparency, accountability, and corrup- Streamlining CPIA criteria tion in the public sector (q16). Administra- The CPIA is quite exhaustive in its coverage of tive corruption is assessed in the criteria on the main determinants of growth and poverty efficiency of revenue mobilization (q14) (in the reduction. Indeed, consideration subcomponent on tax administration), quality Equity of public resource needs to be given to streamlining it. of public administration (q15) (in the subcom- use is covered in other ponent on merit and ethics), and transparency, criteria and could be First, the criterion regarding the accountability, and corruption in the public dropped or reformulated. equity of public resource use (q8) is sector (q16). 32 R E L E VA N C E O F T H E C P I A Fourth, interviews with Bank staff conducted Table 2.5: Simulation Results: Changes in PBA from for this evaluation suggest that it is onerous for Dropping q8 country teams to have to answer 85 questions to arrive at one rating for the environment Countries that would lose PBA Countries that would gain PBA criterion.34 This is particularly the case in light of the mixed evidence of the environment on Percentage Percentage Country lost Country gained growth, as well as the mixed evidence of the environment on poverty. IEG recommends that Armenia ­0.04 Mozambique 0.12 Bank management drastically simplify the assess- Honduras ­0.13 Bolivia 0.13 ment of this criterion. Madagascar ­0.13 Tanzania 0.18 Azerbaijan ­0.18 Sierra Leone 0.37 Fifth, the three economic management criteria Nigeria ­0.23 Nepal 0.41 are not conceptually distinct from each other, unlike the rest of the CPIA criteria. Discussions Lao PDR ­0.27 Zambia 0.42 with the relevant network reviewer indicate Tajikistan ­0.32 Senegal 0.43 that the three criteria are indeed assessed as an Burkina Faso ­0.39 Malawi 0.44 integral whole. Yet separate scores are prepared Nicaragua ­0.44 Mongolia 0.44 and reported for each of the three criteria, Georgia ­0.57 Ghana 0.46 which could lead to double (or triple) counting or confusion. For example, a country that has Vietnam ­0.69 Mali 0.72 suffered deterioration in fiscal policy would Niger ­0.85 Mauritania 0.72 experience a reduction in both the fiscal policy Yemen, Rep. ­0.88 Guinea 0.72 and the macroeconomic management ratings, Chad ­0.98 Kyrgyz Republic 1.04 which means that such deterioration would be Uganda ­1.01 Bangladesh 1.05 double counted. Yet if Bank staff tries to avoid Rwanda ­1.14 Cameroon 1.05 double counting by downgrading only the fiscal policy but not the macroeconomic management Ethiopia ­1.63 Kenya 1.31 rating, the resulting ratings would appear contra- Papua New Guinea ­1.71 Sri Lanka 1.33 dictory. Although there may be merit in preparing Uzbekistan 1.40 three separate scores, Bank management needs Benin 1.64 to consider publishing only the consolidated Cambodia 1.79 economic management rating to avoid the Moldova 1.81 impression of contradictory scores when staff are avoiding double (or triple) counting. Bosnia and 2.49 Herzegovina Omissions to the CPIA criteria Source: IEG. Note: PBA=performance-based allocation. Notwithstanding the exhaustiveness of the CPIA coverage, there are a few omissions. First, there is the exclusion of socioeconomic groups other Coordinating assessment of CPIA Overlaps could be reduced than gender (such as by race, caste, and ethnic- criteria between the criteria in the ity). IEG recommends that assessment of the The reviews of a few criteria need to public sector management treatment of such socioeconomic groups be be coordinated. This has emerged and institutions cluster. included in the CPIA. in the context of the trade criterion, which needs to be evaluated in connection Second is the absence of any reference at all to with the assessment of the labor criterion. agriculture. As mentioned, this can be remedied Similarly, fiscal policy (q2) needs to by adding a reference to agriculture in the be evaluated in conjunction with the Assessment of the criterion on fiscal policy with reference to public quality of budgetary and financial environment criterion goods. management (q13). needs to be simplified. 33 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT The absence of agriculture Content of the CPIA criteria criterion derives findings and recommendations and socioeconomic The 2004 restructuring of the CPIA was for revising that criterion. groups other than gender the last time its criteria were reviewed needs to be addressed. and revised. Bank management Revise trade criterion also agreed with the external panel The importance of complementary institutions recommendation at the time that, for the sake for global integration is not adequately reflected of continuity and comparability, the CPIA criteria in the weighting scheme for the trade criterion, will not be revised too frequently (World Bank nor is adequate attention given to exports. IEG 2004d). Bank management further indicated that recommends that trade restrictiveness and trade periodic reviews (for example, every three years) facilitation be given equal weights (replacing the of the CPIA will be undertaken by an external current weighting scheme of 75 and 25 percent, technical advisory committee charged with respectively, on these two subcomponents). reviewing the CPIA methodology, procedure, and ratings quality. In addition, a subcomponent on exports needs to be added (with the same weight as the other This evaluation has found that the time has come two) to assess export performance and policies for Bank management to undertake a thorough and institutions to reduce anti-export bias. The review and revision of the CPIA. The subcomponent on exports needs to include the Add a subcomponent general growth literature review assessment of export rebate and duty drawback on exports to the trade conducted for this evaluation derives that should be transferred from the criterion criterion; assign equal findings and recommendations for on efficiency of revenue mobilization (q14) as weights to exports, restructuring the trade criterion; suggested in chapter 2 (in the section pertain- trade restrictiveness, likewise, an in-depth literature review ing to streamlining CPIA criteria). All three trade and trade facilitation. undertaken for the financial sector subcomponents--trade restrictiveness, trade facilitation, and exports--need to be given equal weights. Simulations of equal weights for the trade Table 2.6: Simulation Results: Changes in Trade subcomponents on restrictiveness and facilita- Ratings Arising from Changes in Weights of Trade tion indicate that there will be no change in Subcriteria by Numbers and Shares of IBRD and IDA ratings on trade for half of the countries (70 of Countries 140 countries), with slightly more than half of the IBRD countries (52 percent) and slightly less than No half of the IDA countries (48 percent) experienc- change in Rise in Fall in ing no rating changes (table 2.6). ratings on ratings on ratings on trade trade trade Total About equal numbers of IBRD countries will gain Number of countries and lose ratings on trade (16 and 15 countries, IBRD 34 16 15 65 respectively), although many more IDA countries IDA 36 12 27 75 would lose (27 countries) than would gain (12 countries). This implies that IDA countries have Total 70 28 42 140 worse ratings for trade facilitation than for trade Share of countries (%) restrictiveness compared with IBRD countries. IBRD 52 25 23 100 For all but one country, the change in ratings IDA 48 16 36 100 (whether up or down) is 0.5 points; the exception is Tunisia, which would gain 1 point (table 2.7). Total 50 20 30 100 Source: IEG. Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development The proposed change would increase the Association. comparability of the CPIA trade rating with that 34 R E L E VA N C E O F T H E C P I A Table 2.7: Simulation Results: IBRD and IDA Countries That Would Experience Changes in Trade Ratings Due to Changes in Weights for Trade Subcriteria Fall in ratings on trade Rise in ratings on trade IDA IBRD IDA IBRD Malawi Trinidad and Tobago Cape Verde Morocco Burkina Faso Colombia Uganda Iran, Islamic Rep. Săo Tomé and Principe Montenegro Senegal Malaysia Niger Micronesia, Fed. Sts. Nigeria Guatemala Tajikistan Thailand Ethiopia Uruguay Guyana Ukraine Rwanda Swaziland Azerbaijan Chile Bosnia and Herzegovina Algeria Lao PDR Albania Bangladesh Namibia Moldova Croatia Lesotho St. Kitts and Nevis Mauritania Bulgaria Sri Lanka Estonia Yemen, Rep. Lebanon Eritrea Botswana Mongolia Kazakhstan Pakistan South Africa Armenia Dominican Republic Korea, Rep. Papau New Guinea Paraguay Belize Chad Costa Rica Latvia Comoros Tunisia Bolivia Honduras Kyrgyz Republic Georgia Congo, Dem. Rep. Togo Timor-Leste Côte d'Ivoire Haiti Angola Congo, Rep. Source: IEG. Note: All changes are 0.5 points, with the exception of Tunisia which would experience a 1-point gain. IBRD= International Bank for Reconstruction and Development; IDA= International Development Association. of a comparator--the Enabling Trade Index 0.75.36 This in turn reflects the greater compara- (ETI). Specifically, the rank correlation coeffi- bility of the ratings on customs/border adminis- cient between the proposed CPIA trade rating tration than on trade restrictiveness (tariffs and and one based on the ETI35 will rise from 0.70 to nontariff barriers) between the CPIA and the ETI. 35 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Indeed, the rank correlation for customs/border Second, compared to almost every other CPIA administration between those two indexes is criteria, there is a greater focus in this criterion 0.77, compared with a rank correla- on assessing intermediate outcomes rather than The financial sector tion of 0.59 for trade restrictiveness. policies and institutions. This is particularly criterion focuses more on the case in the assessment of financial depth, intermediate outcomes. The proposed change would alter which focuses almost entirely on intermedi- It would be useful to the PBA, although not very signifi- ate outcomes, such as size of financial markets, include in the assessment cantly.37 Many more countries would interest rate spreads, and so on. It would be those policies and lose than would gain, reflecting the useful to include in the assessment policies and institutions that foster larger number of countries that have institutions that foster an enabling environment an enabling environment worse ratings on trade facilitation for the financial sector such as the legal, contrac- for the financial sector. than on trade restrictiveness. The tual, informational, and governance framework. magnitudes of the changes would be small, however. Bangladesh would gain the most Third, some of the indicators used in the assess- (by 1.6 percent), whereas Chad would lose the ment of the financial sector criterion can be most (3.4 percent) (table 2.8). strengthened. One example is the indicator on banking system soundness, which specifies two Financial sector criterion alternative intermediate outcome measures A review of the literature indicates that the related to non-performing loans (NPLs). 40 financial sector criterion does cover the Although NPLs may predict crises to some extent, dimensions along which finance is currently they are typically a lagging indicator, with high thought to be important: stability; depth and values suggestive of a problem that has already efficiency; and access. However, the way in which crystallized. Hence these NPL measures are crude some of the dimensions are currently being and inadequate even as indicators of current or assessed can be strengthened. imminent problems. This is clearly reflected in the fact that NPLs for residential mortgages did First, the application of equal weights to the three not provide a sensitive early warning system in financial sector dimensions can be revisited, the recent crisis in advanced economies. There particularly in light of the ongoing global financial are other good or even better flags for systemic crisis, as well as the considerable evidence of a risk, for example, indicators of foreign exchange large impact of banking crises on output losses risk in finance such as the dollarization of banking (Hoggarth, Reis, and Saporta 2002)38 and on the deposits or assets (De Nicoló, Honohan, and national budget (Laeven and Valenciana 2008; Ize 2005; Cashin and Duttagupta 2008). Rapid Honohan 2008b).39 Further, it is also growth of credit also needs to be monitored as Some of the indicators widely accepted that financial stabil- a possible warning sign. The assessment of this of the financial sector ity is a prerequisite for the effective dimension would be strengthened by taking into criterion need to be deployment of many types of develop- account such indicators. strengthened. ment assistance (although there is less systematic evidence on this front Appendix F presents a more detailed discussion (Honohan 2009). Hence, it would be useful for of the indicators in the CPIA financial sector Bank management to consider giving financial criterion based on a review of the literature. It stability a larger weight than the other two also offers recommendations for restructuring dimensions in the CPIA criterion. and strengthening the criterion. 36 R E L E VA N C E O F T H E C P I A Table 2.8: Simulation Results: Changes in PBA Arising from Changes in Weights for Trade Subcriteria Countries that will lose PBA (%) Countries that will gain PBA (%) Country Percentage lost Country Percentage gained Mali ­0.6 Rwanda 1.2 Zambia ­0.6 Senegal 1.3 Benin ­0.6 Uganda 1.4 Cambodia ­0.6 Bosnia and Herzegovina 1.4 Cameroon ­0.6 Ethiopia 1.4 Ghana ­0.6 Sri Lanka 1.4 Guinea ­0.6 Nigeria 1.5 Kenya ­0.6 Bangladesh 1.6 Madagascar ­0.6 Mozambique ­0.6 Nepal ­0.6 Nicaragua ­0.6 Sierra Leone ­0.6 Tanzania ­0.6 Uzbekistan ­0.6 Vietnam ­0.6 Armenia ­2.2 Georgia ­2.3 Moldova ­2.3 Burkina Faso ­2.4 Malawi ­2.4 Mongolia ­2.4 Honduras ­2.4 Bolivia ­2.5 Azerbaijan ­2.6 Papau New Guinea ­2.6 Kyrgyz Republic ­2.6 Niger ­2.6 Mauritania ­2.7 Yemen, Rep. ­2.7 Lao PDR ­2.8 Tajikistan ­2.9 Chad ­3.4 Source: IEG. Note: PBA= Performance-based allocation. Ranking changes based on equal weights for all CPIA criteria. 37 Women weaving on loom, China. Photo by Curt Carnemark/World Bank 38 Chapter 3 Evaluation Highlights · The CPIA correlates well with simi- lar indicators in terms of both the relative rankings of countries and the direction of change. · CPIA ratings correlate better for IBRD than for IDA countries with ratings of similar indicators--this could be because ratings for IDA countries take into account the stage of development. · Accounting for the stage of devel- opment is problematic because of the judgment involved and because of uneven practice across the Bank. · Having network reviewers validate ratings helps minimize potential bias in the ratings. Morocco. Photo by Curt Carnemark/World Bank. Reliability of the CPIA Ratings A ssessing the reliability of the CPIA ratings is an intrinsically difficult if not impossible task, given that a benchmark (that is, the "true" rat- ing) does not exist. Recognizing this major limitation, this evaluation assesses reliability in two ways: it compares the CPIA ratings with those of similar indicators, and it reviews the Bank's CPIA ratings generation process. With respect to the former, CPIA ratings are found development differently, which means that even to correlate well with similar indicators in terms of more unevenness is introduced into the CPIA the rankings of countries and in terms of direction ratings exercise. of change. This means that the CPIA ratings are not out of line with other indicators that measure The quality of CPIA ratings could be enhanced by similar policies and institutions. With respect to minimizing the amount of subjectivity involved the latter, the Bank's review processes are found in the rating exercise. This could be done by to guard against potential biases in ratings. excluding accounting for the stage of develop- ment from the ratings exercise. CPIA ratings are found to correlate better for IBRD than for IDA countries. One reason for this Comparability with Other Indicators could be that more information is available on Fourteen indicators are identified that could be IBRD than on IDA countries, which increases the compared with the CPIA (table 3.1). Two of the likelihood of different institutions having similar 14 are strictly comparable with the CPIA--these assessments on IBRD countries. Another reason are the CPIA ratings by the Asian Development for this could be the need to take into account Bank (ADB) and the African Development Bank the stage of development in the CPIA ratings. This (AfDB) (appendix H). The ADB uses exactly the means that more judgment is involved in rating same questionnaire as the Bank. For the AfDB, all IDA countries, because accounting for the stage the questions are either exactly the same or very of development is more important for IDA than close to those of the Bank with the exception of for IBRD countries. This introduces additional the question on trade (q4), for which the AfDB subjectivity into the rating exercise, which is includes an assessment on economic coopera- already centered on the expert judgment of staff. tion and regional integration.1 The rest of the The issue is further complicated by the fact that indicators are selected to match as closely as different networks treat the issue of the stage of possible with relevant CPIA criteria, some of 41 42 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table 3.1: Rank Correlation Coefficients between CPIA Ratings and Comparator Ratings for 2007 Bertelsmann Index of Corruption Ibrahim Index Gender Gender Transformation Economic Perception of African Gap Empowerment CPIA ADB AfDB ICRG GCI ETI DB Index Freedom WGI Index Governance ESI Index Measure Average q1 0.74 0.82 0.78 (26) (50) q2 0.61 0.82 0.72 (26) (50) q3 0.67 0.91 0.79 (26) (50) q4 0.67 0.72 0.71 0.70 (26) (91) (85) q5 0.62 0.75 0.71 0.67 0.69 (26) (50) (91) (139) q6 0.80 0.84 0.62 0.55 0.57 0.79 0.67 0.89 0.72 (26) (50) (90) (85) (139) (111) (114) (140) q7 0.78 0.66 0.67 0.61 0.68 (26) (50) (93) (60) q8 0.58 0.80 0.69 (26) (50) q9 0.80 0.68 0.69 0.72 (26) (50) (91) q10 0.72 0.77 0.72 0.74 (26) (50) (111) (continued on next page) Table 3.1: Rank Correlation Coefficients between CPIA Ratings and Comparator Ratings for 2007 (continued) Bertelsmann Index of Corruption Ibrahim Index Gender Gender Transformation Economic Perception of African Gap Empowerment CPIA ADB AfDB ICRG GCI ETI DB Index Freedom WGI Index Governance ESI Index Measure Average q11 0.71 0.77 0.59 0.69 (26) (50) (110) q12 0.75 0.82 0.73 0.69 0.74 0.69 0.88 0.76 (26) (50) (95) (92) (111) (114) (140) q13 0.54 0.89 0.72 (26) (50) q14 0.61 0.72 0.67 (26) (50) q15 0.68 0.80 0.66 0.83 0.74 (26) (50) (95) (140) q16 0.72 0.79 0.51 0.82 0.75 0.86 0.86 0.77 0.76 (26) (50) (95) (111) (114) (140) (135) (46) Average 0.69 0.79 0.63 0.69 0.63 0.62 0.77 0.70 0.87 0.86 0.77 0.59 0.67 0.61 0.72 Source: IEG. Note: The number of observations is presented in parenthesis below the correlation coefficients. All correlations are for 2007 ratings except for the Ibrahim Index of African Governance (2006 ratings) and Environmental Sustainability Index (ESI) (2005 ratings). ADB = Asian Development Bank; AfDB = African Development Bank; DB = Doing Business; ETI = Enabling Trade Index; GCI = Global Competitiveness Index; ICRG = International Country Risk Guide; WGI = Worldwide Governance R E L I A B I L I T Y O F T H E C P I A R AT I N G S Indicators. All coefficients are significant at the 1 percent level. 43 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Box 3.1: Comparator Indicators This evaluation selected 12 other indicators, in addition to the · The rest are produced by the Bertelsmann Foundation (Ber- CPIA ratings produced by the AfDB and the ADB, to compare telsmann Transformation Index), Heritage Foundation (Index with the Bank's CPIA: of Economic Freedom), the Political Risk Services Group (the International Country Risk Guide), Transparency International · Three are produced by the Bank--the Logistics Performance (Corruption Perception Index), the United Nations Devel- Index, Doing Business, and the Worldwide Governance In- opment Programme (the Gender Empowerment Measure), dicators. and the Mo Ibrahim Foundation (Ibrahim Index of African · Four are produced by the World Economic Forum--the En- Governance). abling Trade Index, the Environmental Sustainability Index, the Global Gender Gap Index, and the Global Competitive- Appendix I provides details on the specific subindicators used ness Index. for comparison. Source: IEG. which are guideposts for certain CPIA criteria the CPIA criteria varies. Some overlap much (box 3.1). better with the content of certain CPIA criteria than others. Nonetheless, for each criterion the For certain CPIA criteria, very few comparators correlation coefficients with both the develop- can be identified. In particular, it is difficult to ment and nondevelopment bank comparators identify comparators other than those by the are similar and relatively high, averaging between ADB and AfDB for the three economic manage- 0.7 and 0.8 (see last column of table 3.1). This ment criteria (q1, q2, and q3),2 and the criteria on provides some assurance that the CPIA ratings equity of public resource use (q8), the quality of are not out of line with those of other indicators budgetary and financial management (q13),3 that assess similar policies and institutions. The and efficiency of revenue mobilization (q14). correlation coefficients presented in table 3.1 Hence, for those criteria, only comparisons with also indicate that there is not much difference ratings by ADB and AfDB are made. between all 16 CPIA criteria in terms of their comparability with other indicators. In contrast, quite a few more comparators are found for the criteria on business regulatory Although on average the CPIA ratings correlate environment (q6) and transparency, account- well with other indicators, there is a dispersion to ability, and corruption in the public sector the correlation coefficients that range from a low (q16). For both these criteria, six other indicators of 0.54 (for q13 with ADB) to a high of 0.89 (for in addition to the ones from ADB and AfDB are q13 with AfDB and q6 with the World Governance found. Indicators). Several possible reasons may account for this dispersion. First, as mentioned, except Rank correlations for the AfDB and the ADB, the other indicators Interpretation of the correlation coefficients4 of are not assessing the exact same criteria as the the CPIA ratings with other indicators is compli- CPIA. Second, judgment is involved in the rating cated by the fact that the content of the indica- exercise (by the Bank and by virtually all the other tors, other than those of the other development institutions; see table 3.2), which is exacerbated banks, is not exactly the same as the CPIA. The when there is not enough information available overlap between these other indicators and on the criteria. 44 R E L I A B I L I T Y O F T H E C P I A R AT I N G S Table 3.2: Other Indicators--Expert Judgment or Hard Data? Expert Judgment/ Expert Judgment/Survey External data source CPIA Criterion Survey Hard data and Hard Data ADB q1 to q16 AfDB q1 to q16 Global Competitiveness Index q4 q5 q6 q9 q12 International Country Risk Guide q12 q15 q16 Enabling Trade Index q4 q6 Doing Business q5 q6 Bertelsmann Transformation Index q6 q10 q12 q16 Index of Economic Freedom q6 q12 q16 Worldwide Governance Indicators q6 q12 q15 q16 Corruption Perception Index q16 Ibrahim Index of African Governance q16 Environmental Sustainability Index q11 Gender Gap Index q7 Gender Empowerment Measure q7 Source: IEG. Note: ADB= Asian Development Bank; AfDB= African Development Bank. 45 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT For example, even though the ADB uses the same the points of reference here because they are the CPIA questionnaire as the Bank, the correlation most comparable with the Bank's ratings both coefficients for two criteria, q8 and q13, are because of their content, and because all three relatively low (the respective correlation coeffi- institutions take into account country context in cients are 0.58 and 0.54). Very different rankings the ratings whereas the other indicators do not.) for two to three countries (of 26 countries) are The comparisons indicate that, overwhelm- responsible for the relatively low correlations ingly, these other indicators correlate better overall. In the case of q8, these are Micronesia, with the Bank's ratings than with ratings by Cambodia, and Timor-Leste, and for q13, these AfDB and ADB (table 3.3). are Azerbaijan and Tonga. Two of these countries are Pacific Islands, for which the Bank has little Comparing changes in ratings up-to-date or firsthand information (accord- The two previous sections indicate that the ing to region and network reviewers who were level of the CPIA ratings (and implied rankings) interviewed for this evaluation). compare relatively well with other indicators. This section examines how well the changes in For all of the criteria for which there are compara- ratings compare with changes in the comparator ble indicators other than those from ADB and indicators between 2006 and 2007. AfDB (that is, all criteria except for q1, q2, q3, q8, q13, and q14 as discussed above), Strictly speaking, a similar assessment of a partic- Other comparator comparisons of rank correlations were ular criterion by different institutions would indicators correlate undertaken to assess whether these imply that a change or no change in ratings by better with the Bank's other indicators correlate better5 with one institution would be associated with similar CPIA ratings than the Bank's CPIA or with those of the movements in ratings by the other institu- with the ratings by the other two development banks. (The tions. Because the timing of the CPIA assess- AfDB and the ADB. ratings by the AfDB and the ADB are ment exercise varies across these institutions, a Table 3.3: Are Other Comparator Indicators Closer to the Bank or to AfDB and ADB? Majority of other indicators correlates better with Majority of other indicators correlates better with CPIA criterion Bank AfDB Bank ADB q4 n.a. n.a. q5 q6 No difference q7 q9 No difference No difference q10 No difference q11 q12 q15 q16 Source: IEG. Note: The comparison for q4 is not valid with AfDB because AfDB defines q4 differently from the Bank. ADB= Asian Development Bank; AfDB= African Development Bank; n.a.= not applicable. 46 R E L I A B I L I T Y O F T H E C P I A R AT I N G S change in the assessment by one institution may In the comparison with the AfDB, for The Bank's CPIA ratings not always be associated with a similar change in 10 of the 15 criteria (q4 is excluded and those of AfDB assessment by the other institution. However, from the analysis for reasons stated and ADB correlate at the very least, these assessments--if they are earlier), none of the countries has well in terms of similar--would not contradict each other or, ratings that move in the opposite direction of change. in other words, the ratings would not move in direction. For four criteria--q2 opposite directions. (fiscal policy), q7 (gender), q8 (equity of public resource use) and q13 (quality of budgetary The comparisons of changes in ratings indicate and financial management)--the ratings that the Bank's assessments are very similar to move in the opposite direction for only 2 percent those of the AfDB and the ADB. Only for a few of the countries (one out of 50 countries). For criteria and a very small share of countries do q1 (macroeconomic management), a slightly the ratings of the Bank and AfDB/ADB move in higher 4 percent of the countries (2 out of opposite directions (table 3.4). 50 countries) have ratings that move in the Table 3.4. Comparison of Changes in CPIA Ratings 2006­07 between the Bank, AfDB, and ADB Comparison with AfDB Comparison with ADB Change for one Change for one Change in Change in institution but Change in the Change in institution but the same the opposite no change in same direction the opposite no change in Criterion direction(%) direction (%) the other (%) (%) direction (%) the other (%) q1 60.0 4.0 36.0 50.0 4.2 45.8 q2 42.0 2.0 56.0 41.7 0.0 58.3 q3 64.0 0.0 36.0 33.3 0.0 66.7 q4 n.a. n.a. n.a. 58.3 0.0 41.7 q5 72.0 0.0 28.0 54.2 4.2 41.7 q6 66.0 0.0 34.0 75.0 0.0 25.0 q7 58.0 2.0 40.0 62.5 0.0 37.5 q8 52.0 2.0 46.0 54.2 0.0 45.8 q9 66.0 0.0 34.0 66.7 0.0 33.3 q10 74.0 0.0 26.0 50.0 0.0 50.0 q11 64.0 0.0 36.0 66.7 4.2 29.2 q12 80.0 0.0 20.0 50.0 0.0 50.0 q13 56.0 2.0 42.0 33.3 0.0 66.7 q14 72.0 0.0 28.0 54.2 0.0 45.8 q15 70.0 0.0 30.0 70.8 0.0 29.2 q16 56.0 0.0 44.0 54.2 0.0 45.8 Source: IEG. Note: q4 is excluded from this analysis because of the additional dimension of regional integration that is included in q4 by AfDB but not the Bank. ADB= Asian Development Bank; AfDB= African Development Bank. 47 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT opposite direction. In the comparison with Two possible reasons may account for these ADB, for 13 of 16 criteria, none of the countries findings. First, in general, more information have ratings move in the opposite direction. For is available on IBRD countries than on IDA the remaining 3 criteria--q1 (macroeconomic countries (see appendix J) which increases the management), q5 (financial sector), and q11 likelihood of different institutions (AfDB as well (environmental sustainability), 4 percent of as other institutions) having similar assessments the countries (1 of 26 countries) have ratings on IBRD countries as the Bank. Second, the stage move in the opposite direction. of development (taken into account by the Bank and the AfDB) is more pertinent for IDA than The same comparison with the other indicators for IBRD countries, and the additional judgment is not reported here because the CPIA ratings (by involved in accounting for the stage of develop- the Bank and the other two development banks) ment would likely make ratings for IDA countries are more discrete (with intervals of 0.5) than the less comparable. Regardless of whether one other indicators. This implies that changes in the or both reasons are valid here, CPIA ratings ratings of the other indicators may not correspond correlate better with those of other indicators to changes in CPIA ratings, even if the assessments for IBRD than for IDA countries. are similar to those of the Bank. In other words, small changes in policies and institutions may lead Conclusions on comparability of CPIA with to a change in a rating that is on a more continuous other indicators scale (such as the Global Competitiveness Index), The findings indicate that CPIA ratings for all whereas the same small change would not be 16 criteria correlate relatively well with those of reflected in changes in the CPIA rating because it similar indicators in terms of both the relative takes a relatively significant change in policies and rankings of countries as well as the direction of institutions for the CPIA rating to change by 0.5. change. The rank correlations of CPIA ratings with ratings of other indicators average between Therefore, comparisons of changes in CPIA ratings 0.7 and 0.8 for each of the 16 CPIA criteria. Ratings with changes in the ratings of other indicators could of other indicators correlate better with CPIA only be made for instances where the CPIA ratings ratings by the Bank than those by the AfDB and change. They cannot be made in cases when CPIA the ADB. Finally, CPIA ratings correlate better for ratings do not change. This restricts the number IBRD than for IDA countries with ratings of other of observations significantly, because CPIA ratings indicators. do not change very much over time. The restricted number of observations per criterion (these are at CPIA Ratings Generation Process most slightly above 20, and in many instances well The central determinant of CPIA ratings is the below 20) weakens the confidence in the analysis; professional judgment of Bank staff, who can also hence the results are not reported here. draw on other indicators (including outcome indicators/hard data) provided as guideposts IBRD versus IDA ratings in the CPIA Questionnaire. The ratings are The Bank's CPIA ratings correlate better with produced in a multistep process, which entails ratings from AfDB and other institutions for IBRD two levels of review--first at the regional level, countries than for IDA countries6 (table 3.5). AfDB and then at the network level (see box 3.2). ratings are closer to those of the Bank for twice In cases where the regions and the networks as many criteria for IBRD than for IDA countries disagree over the final ratings, the networks have (6 versus 3 criteria). Ratings of other the final say unless the regions have supporting CPIA ratings correlate indicators are overwhelmingly closer evidence. better with other to Bank ratings for IBRD than for IDA comparator indicators countries. They are closer to Bank Regional review for IBRD than for ratings for 8 criteria for IBRD countries Interviews with regional reviewers indicate that IDA countries. and for no criteria for IDA countries. there is no one standard review practice across 48 R E L I A B I L I T Y O F T H E C P I A R AT I N G S Table 3.5: Rank Correlations between the CPIA and Other Indicators: IBRD versus IDA Countries Bank ratings correlate better with Bank ratings correlate better with AfDB ratings for other Indicators for CPIA Criterion IBRD countries IDA countries IBRD countries IDA countries q1 n.a. n.a. q2 No difference n.a. n.a. q3 n.a. n.a. q4 n.a. n.a. q5 q6 No difference q7 No difference q8 No difference n.a. n.a. q9 No difference q10 No difference q11 No difference q12 q13 n.a. n.a. q14 n.a. n.a. q15 No difference q16 Source: IEG. Notes: a. A better correlation is defined here as a correlation coefficient that is higher by at least 0.05. b. q4 is excluded from the comparison with AfDB because AfDB defines q4 differently from the Bank. c. For q1, q2, q3, q8, q13, and q14, no other indicators except for AfDB (and ADB) can be identified as comparators, as mentioned. ADB= Asian Development Bank; AfDB= African Development Bank; IBRD= International Bank for Reconstruction and Development; IDA= International Development Association; n. a. = not applicable. regions. One factor that influences the regional generally knowledgeable about their review practices is the size of the region. sectors for many countries in the There is no one standard region, whereas staff in the Chief CPIA review practice For regions with numerous countries, such as the Economist's office are knowledgeable across regions. Africa and the Europe and Central Asia Regions, about all the countries in the region. reviewers undertake statistical exercises using Hence these regional reviews do not entail cross- external indicators to review country team rating country statistical exercises. proposals. In the Africa Region, sector special- ists undertake such exercises, which are used as Further, though a regional review is meant to inputs in the regionwide review process before ensure intraregional rating comparability prior the first round of rating proposals are submitted to submitting the first round of rating propos- to Operations Policy and Country Studies (OPCS). als to OPCS, interviews with regional reviewers indicate that not all regions do that. The varying In smaller regions, such as the Middle East and extents to which regions undertake this review North Africa and South Asia, sectoral staff are may account for the varying extents to which 49 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Box 3.2: The Process of Preparing CPIA Ratings The process begins with a benchmarking phase, which entails In the third step, the network anchors and other central units rating a small representative sample of countries drawn from review the ratings at the Bank-wide (global) level to ensure cross- all the regions (see chapter 1, "Other Changes in the CPIA"). regional comparability of ratings. This is followed by a roll-out phase, during which the rest The fourth step is somewhat different for the benchmarking of the countries are rated. Both phases entail a multistep versus the roll-out phase. For the benchmarking phase, the fourth procedure. step entails a meeting of representatives from Operations Policy In the first step, the country teams generate a set of proposed and Country Services (OPCS), the regions, networks, and central ratings for their respective countries. This step is usually led by departments to review the proposed ratings for all of the criteria country economists with participation from sector specialists and and for all of the benchmark countries, after which the ratings are country management. "frozen" and the roll-out phase proceeds. For the fourth step of In the second step, the Regional Chief Economist offices review the roll-out phase, most of the ratings are finalized through virtual and revise (as necessary) the ratings for the countries within the communication because of the large number of countries involved. respective regions to ensure cross-country comparability within Meetings are only held to discuss the few cases that have not been each region. resolved by virtual communication. Source: IEG, based on interview with OPCS. networks disagree with the regions over the the review, which varies quite significantly across initial rating proposals (table 3.7). criteria (box 3.3). All regional reviewers were asked the open-ended Expert judgment and potential conflict of question of which criteria are difficult to assess. interest Reviewers found that the criteria in the public As indicated by the external panel in the 2004 sector management and institutions cluster were review of the CPIA (World Bank 2004a), the the most difficult, pointing to the lack of data depth of country knowledge by Bank staff is a and the judgment involved. One of the network major strength of the exercise. The practice of reviewers for this cluster indicated that a lot of relying on expert judgment for ratings is also judgment is involved in most of the criteria in used by virtually all of the other indicators against that cluster and pointed out the criterion on which the CPIA was compared in the previous transparency, accountability, and corruption section (table 3.2). as an example. Although the expert judgment of Bank staff is Network review Network review clearly an asset in the CPIA exercise, at the same practices also vary. The review practices of the networks time there is a potential conflict of interest in vary depending on various factors, having staff provide ratings, particularly for IDA including: (i) the extent to which other quanti- countries. This potential for conflict of interest tative indicators are available for cross-checking arises from the fact that ratings produced by the CPIA ratings; (ii) the extent to which any staff are in turn used for allocating IDA resources other information is available on the criteria at for the same countries on which the work all; (iii) the importance the particular network programs of those staff depend. Therefore, staff accords to the exercise (and hence the amount may potentially be upwardly biased in assigning of resources devoted to it); and (iv) the clarity ratings for their countries. The regional review is of the criteria content and associated ease of meant to adjust for such potential biases at the assessment. These factors affect the extent of regional level, although there could still be issues 50 R E L I A B I L I T Y O F T H E C P I A R AT I N G S Box 3.3: The Network Reviews of CPIA Ratings For the economic management cluster (criteria q1 to q3), the administration (q15), and transparency, accountability, and corrup- reviewers read every write-up submitted by the region, as well tion in the public sector (q16), the networks use other quantitative as reports from the International Monetary Fund, Debt Sustain- indicators to cross-check the ratings. ability Assessments, and private sector reports for the country For the gender criterion (q7), the network actually first generate being reviewed. Many reviewers in the Economic Policy and the ratings based on quantitative indicators which are passed onto Debt Department (PRMED) are involved in this exercise, with the region for review. These initial gender ratings are then adjusted each reviewer assigned about six countries. A coordinator if country teams provide additional country-specific information then reviews about 60 percent of all the reviewers' comments that is not captured by the ratings. Some networks supplement to ensure consistency of ratings across countries. assessments based on quantitative indicators with the write-ups Perhaps because of the resource-intensiveness of the review, submitted by the regions. these were also the criteria on which there were the most com- For the financial sector (q5) for fiscal 2009, building human ments. In 2007, for each of the criteria q1, q2, and q3, the network resources (q9), and efficiency of revenue mobilization (q14), the commented on 86 percent of the countries, compared with com- networks review all of the write-ups that were submitted. The ments on an average of 38 percent of the countries for all 16 CPIA network reviewer for q14 supplements these with quantitative criteria (appendix K). Further, these were also the criteria for which indicators from various other sources. the network disagreed with the regional proposals for a higher For the equity of public resource use (q8) and social pro- share of countries than for most other criteria (see table 3.7). tection and labor (q10), the networks' review focuses only on For the criteria on trade (q4), business regulatory environment countries for which the proposed ratings are different from the (q5), financial sector (q6) up to last year, gender (q7), property previous years. For the environment (q11), the network relies rights and rule-based governance (q12), quality of budgetary and more on the region's judgment because it has little other infor- financial management (q13), parts (c) and (d) of quality of public mation on the criterion. Source: IEG, based on interviews with network reviewers. with the levels of the ratings even if the relative and the Caribbean, the networks challenged rankings of countries are adjusted at the regional initial regional proposals for a larger share of level. IDA than IBRD countries. For both IBRD and IDA countries, there was greater disagreement The network review--among other functions--is between network and regions for Europe and meant to adjust potential biases in the levels of Central Asia than for all the other regions (for 16 ratings across regions (box 3.3). The evidence from and 20 percent of Europe and Central the 2007 review process indicates that there was not Asia countries, respectively). For 7 of The networks challenged much difference between IBRD and IDA countries the 16 criteria, the networks disagreed the regional proposals in terms of the extent of network disagreement more often with regional proposals of more often for IDA with the regions' initial rating proposals. For all ratings for IBRD countries, whereas than for IBRD countries countries, the networks disagreed with about 12.5 for 9 of the 16 criteria, they disagreed for all regions except percent of the initial regional ratings proposed more often with ratings for IDA Latin America and for IDA countries, compared with only a slightly countries (table 3.7). the Caribbean. lower share of 11.8 percent of the ratings for IBRD countries (table 3.6). Ratings were more likely to be challenged by the networks when the regions proposed an The differences were much more significant increase from 2006, and much more so for IDA when the comparison was made at the regional than for IBRD countries. Specifically, when the level. For every region except Latin America networks challenged regional proposals, it was 51 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table 3.6: Numbers and Shares of Initial Regional Rating Proposals on Which the Networks Disagreed with the Regions, by Criteria, for 2007 Total For IBRD countries For IDA countries No. of times No. of times No. of times networks Share of all networks Share of all networks Share of all CPIA criterion differed ratings (%) differed ratings (%) differed ratings (%) q1 24 17.1 12 18.5 12 16.0 q2 28 20.0 12 18.5 16 21.3 q3 27 19.3 12 18.5 15 20.0 q4 12 8.6 6 9.2 6 8.0 q5 15 10.7 9 13.8 6 8.0 q6 11 7.9 6 9.2 5 6.7 q7 28 20.0 11 16.9 17 22.7 q8 8 5.7 3 4.6 5 6.7 q9 22 15.7 10 15.4 12 16.0 q10 6 4.3 2 3.1 4 5.3 q11 10 7.1 4 6.2 6 8.0 q12 10 7.1 5 7.7 5 6.7 q13 17 12.1 4 6.2 13 17.3 q14 22 15.7 12 18.5 10 13.3 q15 13 9.3 7 10.8 6 8.0 q16 20 14.3 8 12.3 12 16.0 Total 273 12.2 123 11.8 150 12.5 Source: IEG, based on World Bank data. Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association. Networks challenged found that in 59 percent of the time higher for IBRD countries (86 percent) than for regional proposals more the regions had proposed an increase. IDA countries (73 percent) (table 3.9). Thus, the often for Europe and This ratio was 66 percent for IDA conclusion can be drawn that for those 73 and 86 Central Asia--for 18 countries compared to a much lower percent of instances, there was indeed an upward percent of the ratings 50 percent for IBRD countries (table bias in ratings. However, these instances made up compared to 12 percent 3.8). This indicates that the networks only 6 percent of the ratings for IDA countries and for all regions. perceived more of an upward bias in about 5 percent of the ratings for IBRD countries, the ratings for IDA countries than which implies that there was not a severe upward for IBRD countries. bias in ratings for either group of countries. The role of the networks Regarding the instances in which the Taking into account all disagreements--that is, in validating the ratings networks challenged a rating increase including also instances where regions proposed helps to minimize from the regions, the networks no change or a decrease in ratings in addition to the potential for bias prevailed in an overwhelming majority an increase in ratings--the 73 percent in which in the ratings. of 77 percent of the time. The share was network views prevailed dropped somewhat to 52 R E L I A B I L I T Y O F T H E C P I A R AT I N G S Table 3.7: Number and Share of Initial Regional Rating Proposals on Which the Networks Disagreed with the Regions, by Region and IBRD and IDA Countries for 2007 Number of rating proposals on which networks Share of rating proposals on which networks disagreed with regions disagreed with regions (%) Region Total IBRD IDA Total IBRD IDA Africa 82 10 72 11.4 7.8 12.2 East Asia and Pacific 28 6 22 8.8 4.2 12.5 Europe and Central 81 55 26 17.5 16.4 20.3 Asia Latin America and 55 40 15 12.3 13.2 10.4 Caribbean Middle East and 16 12 4 10.0 9.4 12.5 North Africa South Asia 11 n.a. 11 8.6 n.a. 8.6 Total 273 123 150 12.2 11.8 12.5 Source: IEG, based on data from OPCS. Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association. 68 percent, which is still high (table 3.10). In the criterion on building human resources. For sum, the evidence reflects the central role of the these four criteria, the networks commented on networks in the review process, which helps to the initial regional rating proposals even when minimize potential biases in ratings. the regions did not propose a change in ratings from the previous year (table 3.11). As discussed The networks prevailed in the majority of the cases in box 3.3, these were also the criteria on which for all regions except South Asia and prevailed the networks had invested the most time. For the most often for Latin America and the Caribbean. four criteria the reviewer read all of the write-ups. The networks also prevailed more often for IBRD Additionally, for the three criteria on economic than for IDA countries overall (77 versus 61 management, each reviewer was responsible for percent of the time). This could either mean that the regions have more supporting evidence for the ratings they proposed for IDA than for IBRD countries, or that the regions make a bigger effort Table 3.8: Network Disagreements with Initial for IDA countries because IDA funds are involved Regional Rating Proposals (this evaluation has obtained anecdotal evidence from interviews with World Bank staff on this). IBRD IDA The nature of the network review (that is, the All countries countries countries Regional proposals (%) (%) (%) resource-intensiveness of the review) appears to have an effect on the extent of the review, and Lower rating than 2006 1.9 2.7 1.3 hence possibly on the quality of the ratings. The Same rating as 2006 39.2 47.8 32.7 evidence from the 2007 review process indicated Higher rating than 2006 58.9 49.6 66.0 that four criteria stood out as having been more Source: IEG based on World Bank data. rigorously reviewed than other criteria, including Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development the three economic management criteria and Association. 53 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Conclusions Table 3.9: Share of Instances Where Networks Analysis of the 2007 review process indicates that Prevailed When Networks Disagreed with Regions the networks perceived more of an upward bias over Proposed Increases in Ratings from 2006 in ratings for IDA than for IBRD countries. This upward bias did not seem very severe, as it could All countries IBRD countries IDA countries only be detected in about 6 percent of the ratings Region (%) (%) (%) for IDA countries and about 5 percent of the ratings for IBRD countries. Analysis of the 2007 Africa 74.5 83.3 73.5 review process also indicates that the networks did East Asia 80.0 100.0 78.6 have a central role in the ratings review process, and Pacific as they prevailed in the majority--although not Europe and 69.8 82.6 55.0 all--of the cases when there were initial disagree- Central Asia ments between them and the regions. This helps Latin 92.6 89.5 100.0 to minimize potential biases in the ratings. Finally, America and the quality of the ratings is likely to be enhanced Caribbean by greater intensiveness in the network review, Middle East 81.8 85.7 75.0 which would entail more resources than have and North Africa been provided for the exercise. South Asia 75.0 n.a. 75.0 Country context Total 77.4 85.7 72.7 The greater comparability of the ratings on IBRD Source: IEG, based on OPCS data. than IDA countries with other indicators highlights Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development one issue with respect to the CPIA ratings genera- Association. tion process--that aspect of country context that refers to the stage of development. Specifi- cally, the questionnaire stated, "The criteria were about six countries for which he or she also read developed to ensure that, to the extent possible, reports from the International Monetary Fund their contents are developmental neutral; that (IMF), Debt Sustainability Assessments, and the the higher scores do not set unduly demanding private sector reports. standards, and can be attained by a country that, Table 3.10: Share of Instances Where Networks Prevailed When Networks Disagreed with Regions, 2007 Region All countries (%) IBRD countries (%) IDA countries (%) Africa 68.3 90.0 65.3 East Asia and Pacific 60.7 66.7 59.1 Europe and Central Asia 63.0 70.9 46.2 Latin America and Caribbean 87.3 87.5 86.7 Middle East and North Africa 68.8 66.7 75.0 South Asia 27.3 n.a. 27.3 Total 68.1 77.2 60.7 Source: IEG, based on OPCS data. Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association. 54 R E L I A B I L I T Y O F T H E C P I A R AT I N G S Table 3.11: Shares of Countries on Which Networks Commented in 2007 When Regions Proposed the Same Ratings as in 2006 CPIA criterion All countries (%) IBRD countries (%) IDA countries (%) q1 89.2 88.9 89.6 q2 84.5 89.6 80.0 q3 85.3 88.6 82.4 q4 50.8 58.5 44.8 q5 3.4 5.9 1.5 q6 3.6 5.7 1.7 q7 28.8 28.8 28.8 q8 9.2 9.3 9.2 q9 83.8 88.5 80.0 q10 44.6 44.6 44.6 q11 0.0 0.0 0.0 q12 13.7 16.7 11.4 q13 1.0 0.0 1.9 q14 14.0 18.0 10.9 q15 5.0 7.4 3.1 q16 8.4 8.0 8.8 Total 31.5 33.7 29.6 Source: IEG, based on data from OPCS. Note: IBRD= International Bank for Reconstruction and Development; IDA= International Development Association. given its stage of development [italics by IEG], achieve similar results. Still others interpret it to has a policy and institutional framework that mean the stage of development, but account for strongly fosters growth and poverty reduction." it in different ways, with some ways being more (CPIA Questionnaires 2004­08). (The most subjective than others. Those who account for recent CPIA Questionnaire available at the time the stage of development objectively of this writing was for 2008). adjust the indicators they are assessing The greater comparability by per capita incomes (for the gender of the ratings for IBRD Different practices are adopted by the regions criterion, and for the finance criterion than for IDA countries and networks with respect to country context. until two years ago). Regardless of with other indicators Discussions with regional reviewers indicate how country context is interpreted, highlights the problem that none of the regions take country context many network reviewers find the with accounting for the into account in the rating exercise. Discussions concept difficult to implement, and stage of development with network reviewers indicate that "country had comments such as "...this is the in the CPIA ratings. context" is interpreted quite differently by differ- single toughest thing." ent network reviewers. Some interpret it to mean that the country-specific information provided Aside from the mixed interpretation of "country by the country teams needs to be taken into context"--which can distort the quality of the account in the assessment. Some interpret it to ratings--the fact that judgment is involved in mean that different policies and institutions can accounting for "country context" introduces 55 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Regardless of how it is further subjectivity to an exercise that to adjust the ratings quantitatively, using per interpreted, network already relies centrally on judgment. capita incomes. The issue then arises as to which reviewers find the concept Minimizing the amount of subjectivity criteria or subcriteria to adjust, because the stage of country context involved in the rating exercise would of development is only pertinent for some (sub) difficult to implement. help to enhance the quality of the criteria but not others. For example, the develop- ratings. ment of the financial sector clearly depends on the stage of development of the country. The first interpretation of country context by Therefore, it is reasonable to adjust the indicator network reviewers--taking into account country- being assessed (for example, private sector credit specific information--is reasonable. After all, as a GDP) by per capita incomes, which had the deep country knowledge of Bank staff is the previously been, but is no longer, done by the major value added that the Bank brings to the network reviewer. It is not clear how much the CPIA rating exercise. stage of development matters for other criteria. Thus, deciding which criteria or subcriteria to Accounting for the An example can be found in the gender adjust for the stage of development would itself stage of development is criterion, which assesses, among be controversial. problematic because of other things, the share and growth the judgment involved rate of parliamentary seats occupied In addition to which (sub)criteria to adjust and the uneven practice by women. However, a straightfor- for stage of development, how to adjust such across the Bank. ward use of these indicators could (sub)criteria is also an issue. Although it is be misleading. For example, a recent more straightforward to adjust quantitative study on Bangladesh revealed that although indicators (although the methodology can be gender quotas increased the total number of subject to debate), when it comes to qualita- women in political arenas, their representation in tive indicators (and many CPIA criteria are the decision-making process is still not ensured assessed on such indicators), it is very difficult as elected female representatives in Bangladesh to adjust and the process could be open to a lot face social, cultural, and religious challenges of arbitrariness. which hinder their participation (Panday 2008). This is the very kind of useful qualitative informa- The adjustment of CPIA ratings by the stage of tion that the country team could potentially development has affected the quality of these provide in terms of "country context." ratings, to the extent that some network review- ers indicated to the IEG team that they do not Regarding the second interpretation that differ- use the ratings for their own analytical work. In ent policies and institutions can lead to similar contrast, network reviewers who do not adjust results, the literature review conducted for this the CPIA ratings for the stage of development evaluation indicates that, by and large, the CPIA indicated to the IEG team that they do use the criteria overlap with what are considered as the ratings for their own analytical work, reflecting consensus determinants of growth and poverty the confidence they have in the ratings. reduction by the wider research community, and not just by the Bank (chapter 2). However, the IEG recommends excluding accounting for the trade criterion is problematic and needs to be stage of development from the CPIA exercise. If revised, as discussed in chapter 2. this cannot be done, at the very least it is important to clarify and justify in the CPIA guidelines which Implementation of the third interpretation of criteria need to take into account the stage of country context--the stage of development-- development and how such adjustments need to is also problematic. The simplest way would be be made. 56 Chapter 4 Quay crane on docks, Sri Lanka. Photo by Dominic Sansoni, World Bank Findings and Recommendations T he CPIA criteria are largely relevant for sustaining growth and improving welfare. By and large, the CPIA covers the determinants-- policies and institutions--of growth and poverty reduction identified in the literature. However, some criteria can be streamlined and one needs to be added. The assessment of certain criteria needs to be coordinated, and the content of all criteria reviewed (see recommendations later in this chapter). Overview whether the criteria represent the policies This evaluation takes the premise that beyond and institutions that are important for aid informing IDA allocation, the CPIA is useful as to lead to growth. Much of the literature on a broad indicator of development effectiveness. aid effectiveness uses cross-country empirics to It reviews the appropriateness of the CPIA as estimate the impact of aid on growth, with growth an indicator that assesses the conduciveness of representing aid effectiveness. The review of the a country's policies and institutions to foster- literature indicates that there is limited consensus ing poverty reduction, sustainable growth, and on the impact of aid on growth itself, and on the the effective use of development assistance. It conditions under which aid can have a positive assesses the relevance of the content of the CPIA impact on growth. Also, this evaluation could not through a review of the economics literature. estimate the impact of IDA assistance and CPIA It also assesses the reliability of CPIA ratings in ratings on growth because the restructuring of two ways--through comparing CPIA ratings with the CPIA in 2004 has resulted in a discontinuity similar indicators, and through reviewing the in the CPIA series. CPIA ratings generation process. Based on these assessments, the main findings and recommen- However, CPIA ratings are found to be dations are as follows. positively associated with Bank loan perfor- mance. Specifically, empirical analysis conducted Main Findings for this evaluation finds that the ratings of the CPIA content overall CPIA as well as those for each of the CPIA The evidence is mixed as to whether the clusters are negatively associated with the share CPIA criteria are relevant for aid effective- of problem loans (that are in turn correlated with ness as defined in the literature--that is, loan performance). 59 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Empirical analysis also indicates that there The trade criterion does not reflect the is insufficient evidence from the data to importance of complementary institu- conclude that cluster D associates better tions for improving trade performance. with loan performance than the other Incorporating export performance in the three clusters. The new country performance assessment of the trade criterion would also rating that is used for IDA allocations has made reflect the evidence that integrating into the explicit the relative weights applied to the differ- global economy--an important determinant ent clusters of the CPIA, that is, 8 percent on each of growth--requires integration on both the of CPIA clusters A, B, and C, and 68 percent on export and import fronts. Country experience CPIA cluster D. Neither the literature review on further indicates that complementary factors-- the determinants of growth, poverty reduction including trade facilitation--are also important and development effectiveness, nor the empiri- for export growth and, in fact, more important cal analysis conducted by IEG, has provided than further tariff reduction once countries evidence to justify these (or any other) specific reach moderate tariff levels (which practically weights. It can therefore be surmised that the all countries currently have). Yet not only does way the CPIA is currently being used for IDA the trade criterion of the CPIA focus mostly on allocation--that is, with a large emphasis on the import side, but the much larger weight cluster D--seems to be driven much more by accorded to trade restrictiveness (two-thirds) fiduciary and possibly other concerns of donors than to trade facilitation (one-third) also does than by the objectives of achieving sustained not give enough importance to complementary growth and poverty reduction. factors. The CPIA strives to allow for country specific- Accounting for country specificity requires ity, although there are some potential substantial judgment. Incorporating outcome pitfalls. An important aspect of country specific- variables in the assessment of CPIA criteria allows ity is that different policies and institutions can for country specificity, although it needs to be produce similar outcomes. The CPIA strives to recognized that this entails substantial judgment. provide for this aspect of country specificity in On the one hand, the reason the CPIA focuses its instruction to staff: when assessing policies on assessing policies and institutions is to avoid and institutions, outcomes need to be taken into penalizing countries for not achieving certain account. Indeed, outcome indicators are included outcomes because of exogenous factors. On the in the assessment of certain CPIA criteria (for other hand, assessing outcomes could penalize example, finance and gender), but they could be countries at a lower stage of development for not added to other criteria, in particular trade. achieving those outcomes (such as the share of private sector credit as a share of GDP). Thus, The CPIA does not adequately allow for substantial judgment is needed to take these two country specificity in its trade criterion. The aspects into account in a balanced fashion. way in which the trade criterion is specified does not allow for different approaches to trade liberal- The debate over the weighting scheme is ization that have proven successful in country not very relevant for the use of the CPIA as experiences. The specification of particular tariff a broad index, although it is very relevant rates for different ratings reflects a one-size-fit- in its use in the PBA formula. With respect to all approach to trade liberalization that is not the use of the CPIA as a broad index of policies supported by country experience. Revising this and institutions, the debate over the weight- criterion by changing the way trade restrictive- ing scheme is not very relevant given the high ness is assessed, and including an assessment correlation between the ratings of the CPIA of export performance as the outcome variable, clusters. However, the weights applied to the would allow for more country specificity to be different CPIA clusters do matter for the alloca- incorporated into the criterion. tion of IDA funds. 60 F I N D I N G S A N D R E C O M M E N D AT I O N S These findings raise the question of the useful- have enough information to meaningfully rate ness of aggregating the different CPIA clusters this subcomponent. into an overall index according to any predeter- mined weighting scheme. In the case of the CPIA There are overlaps in the assessment of some as a broad index of development effectiveness, it criteria. Tax policy is an intrinsic part of fiscal does not allow for country specificity which would policy and can be assessed as part of it rather than imply different weights on the clusters depend- separately, as is the case now. Judicial indepen- ing on the initial conditions and the countries' dence is assessed in two different criteria and stage of development. In the case of the CPIA as corruption in three different criteria of cluster D, an indicator for the allocation of IDA funds, the all of which can be streamlined. overall CPIA index is no longer used as such. The assessment of the environment criterion is The CPIA is missing an assessment on other onerous. It requires country teams to answer 85 disadvantaged socioeconomic groups aside questions to arrive at one rating. from gender. Currently, only gender is being assessed with respect to equality. Yet country The three economic management criteria-- evidence indicates that social exclusion of other macroeconomic management, fiscal policy, groups could have severe poverty and growth and debt policy--are conceptually not distinct implications. from each other, and hence need to be, and indeed are assessed together. Yet separate scores Important interlinkages between certain are prepared and published for each of the three criteria are not reflected in the CPIA. Country criteria (for IDA countries), which could lead to evidence indicates that intersectoral labor mobility confusion. needs to be ensured before trade liberalization proceeds. Otherwise, trade liberalization could The in-depth literature review of the exacerbate poverty. Similarly, the assessment financial sector criterion reveals room for of fiscal policy (q2) and the quality of budget- improvement. Although the criterion covers ary and financial management (q13) needs to the dimensions along which finance is currently go hand-in-hand so that the fiscal aspect of the thought to be important--stability, depth and country in its entirety is realistically captured. efficiency, and access--the relative weights of the three dimensions (which are currently equally Assessment or reporting of certain CPIA weighted) need to be revisited. This follows criteria can be streamlined or restruc- from the considerable evidence of a large impact tured. The current content of the criterion on of banking crises on output losses and on the equity of public resource use is redundant. This national budget. At the same time, the evidence does not mean that equity is not important. In on microfinance is mixed. Further, there is fact, equity has been identified in the literature a greater focus in this criterion on assessing as one of the determinants of growth on which intermediate outcomes rather than policies and there is consensus. However, the measures institutions, in particular regarding the financial identified in the literature as being important depth dimension. It would be useful to include for equity--property rights, access to credit, policies and institutions for fostering an enabling access to education, gender equality, and income environment for the financial sector here, transfers--are already covered by other CPIA namely the legal, contractual, informational, and criteria. The criterion is currently assessed on governance framework. Also, the indicators for two fronts--the public expenditure and public assessing financial stability can be strengthened. revenues sides. The former is covered by other CPIA criteria, whereas the assessment of the CPIA ratings latter requires incidence analysis of taxes, which The CPIA ratings correlate relatively well is rarely done. As a result, Bank staff does not with similar indicators in terms of relative 61 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT rankings of countries and direction of of Bank staff depend. However, analysis of the change. For each of the 16 CPIA criteria, the 2007 review process indicates that there did not rank correlations of CPIA ratings with similar seem to be much of an upward bias in ratings for indicators average between 0.7 and 0.8. Other either IDA or IBRD countries. Such an upward indicators are found to correlate better with the bias could be detected in only 6 percent of the CPIA ratings by the Bank than by the AfDB and ratings for IDA countries, and a slightly lower 5 the ADB, which are the closest comparators to percent of the ratings for IBRD countries. the Bank, as they use almost exactly the same CPIA guidelines as the Bank. The multistep review process of the CPIA, with the networks having the central role CPIA ratings correlate better with other in the validation of the ratings, helps guard indicators for IBRD than for IDA countries. against the potential biases in ratings. Specifi- This could be because more information cally, the networks were found to prevail for a large is available on IBRD countries than on IDA majority of the time when they disagreed with the countries, which increases the likelihood of regions over the latter's initial rating proposals in different institutions having similar assess- 2007. Interestingly, the networks prevailed more ments on IBRD countries. This could also be often for IBRD countries than for IDA countries. because the Bank takes into account the stage of This could either mean that the regions have more development when rating countries. This is more supporting evidence for the ratings they proposed pertinent for IDA countries and entails judgment, for IDA than for IBRD countries, or that the which would likely make ratings for IDA countries regions put up a greater effort for IDA countries less comparable with those by other institutions. because IDA funds are involved (this evaluation has obtained anecdotal evidence on that front). Accounting for the stage of development in the CPIA ratings is problematic. As noted, Both the regional and network reviewers accounting for the stage of development could pointed out the high degree of judgment have affected the quality of the CPIA ratings, involved in rating the criteria in the public not only because of the judgment involved, but sector management and institutions also because of the different practices employed cluster (D). This calls into further question across the Bank. None of the regional reviewers the large weight this cluster has in the IDA and only some of the network reviewers take allocation formula. All of the regional review- the stage of development into account in their ers who were asked the open-ended question ratings. The network reviewers who take stage of which criteria they found particularly difficult of development into account do it in a variety of to rate gave cluster D as their response. One of ways--some by adjusting quantitative indicators the network reviewers for cluster D indicated with per capita incomes, and some by their own that a lot of judgment was involved in most of judgment. Further, accounting for the stage of the criteria in that cluster and pointed to the development means that the CPIA is no longer criterion on transparency, accountability, and an index in the true sense of the word. corruption as an example. The strength of the CPIA ratings is Bank Recommendations staff 's professional judgment. The central Based on the above findings, IEG has several determinant of the ratings is the expert judgment recommendations to Bank management of Bank staff (with deep country knowledge), for improving the CPIA as an indicator that which is clearly the major asset of this exercise. represents the policies and institutions that At the same time, however, there is a potential are important for sustaining growth, fostering conflict of interest in having Bank staff provide poverty reduction (or enhancing welfare more ratings that are used for allocating IDA resources broadly), and the effective use of development to those countries on which the work programs assistance. 62 F I N D I N G S A N D R E C O M M E N D AT I O N S These recommendations are aimed at enhanc- · Revision of the trade criterion (q4). A sub- ing the CPIA as a broad indicator of policies and component on exports needs to be added that institutions, more than just as an indicator for the assesses export performance and policies and allocation of IDA funds. If the CPIA were viewed institutions to reduce anti-export bias (such as only in the latter context, then the question export rebate and duty drawback). This new could be raised as to the necessity of rating IBRD subcomponent, and the existing subcompo- countries. nents on trade restrictiveness and trade facili- tation, need to all receive equal weights (that Adoption of these recommendations could result is, one-third weight each). The tariff rates in in a discontinuity in the CPIA ratings, which Bank the trade restrictiveness subcomponent need management has been trying to avoid. However, to be revised to reflect country experience it is important that the CPIA reflect the latest that at moderate tariff levels (which almost thinking in development paradigm and lessons all countries have), further tariff reduction is learned (both of which are stated intentions less important than complementary factors of the Bank regarding the CPIA). It would also (such as macroeconomic stability and trade provide the opportunity to address an issue that facilitation) for global integration. some network reviewers have raised regarding · Dropping or reformulating the criterion on eq- the quality of the ratings for some criteria because uity of public resource use (q8), as the current of what they perceive as inflated baseline ratings content is already covered by other criteria. from a few years ago. The proposed recommen- · Addition of an assessment of other marginal- dations are as follows: ized socioeconomic groups to the CPIA. The assessment of other marginalized socioeco- Disclose ratings for IBRD countries. Disclo- nomic groups could either be added as a new sure is important for accountability and transpar- criterion (in place of the criterion on equity of ency and would further enhance the quality of public resource use, which IEG recommends the ratings. dropping) or added to a reformulated crite- rion on equity of public resource use. Remove accounting for the stage of · Revision of the financial sector criterion (q5). development in the CPIA rating exercise. The weights on the three subcomponents-- If this cannot be done, at the very least the CPIA stability, depth and efficiency, access--need guidelines need to specify and justify which to be revisited in light of the importance of criteria should take into account the stage of stability and the mixed evidence on microfi- development and how the adjustments should nance. Policies, regulations, and institutions be made. for fostering an enabling environment for the financial sector need to be added. The indica- Undertake a thorough review of each tors used in the assessment of financial stabil- CPIA criterion and revise as necessary. It is ity need to be strengthened. recommended that the review entail an in-depth · Combining the assessment of tax policy literature review for each criterion and reflect (q14a) with the assessment of fiscal policy. the latest thinking on development and lessons That part of tax policy that assesses import learned. The review needs to take into account tariffs is already being assessed in the trade the balance between liberalization and regula- criterion, whereas the part on export rebate tion. It also needs to examine the clustering of and duty drawback need to be incorporated the criteria, in particular having social sectors into the revised trade criterion as suggested and the environment in one cluster. Guideposts above. for assessing the criteria need to be reviewed at · Streamlining the assessment of judicial inde- the same time. It is also recommended that the pendence in the public sector management following be taken into account in the review and and institutions cluster. Currently, judicial in- revisions: dependence is assessed in both the criterion 63 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT on property rights and governance (q12) and Consider not producing an overall CPIA the criterion on transparency, accountability, index although continue to produce and and corruption in the public sector (q16). publish the separate CPIA components. · Streamlining the assessment of corruption in IDA is already using the components separately the public sector management and institutions in the PBA formula. With respect to the use of cluster. Currently corruption is assessed in the the CPIA as a broad index of policies and institu- criteria on efficiency of revenue mobilization tions, this would allow for country specificity, (q14), the quality of public administration as different weights could be assigned to the (q15), and transparency, accountability, and different clusters depending on the country's corruption in the public sector (q16). initial conditions and stage of development. · Strengthening the assessment of the environ- Producing the different components of the ment criterion (q11) and make the process CPIA without assigning weights to them to more efficient. Currently, staff need to answer arrive at an aggregate index would allow for 85 questions to arrive at one rating. different weights to be applied according to · Reporting only the consolidated score for the country contexts and use. economic management cluster. 64 Appendixes 65 Landscape view of water, glaciers and mountains, Chile. Photo by Curt Carnemark/World Bank. 66 APPENDIX A. 2008 COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT CRITERIA CLUSTER CRITERIA INDICATORS (weight in index) (weight in index) (weight in index) Economic q1 Macroeconomic · Monetary/exchange rate policy with clearly defined price stability objectives management management · Aggregate demand policies focus on maintaining short- and medium-term external balance. (0.25) (0.083) · Avoid crowding out private investment q2 Fiscal policy · Primary balance managed to ensure sustainability of public finances (0.083) · Public expenditure/revenue can be adjusted to absorb shocks · Provision of public goods including infrastructure consistent with medium-term growth q3 Debt policy · Debt burden indicators do not signal debt servicing difficulties (0.083) · External and internal debt contracted with view to achieving/maintaining debt sustainability · Coordination between debt management and other macroeconomic policies · Debt management unit well established, has adequate system for recording and monitoring debt, and good analytical capacity as indicated by regular analytical work on debt · Accurate, timely, and publicly available debt data · Government has clear financing strategy and the legal framework for borrowing is clearly defined. Structural q4 Trade · 75 percent weight for trade restrictiveness: (0.063) policies (0.083) · Average tariff rates, number of tariff bands, maximum tariff band (0.25) · Internal taxes do not discriminate between imported and local products · Transparency and predictability of trade regime including in the use of non-tariff barriers · 25 percent weight for customs/trade facilitation: (0.02) · Reputation of customs with respect to professionalism and corruption · Use of risk management, information technology (IT), physical examination · Processing of collections and refunds · Documentation of customs procedures · Resolutions of appeals of customs decisions. q5 Financial Sector Financial stability (0.083) · Banking sector's vulnerability to shocks · Banking system soundness (share of non-performing loans [NPLs] and level of capital at risk) · Adherence to Basel Core Principles · Quality of risk management in financial institutions · Quality of supervision. Financial sector efficiency, depth, and resource mobilization · Size and reach of financial markets · Development of capital markets · Interest rate spreads · Private sector credit/GDP · Efficiency of microfinance. (continued on next page) 67 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT CLUSTER CRITERIA INDICATORS (weight in index) (weight in index) (weight in index) Structural Access to financial services policies · Development of payment, clearance, and credit reporting systems (0.25) · Share of population with access to formal sector financial services (cont.) · Access of small and medium enterprises (SMEs) to finance · Legal and regulatory framework supporting access to finance. q6 Business Regulations affecting entry, exit, and competition (0.028) Regulatory · Bans on, or, investment licensing requirements Environment · Entry and exit procedures (0.083) · Legal framework (and implementation thereof) to address anti-competitive conduct by firms · Procurement by public sector firms. Regulations of ongoing business operations (0.028) · Operational licensing, permits, compliance and inspection requirements including taxes and customs · State intervention in goods markets (state ownership in competitive sectors, price controls, state making administrative allocation/decisions about production) · Corporate governance laws (and enforcement thereof) to encourage disclosure and protect shareholders rights. Regulations of goods and factor markets (0.028) · Employment law provides for flexibility in hiring and firing · State intervention in labor and land markets limited to regulation and/or legislation to smooth out market imperfections · Procedures to register property are simple and low-cost. Policies q7 Gender Equality Human capital development (0.017) for Social (0.05) · Differences (between male and female) in primary completion rates, and access to Inclusion/ secondary education (female to male enrollment) Equity · Access to delivery care and family planning services (0.25) · Adolescent fertility rate. Access to economic and productive resources (0.017) · Gender disparities in labor force participation, land tenure, property ownership, and inheritance practices. Status and protection under the law (0.017) · The law gives men and women equal individual and family rights · Violence against women considered a crime · Gender disparities in political participation at the national level. q8 Equity of Public Government spending (0.033) Resource Use · Identification of individuals, groups, or localities that are poor, vulnerable, or have unequal (0.05) access to services and opportunities · Adoption of national development strategy with explicit interventions to assist groups identified above · Systematic tracking of composition and incidence of public expenditures and their results feedback into subsequent allocations. Revenue collection (0.017) · Incidence of major taxes (progressive or regressive) and their alignment with poverty reduction priorities. q9 Building Human Health and nutrition including reproductive health (0.017) Resources · Equitable access to basic health services (0.05) · Prevention of malnutrition. 68 APPENDIX A. 2008 COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT CRITERIA CLUSTER CRITERIA INDICATORS (weight in index) (weight in index) (weight in index) Policies q9 Building Human Education (0.017) for Social (cont.) Resources · Sustained progress toward universal basic education, literacy, and more equitable access to Inclusion/ (0.05) early child development program services Equity · Standards for teacher preparation, student learning, and oversight of private/non- (0.25) governmental organization (NGO) providers (cont.) · Systematic tracking of school performance and student learning outcomes and feedback to schools and parents · Policies for post-basic education and training services · Quality, equity of access, and efficiency of resource use. Human immunodeficiency virus/Acquired immunodeficiency syndrome (HIV/AIDS), tuberculosis, malaria · Prevention, treatment, care and support of HIV/AIDS, tuberculosis, and malaria · Track disease prevalence, resources, and program implementation. · Quality and timeliness of services · Focus on the poor · Cost-effective use of public resources q10 Social Social safety net programs (0.01) Protection and · Social protection programs provide income support to poor and vulnerable groups. Labor (0.05) Protection of basic labor standards (0.01) · Ratification and implementation of international core labor standards. Labor market regulations (0.01) · Labor market regulations on health and safety, working conditions, and hiring and firing. Community-driven initiatives (0.01) · Encourage and support communities' own development initiatives or local accountability mechanisms. Pension and old-age savings programs (0.01) · Pension and savings programs provide income security to most potentially vulnerable groups. q11 Policies and · Regulations and policies (and implementation thereof) for pollution and natural resource Institutions for · Information widely available Environmental · Priority setting Sustainability · Sector ministries incorporate environmental concerns. (0.05) Public q12 Property Rights Legal basis for secure property and contract rights (0.017) Sector and Rule-Based · Transparent and well-protected property rights Management Governance · Current and non-corrupt property registries and (0.05) · Enforced contracts. Institutions Predictability, transparency, and impartiality of laws and regulations affecting economic activity (0.25) (0.017) · Transparent and predictable laws and regulations affecting businesses and individuals · Low-cost means for pursuing small claims · Impartial and predictable applications of laws and regulations. Crime and violence as impediment to economic activity (0.017) · Well-functioning and accountable police force protects citizens and their property from crime and violence. (continued on next page) 69 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT CLUSTER CRITERIA INDICATORS (weight in index) (weight in index) (weight in index) Public q13 Quality of Comprehensive and credible budget linked to policy priorities (0.017) Sector Budgetary · Multiyear expenditure projections integrated into budget formulation process Management and Financial · Spending ministries and the legislature consulted in budget formulation, adhering to fixed and Management budget calendar Institutions (0.05) · Budget classification system comprehensive and consistent with international standards (0.25) · Minimal and transparent off-budget items. (cont.) Financial management (0.017) · Budget implemented as planned · Budget monitoring based on management information systems · Negligible payment arrears. Fiscal reporting (0.017) · Reconciliation of banking and fiscal records · Regular in-year fiscal reporting · Timely preparation of public accounts · Timely auditing of accounts and appropriate action taken on budget reports and audit findings. q14 Efficiency Tax policy (0.025) of Revenue · Bulk of revenues from low-distortion taxes such as sales/value-added tax (VAT), property, Mobilization and so forth (0.05) · Low and relatively uniform import taxes · Functional export rebate or duty drawback · Broad tax base · Few arbitrary exemptions. Tax administration (0.025) · Rule-based tax administration · Low administrative and compliance costs · Taxpayer service and information program · Efficient and effective appeals mechanism. q15 Quality Policy coordination and responsiveness (0.0125) of Public · Effective coordination mechanism to ensure high degree of policy consistency. Administration Service delivery and operational efficiency (0.0125) (0.05) · Organizational structures along functional lines with little duplication; regular review of business processes to ensure efficient decision making. Merit and ethics (0.0125) · Hiring and promotion based on merit and performance; ethics prevail. Pay adequacy and management of the wage bill (0.0125) · Sustainable wage bill that does not crowd out public services spending · Pay and benefit levels adequate · Flexibility in paying higher wages for hard-to-fill positions. q16 Transparency, Accountability of the executive to oversight institutions and of public employees for their Accountability, performance (0.017) and Corruption · Strong public service ethic reinforced by audits, inspections, and adverse publicity for in the Public performance failures Sector (0.05) · Independent and impartial judiciary · Corruption monitored and sanctions implemented. Access of civil society to information on public affairs (0.017) · Results and costs of government decisions clear and communicated to public · Citizens can access government documents at nominal cost · Media independent of government and can fulfill critical oversight roles. State capture by narrow vested interests (0.017) · Conflict of interest and ethics rules for public servants observed and enforced · Top government officials required to disclose income and assets and can be prosecuted for malfeasance. 70 APPENDIX B: PUBLIC SECTOR LITERATURE REVIEW Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Property Rights and Rule-Based Governance Legal basis for secure property and contract rights Transparent and · Property rights associated with per capita · Governments can help the well-protected incomes (Besley 1995). poor with access to credit property rights · Property rights associated with investments by establishing functioning (Acemoglu, Johnson, and Robinson 2005; property rights (Fleisig Bardhan 2006b). 1995). · Existence of market exchange presupposes · Obtaining property rights property rights (Rodrik 2003). over land in urban areas · Positive correlation between property rights helps poor households gain and growth--cross-country studies (Knack and access to credit (De Soto Keefer 1995; Mauro 1995; Hall and Jones 1999; 2000; Fields and others Rodrik 1999; De Soto 2000; Rodrik, Subramanian, 2002). and Trebbi 2002; Summers 2003; Kerekes and · Giving the poor land rights Williamson 2008). enhances their ability to · Positive correlation between property rights and utilize and invest in land growth--micro studies (Mazingo 1999; Johnson, they cultivate (Deolalikar McMillan, and Woodruff 2002). and others 2002). · Increase in protection of property rights across the globe of half of one standard deviation would halve global poverty (Besley and Burgess 2003). · Increased protection of property rights has strong effects in reducing poverty (Acemoglu, Johnson, and Robinson 2001). Current and non- · Use of registration system to establish identity corrupt registries of property owners (Shavell 2003). · De Soto (2000) used counter-examples from Malawi and Peru to underline the importance of a current and non-corrupt registry system. Enforced contracts · Contract enforcement and growth (North 1990; Knack and Keefer 1995; Levine 1998; Hall and Jones 1999; Kaufmann, Kraay, and Mastruzzi 1999; Hellman, Jones, and Kaufmann 2003; Summers 2003). (continued on next page) 71 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Predictability, transparency, and impartiality of laws and regulations affecting economic activity Transparent and · Laws and regulations should be transparent to predictable laws and enable very imperfect courts to verify violations regulations affecting and correct wrongs (Hay, Shleifer, and Vishny businesses and 1996). individuals · Unpredictable regulations are a disincentive to investment and hence lower growth: example (Gyimah-Brempong and Munoz de Camacho 2006). · Growth depends positively on the rule of law (Barro 2003). Low-cost means · La Porta, Lopez-de-Silanes, and Shleifer (2008) for pursuing small emphasize that costly contract claims do claims not protect investors [and hence would be a disincentive to investment]. Impartial and · Predictability and impartiality characterizes predictable the quality of the law enforcement, and richer applications of laws countries have higher quality law enforcement and regulations (La Porta and others 1998). Crime and violence as impediment to economic activity Well-functioning and · The police (and the courts) are most directly · Police corruption especially accountable police involved in determining and defending property in slum areas of poorer force to protect rights (Andvig and Fjeldstad 2008). countries may increase citizens and their · High crime rates may have devastating impacts uncertainty of property property from crime on investments and economic growth (Andvig rights of the very poor and violence and Fjeldstad 2008). (Andvig and Fjeldstad 2008). · Crime has direct costs on firms through theft · Violence disproportionately losses and security-related expenses, which affects the poor in Latin reduce competitiveness and lower investment America, eroding their (Bourguignon 1998). assets and livelihoods · Security issues must be addressed to fully (Heinemann and Verner comprehend the nature and possibilities for 2006). socioeconomic development (Londono 1996; · Victims of crimes are more Moser 1996; Moser and Holland 1997; Inter- likely to be the poorer part of American Development Bank 1997; Ayers 1997; the population (Bourguignon Buvinic, Morrison, and Shifter 1998; Call 2000). 1999; Deolalikar and others · Crime and violence are major obstacles to 2002). development objectives including lost growth in Latin America and the Caribbean (Schneidman 1996; Ayers 1997; Buvinic and Morrison 2000) and through losses in human capital (Heinemann and Verner 2006). Quality of budgetary and financial management Good budgetary and financial management is of particular importance in developing countries because the absence of aggregate fiscal discipline could result in large unsustainable deficits that translate into an unstable macroeconomic environment (high inflation, high interest rates, burgeoning current account deficits) that ultimately retard growth (Fischer 1991; Easterly, Rodriguez and Schmidt-Hebbel, 1995; Gupta and others 2005). More effective public expenditure management and macroeconomic and budget stability are important for public expenditures to better serve the poor (Foster and others 2002). 72 A P P E N D I X B : P U B L I C S E C T O R L I T E R AT U R E R E V I E W Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Comprehensive and credible budget linked to policy priorities Multiyear · Aggregate fiscal discipline depends on the Evidence is mixed: expenditure existence of a medium-term expenditure · Serving the poor more projections framework (Campos and Pradhan 1996). effectively through public integrated into · Rationale for multiyear budget approach based expenditure requires a budget formulation on several potential benefits (Boex, Martinez- medium­term process for process Vasquez, and McNab 2000). budget allocation (Foster and others 2002). · Changes in budgeting and expenditure planning unlikely to generate significantly improved performance where core functions continue to operate inadequately (Fozzard and Foster 2001). Spending ministries · Consulting spending ministries is important for and the legislature them to have the correct perception of their budget consulted in budget constraints, which in turn reduces the possibility formulation, adhering of excess spending (resulting from "aid illusion") to fixed budget (McGillivray and Morrissey 2001a, 2001b). calendar · Explicit rules that put specific limits on spending and borrowing and that impose penalties on overspending by line ministries give central ministries more leverage over claimants (Campos and Pradhan1996). Budget classification · No evidence available · No evidence available · No evidence available system comprehensive and consistent with international standards Minimal and · Integration of all expenditures in budget can transparent off- help improve accountability and transparency budget items by imposing political costs on politicians and bureaucrats for violating rules, and raise quality of budgetary and financial management (Campos and Pradhan 1996). Financial management Budget implemented · No evidence available · No evidence available · No evidence available as planned Budget monitoring · No evidence available · No evidence available · No evidence available based on management information systems Negligible payments · No evidence available · No evidence available · No evidence available arrears (continued on next page) 73 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Fiscal Reporting. This seems to be important for economic growth. In Latin America, countries with better fiscal transparency and additional spending controls have average fiscal surpluses of 1.7 percent of gross domestic product (GDP), whereas those with fewer spending controls and lowest levels of transparency have average deficits of 1.8 percent of GDP (Alesina 1997). Fiscal transparency--and transparency of the banking sector--is crucial in reducing vulnerability to economic shocks, particularly following the Asian Economic Crisis (International Monetary Fund 2000). The latter stemmed from inadequate disclosure of risks by government and banks (Fozzard and Foster 2001). Fiscal reporting is a core element of transparency, empowerment, and accountability (World Bank 2004g). Reconciliation of · No evidence available · No evidence available · No evidence available banking and fiscal records Regular in-year fiscal · No evidence available · No evidence available · No evidence available reporting Timely preparation of · No evidence available · No evidence available · No evidence available public accounts Timely auditing · No evidence available · No evidence available · No evidence available of accounts and appropriate action taken on budget reports and audit findings Efficiency of revenue mobilization Tax policy Bulk of revenues Value-Added Tax (VAT) VAT from low-distortion · VAT is central to a good tax system in most · Although VAT has long taxes (sales/VAT, countries (Bird 2005; Bahl and Bird 2008). been known to be likely property, and so on) · It is a low distortionary manner to raise taxes, regressive (Ahmad and non-cascading, and does not interfere with Stern 1987; Cnossen 2004; production efficiency (Burgess and Stern 1993). Bird 2005), a recent survey · It is also a good instrument to reduce potential of studies of consumption for corruption as it minimizes contact between tax incidence have shown taxpayers and tax administrators and moves significantly less regressive toward self-assessment (Bahl and Bird 2008). results than those reported · It is a broad-based tax which is an important part for similar taxes in earlier of tax policy (Ames and others 2001). surveys (Bird and Gendron 2006). Land (Property) Tax · VATs in developing countries · If well designed and political opposition is well are not always, or not handled, property taxes and especially land taxes necessarily, regressive (Bird can raise substantial revenues (Heady 2001). and Gendron 2007). · Property taxation is a potential source of · Even when VAT is significant income for many municipal and regressive, it is found to be metropolitan authorities to whom central more progressive than the governments have devolved increasing import and excise taxes that responsibilities without commensurate increases it replaced (Gemmell and in fiscal transfers (Bird and Slack 2002; Dillinger Morrissey 2003). 1992; Mikesell 2003). · Taxing property is much easier now because of digital databases of modern property registries (Fjeldstad and Moore 2007). 74 A P P E N D I X B : P U B L I C S E C T O R L I T E R AT U R E R E V I E W Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Tax policy · To the extent that the (cont.) poorest sectors of the society remain outside of the market economy, a VAT may be broadly progressive (Bird and Gendron 2007; Bird and Zolt 2007). Property Tax · Property incomes and property wealth are significantly undertaxed and an important source of inequity (Fjeldstad and Moore 2007). Broad tax base · Exemptions in many developing countries · Exemptions in many and few arbitrary generally protect the interest of powerful groups, developing countries exemptions so reducing exemptions not only raises revenues, generally protect the but also improves economic efficiency and interest of powerful groups, income distribution (Heady 2001). so reducing exemptions not · Most developing countries do not have the only raises revenues, but right conditions (macroeconomic stability and also improves economic a stable political and administrative system) efficiency and income under which exemptions work, with the result distribution (Heady 2001). that exemptions reduce revenues and complicate · Exempting only five narrowly the fiscal system without achieving their stated defined items in Jamaica objectives (Bird and Zolt 2007). cut the VAT burden on the · Tax incentives (through exemptions) are often lowest 40 percent of the distortive and inefficient and divert scarce income distribution in half resources into less than optimal use (McLure (Bird and Miller1989). 1999). · There may be a limited role for simple incentives, for example as part of a growth-oriented fiscal strategy as the East Asian experience suggests (Bird and Chen 1998). Tax administration. Reforms to tax administration are just as important as tax policy reforms for overall fiscal reform (Mookherjee and Das- Gupta 1995; Devas, Delay, and Hubbard 2001). It is estimated that two-thirds of the rapid increase in Argentina's tax revenues (from 13 to 23 percent of GDP) over the 1989­92 period was attributable to improved administration effort. Rule-based tax · An efficient tax administration has to be rule administration based--the legal environment is important (Bird 2003). · A stable transparent tax system inspires more confidence in its fairness and will result in greater compliance (Boskin 2006). · A less discretionary tax administration is likely to encourage political mobilization of taxpayers around taxation issues, and reduce temptations to pursue corrupt deals (Moore 2004). (continued on next page) 75 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Low administrative · One of the most efficient ways to facilitate tax and compliance costs compliance is to decrease compliance costs (Bird 2003). · Low compliance costs also lower the potential of corruption as they lower the amount of a bribe a taxpayer might be willing to pay to avoid taxes (Bahl and Bird 2008). · It lowers the costs of operating in the formal sector and hence facilitates growth (Bird 2008). · It has a direct effect on economic growth since high compliance costs divert resources toward administering and complying with taxes (Bird 2008). Tax payer service · Provision of extensive information for taxpayers and information reduces compliance costs (Bird 2003; Braithwaite program 2003; Fjeldstad and Moore 2007). · Taxpayer information programs reduce the potential of corruption, and of non-compliance (Bahl and Bird 2008). Efficient and · A time-bound appeal procedure reduces delays effective appeals in payments (Mexico versus India) (Mookherjee mechanism and Das-Gupta 1995). Quality of public administration This is a very important element in the literature on sustained growth. The literature finds that administrative reforms can foster faster economic growth and sustain poverty reduction by removing the obstacles to private sector development that a poorly performing public sector creates. Reforms can also: increase public resources for priority spending; reduce corruption; and increase accountability of the public sector. A seminal paper (Mauro 1995) finds that the efficiency of the bureaucracy is associated with better rates of investment and growth. Deolalikar and others (2002) underline that administrative reforms, including reform of the bureaucracy and civil service, are one of the major areas of reform involving public institutions for poverty reduction. Policy coordination and responsiveness; service delivery and operational efficiency Effective · Improving the quality of the bureaucracy requires coordination and improving coordination among agencies with organizational overlapping functions; organizational structures structures along should be formed along functional lines functional lines (Deolalikar and others 2002). Merit and ethics Hiring and promotion · Inefficiency of public administration stems from based on merit and constraints imposed by the civil service system performance; ethics on human resource management, especially prevail hiring, firing, promotion, and rewards (Devas, Delay, and Hubbard 2001). 76 A P P E N D I X B : P U B L I C S E C T O R L I T E R AT U R E R E V I E W Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Quality of public · Using a "Weberianness Scale" that measures administration meritocratic recruitment and long-term, (cont.) predictable, rewarding careers, Evans and Rauch (1999) found that these characteristics significantly enhance prospects for economic growth even after taking into account initial levels of per capita GDP and human capital. This is because longer­term horizons associated with predictable, rewarding careers will increase the bureaucracy's propensity to advocate public sector infrastructure rather than consumption expenditures. Meritocratic recruitment also increases competence. · Effective bureaucracy is important, or even essential, for implementing or maintaining a policy environment conducive to economic growth (Rauch and Evans 2000). Pay adequacy and management of wage bill Sustainable wage · Containing salary expenditures (through bill does not crowd downsizing) will help increase public resources out public services for priority spending (Deolalikar and others 2002; spending Bardhan 2006b). Pay adequacy and Mixed evidence: management of the · Seminal paper (Becker and Stigler 1974) shows wage bill that high wages paired with non-zero audit probability could be used to deter misbehavior and corruption. · Empirically, no evidence that wages deter corruption (Van Rijckeghem and Weder 1997; Rauch and Evans 2000; Treisman 2000). · Higher wages are correlated with higher corruption (La Porta and others 1999). · Di Tella and Schargrodsky (2003) explain the apparent empirical failure of the Becker-Stigler hypothesis because the empirical studies include observations from environments with no active audits where the probability of being punished for corruption is near zero, or very high auditing levels where the probability of being punished is near one. · Incentive pay structure is one of the most effective ways of fighting corruption (Bardhan 1997, 2006a; Chand and Moene 1999; Van Rijckeghem and Weder 2001; Di Tella and Schargrodsky 2003; Andvig and Fjeldstad 2008). (continued on next page) 77 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Pay adequacy and · Consensus among international organizations management of and leaders of African states that one of the wage bill causes of poor tax administration is low wages (cont.) of officials (Werlin 1979; Due 1988; Kiser and Sacks 2007; Jenkins 1994; Devas, Delay, and Hubbard 2001). · Adequate pay needed to attract competent individuals for budgetary institutions (Campos and Pradhan 1996) and for the judiciary (Posner 1998). Flexibility in paying · Adequate compensation is particularly important higher wages for tax officials in developing countries (with large informal sectors, low levels of literacy and public morality, poor salary structure for public servants, poor communications, malfunctioning judicial systems, and entrenched interests against radical reforms) (Bird 2003). · Reforms in tax enforcement in many countries which include a bonus to the tax officer based on tax collection have often been associated with greater tax compliance, higher revenues, and lower corruption (Mookherjee 1995; Mookherjee and Das-Gupta 1995). Transparency, accountability, and corruption in the public sector Accountability of the executive to oversight institutions and of public employees for their performance. Accountability is very important in the literature on sustained growth, poverty reduction and the effective use of development assistance. A number of empirical studies show the benefits of accountability for the quality of government (Besley and Case 1995; La Porta and others 1999; Adsera, Boix, and Payne 2003; Eijffinger and Geraats 2005; Olken 2007; Dyck, Moss, and Zingales 2008; Ferraz and Finan 2008; Bjorkman and Svensson 2009; Djankov and others 2009). Accountability of elected officials is also found to be directly important for economic growth (Kaufmann, Kraay, and Zoido-Lobaton 1999; Besley and Burgess 2003). Strong public service · A randomized field experiment using over 600 · For development ethic reinforced by village road projects in Indonesia finds that assistance to be used audits, inspections, the probability of external audits substantially more effectively, and adverse publicity reduces missing funds in the project, and the donors should for performance benefits of the audits exceeded their costs reinforce or support failures (Olken 2007). audit findings of · Monitoring policies (together with doubling of Auditor Generals of wages) results in a large decline in prices paid by the countries (Stevens all public hospitals in Buenos Aires for a number 1999). of very basic supplies (Di Tella and Schargrodsky 2003). · Increased government monitoring and media campaign informing local communities of their entitlement to school funds from the central government of Uganda reduced diversion of funds by intermediating provincial governments from 80 to 20 percent (Reinikka and Svensson 2004). Independent and · Discussed in the property rights and rule-based governance section. impartial judiciary 78 A P P E N D I X B : P U B L I C S E C T O R L I T E R AT U R E R E V I E W Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Transparency, There are two strands of literature, with the strand Corruption increases income Aid has a larger effect accountability, and that finds corruption having a negative impact on inequality: a worsening in in displacing domestic corruption in the growth dominating. the corruption index by one revenues in countries public sector standard deviation increases with higher levels of (cont.) Corruption has a positive impact on growth: the Gini coefficient by 5.4 corruption--doubling of · Where there are pre-existing policy distortions points (Gupta and others 1998). grants as a share of GDP Corruption monitored including pervasive and cumbersome regulations, Corruption increases income is associated with a 1.3 and sanctions corruption may help efficiency and growth (Leff inequality in a sample of percentage point decline implemented 1964; Huntington 1968; Lui 1985; Beck and developing countries and in revenues as a share of Maher 1986; Lien 1986; Bardhan 1997). decreases the share of GDP in relatively corrupt government expenditures on countries, and as much Corruption has a negative impact on growth when: education and health (Gupta, as 3.8 percentage point · Corruption is directly related to variations in the Davoodi, and Terme 2002). decrease in the most growth of per capita income (Knack and Keefer corrupt countries (Gupta 1995). and others 2003). · Corruption and red tape are significantly associated with increased levels of investment, which is shown empirically to be one of the most powerful predictors of growth (Mauro 1995). · Corruption has adverse effects on investment and growth (Bardhan 1997, 2006a). · Corruption is associated with higher military spending as a share of GDP, hence reduction in corruption will improve composition of government spending toward more productive, non-military outlays (Gupta, de Mello, and Sharan 2000). · Higher bribes imply lower profitability of productive relative to rent-seeking investments (Kaufmann, Kraay, and Zoido-Lobaton 1999; Tanzi 1998; Acemoglu and Verdier 2000). · Corruption increases uncertainty, hence reducing investment in physical and human capital (Wei 2000; Alesina and Weder 2002). · Societies' willingness to tax themselves depends on perception that government institutions are honest; hence corruption, VAT evasion, and size of underground economy are found to be closely linked (Bird and Gendron 2006). · Corrupt police may have a negative impact on growth by taxing businesses through predation, or by supplying protection in competition or in cooperation with organized crime units (Andvig and Fjeldstad 2008); police corruption is shown to have significant negative impacts on business activities (Wei 2000). (continued on next page) 79 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Country Policy Literature on: and Institutional Assessment (CPIA) Development Criteria/Indicators Sustained growth Poverty reduction effectiveness Access of civil society to information on public affairs Media independent · Free flow of information pressures (even · No country with a free press · Provision of more of government nondemocratic) governments to public action has ever had a major famine information (for and fulfill critical (Dreze and Sen 1995). (Dreze and Sen 1989). example through oversight roles · Local newspapers increase responsiveness of the use of local Indian state governments to natural disasters newspapers) (Besley and Burgess 2002). increases aid · Media campaign via radio and newspapers efficiency and reduces informing local communities of their entitlement the negative impact to school funds from the central government of aid volatility on of Uganda (along with increased government aid efficiency (Cagé monitoring) reduced diversion of funds by 2009b). intermediating provincial governments from 80 to 20 percent (Reinikka and Svensson 2004). · Negative correlation between press freedom and corruption (Ahrend 2001; Brunetti and Weder 1999). · Mass media makes governments more accountable (Besley, Burgess, and Prat 2002). · Media can provide a counterbalance to the power of vested interests especially by informing voters (Dyck, Moss, and Zingales 2008). State capture by narrow vested interests. Vested interests are a restraint for economic growth (Mokyr 1990; Maine 1980; and Landes 1983). Vested interests (such as those arising from import substitution policies or capital taxes) can be detrimental to economic growth (Krusell and Rios-Rull 1996; Acemoglu and Robinson 2000). Top government · Disclosure is correlated with lower corruption officials required when it is public, identifies sources of income to disclose income and conflicts of interest, and when a country is a and assets and can democracy; but there is no significant evidence be prosecuted for of benefits from disclosure of values of income, malfeasance consumption, and wealth (Djankov and others 2009). 80 APPENDIX C: 2007 CPIA CRITERIA ON ECONOMIC MANAGEMENT, STRUCTURAL POLICIES, AND POLICIES FOR SOCIAL INCLUSION/EQUITY AND EVIDENCE IN THE LITERATURE Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Macroeconomic · Monetary/exchange rate · Sound money and · Adverse impact of Management policy with clearly defined fiscal solvency (Rodrik inflation on the poor (0.083) price stability objectives. 2003). (Easterly and Fisher · Aggregate demand policies · Keep inflation low 2001). focus on maintaining short- and stable; monetary · Inflation hurts the and medium-term external policy stance poor (Dollar and Kraay balance. consistent with low 2002; Ravallion and · Avoid crowding out private and stable inflation Datt 2002). investment. (World Bank 2005). · Potential effect of inflation on poverty, once controlled for direct effect of growth on poverty, is mixed (Epaulard 2003). Fiscal Policy · Primary balance managed · Transparency, · Public investment in · Randomized literature (0.083) to ensure sustainability of sustainable solvency, basic infrastructure on infrastructure: public finances. flexibility, pro-growth facilitates access of water and sanitation · Public expenditure/revenue structure of budgets the poor to markets infrastructure (Ashraf, can be adjusted to absorb (World Bank 2005). or to basic social Berry, and Shapiro shocks. · Infrastructure services (Loayza 1996; 2007; Kremer and · Provision of public goods (Easterly 2001; World Calderon and Serven others 2008). including infrastructure Bank 1994; Sachs 2003, 2004). consistent with medium- 2005, 2008; Collier term growth. 2007). Debt Policy · Debt burden indicators do · Reduce recourse to (0.083) not signal debt servicing external debt (World difficulties. Bank 2005). · External and internal debt contracted with view to achieving/maintaining debt sustainability. · Coordination between debt management and other macroeconomic policies. (continued on next page) 81 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Debt Policy · Debt management unit well (0.083) established, has adequate (cont.) system for recording and monitoring debt, and good analytical capacity as indicated by regular analytical work on debt. · Accurate, timely, and publicly available debt data. · Government has clear financing strategy and the legal framework for borrowing is clearly defined. Trade 75 percent weight for trade · Need to take into · No conclusive (0.083) restrictiveness: (0.063) account promotion of evidence in the · Average tariff rates, export growth (World relationship between number of tariff bands, Bank 2005, 2008a). trade liberalization maximum tariff band. · Need to have and poverty (Winters, · Internal taxes do not complementary McCulloch, and discriminate between policies (World Bank McKay 2004; Harrison imported and local 2005). 2006; Ravallion 2004). products. · For trade liberalization · Transparency and to have positive predictability of trade impact on poverty regime including in the use reduction, needs of non-tariff barriers. to be accompanied by improved 25 percent weight for infrastructure, customs/trade facilitation: adequate competition (0.02) policies, enhanced · Reputation of customs with access to credit, respect to professionalism better education and corruption. and health, and · Use of risk management, low marketing or information technology, intermediation costs physical examination. (Balat, Brambilla, and · Processing of collections Porto 2007; Harrison and refunds. 2006). · When trade reforms are accompanied by labor market reforms (hire and fire, relocation), the adverse impact of trade liberalization on poverty disappears (Goldberg and Pavcnik 2005). 82 A P P E N D I X C : 2 0 0 7 C P I A C R I T E R I A F O R C L U S T E R S A , B A N D C A N D E V I D E N C E I N T H E L I T E R AT U R E Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Trade · Documentation of customs · Negative impact of (0.083) procedures. trade reforms on (cont.) · Resolutions of appeals of poverty is due to customs decisions. limited labor mobility (Topalova 2004, 2005). Financial Sector Financial stability · Considerable evidence · Crises do not (0.083) · Banking sector vulnerability of large impact systematically worsen to shocks of banking crises the Gini coefficient · Banking system soundness on output losses although the poor (share of non-performing (Hoggarth, Reis, and are likely less able to loans and level of capital Saporta 2002). absorb adverse shocks at risk). (Honohan 2004a). · Adherence to Basel Core Principles. · Quality of risk management in financial institutions. · Quality of supervision. Financial sector efficiency, · Finance is a · Financial (banking) depth, and resource determinant of depth is negatively mobilization sustained growth associated with · Size and reach of financial (Aghion, Howitt, and headcount poverty markets. Mayer-Foulkes 2005; (Honohan 2004b). · Development of capital Krebs 2003; Levine · Impact of national markets. 1997, 2003, 2005; income volatility · Interest rate spreads Levine, Loayza, and on child labor is · Private sector credit/gross Beck 2000). insignificant for domestic product (GDP). · Efficient domestic countries with deep · Efficiency of microfinance. financial system is financial systems important for growth (Dehejia and Gatti Access to financial services (World Bank 2005). 2002). · Development of payment, · Cross-country · Financial development clearance, and credit regressions do reduces income reporting systems. not suggest any inequality by · Share of population with strong influence of disproportionately access to formal sector household financial boosting the incomes financial services. access on growth of the poor (Beck, · Access of small and or poverty reduction Demirguc-Kunt, and medium enterprises (SMEs) (Honohan 2008a). Levine 2004). to finance. · Access of SMEs to · Emergence of · Legal and regulatory finance promotes firm microfinance as a framework supporting entry, firm growth, source of credit is access to finance. innovation, and size both efficient and distribution of firms equitable as it has (Beck, Demirguc-Kunt, enabled the poor and Levine 2005; to invest, thereby Ayyagari, Demirguc- promoting growth Kunt, and Maksimovic and reducing poverty 2007; Klapper, Laeven, (Khandeker 2005). and Rajan 2006). (continued on next page) 83 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Financial Sector · Informational, · Massive social (0.083) regulatory, legal, and banking experiment (cont.) judicial framework in India has led to (World Bank 2005). significant falls in rural poverty (Burgess and Pande 2005). · Impact assessments show microfinance in general helps the poor (Hossain 1988; Hashemi, Schuler, and Riley 1996; Khandker 1998, 2005; Khandker and Pitt 2003). Business Regulations affecting entry, · Red tape (Mauro · Improve climate Regulatory exit, and competition (0.028) 1995). for doing business Environment · Bans on or investment · Reducing risks for (Besley and Burgess (0.083) licensing requirements. private investors 2003). · Procedures to enter or exit. (World Bank 2000c). · Legal framework (and · Clear and transparent implementation thereof) to business environment address anti-competitive (World Bank 2000c). conduct by firms. · Procurement by public sector firms. Regulations of ongoing · Importance of investor business operations (0.028) protection (La Porta · Operational licensing, and others 2000). permits, compliance and · Importance of labor inspection requirements market reforms for including taxes and minimizing adverse customs. effects of trade reform · State intervention in goods on the poor (Goldberg markets (state ownership and Pavcnik 2005; in competitive sectors, Topalova 2004, 2005; price controls, state Welch, MacMillan, making administrative and Rodrik 2004). allocation/decisions about · Getting the labor production). market right (World · Corporate governance laws Bank 2008a; Ocampo (and enforcement thereof) 2003; Cardenas, to encourage disclosure Ocampo, and Thorp and protect shareholders 2000). rights. Regulations of goods and factor markets (0.028) · Employment law provides for flexibility in hiring and firing. 84 A P P E N D I X C : 2 0 0 7 C P I A C R I T E R I A F O R C L U S T E R S A , B A N D C A N D E V I D E N C E I N T H E L I T E R AT U R E Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Business · State intervention in labor Regulatory and land markets limited Environment to regulation and/or (0.083) legislation to smooth out (cont.) market imperfections. · Procedures to register property are simple and low cost. Gender Equality Human capital development · Education for girls · Higher gender (0.05) (0.017) (World Bank 2006b, inequality increases · Differences (between 2008a). poverty, and female male and female) in · Fertility (Easterly literacy is one of primary completion rates, 2001). the most important and access to secondary determinants of the education (female to male effects of growth on enrollment). income poverty (World · Access to delivery care and Bank 2001a; Ravallion family planning services. and Datt 2002). · Adolescent fertility rate. Access to economic and · Educating girls and productive resources (0.017) integrating them · Gender disparities in labor into the workforce force participation, land is one way to break tenure, property ownership the intergenerational and inheritance practices. cycle of poverty (World Bank 2008a). Status and protection under the law (0.017) · The law gives men and women equal individual and family rights. · Violence against women considered a crime. · Gender disparities in political participation at national level. Equity of Public Government spending (0.033) · Inequality is · Inequality reduces Resource Use · Identification of individuals, negatively related to the poverty reduction (0.05) groups or localities that are growth: conceptual impact on growth poor, vulnerable, or have papers (Todaro 1997; (Ravallion 2001). unequal access to services Aghion, Caroli, and and opportunities. Garcia-Penalosa 1999; · Adoption of national Bardhan 2000; Hoff development strategy with and Stiglitz 2001). explicit interventions to · Inequality is assist groups identified negatively related above. to growth: empirical · Systematic tracking of evidence (Alesina and composition and incidence Rodrick 1994; Perotti of public expenditures and 1992, 1993, 1996; their results feed back into Persson and Tabellini subsequent allocations. 1994). (continued on next page) 85 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Equity of Public Revenue collection (0.017) · Channels through Resource Use · Incidence of major taxes which inequality (0.05) (progressive or regressive) affects growth: credit (cont.) and their alignment with constraints (Perotti poverty reduction priorities. 1992; Bardhan 2000; the World Bank 2005); political instability, crime, insecurity of property rights (Alesina and Perotti 1996; Bardhan 2000); volatility (Aghion, Caroli, and Garcia- Penalosa 1999); social tension and violent redistribution (Bourguignon 2004). · Micropanel studies show households with few physical and human assets caught in poverty trap reducing chance of economic advancement and lowering overall growth (Christiaensen, Demery, and Paternostro 2002; Woolard and Klasen 2005). · Redistribution measures--income transfers (Skoufias, Davis, and de la Vega 2001; Bourguignon Ferreira, and Leite 2003). Building Human Health and nutrition including Mixed evidence: · Positive effects of Resources (0.05) reproductive health (0.017) · Higher life expectancy health interventions · Equitable access to basic raises growth by aid agencies and health services. (Jamison, Sachs, non-governmental · Prevention of malnutrition. and Wang 2001), per organizations (NGOs) capita income (Weil (see appendix E). 2005), and incentive to acquire schooling (Kalemli-Ozcan, Ryder, and Weil 2000). 86 A P P E N D I X C : 2 0 0 7 C P I A C R I T E R I A F O R C L U S T E R S A , B A N D C A N D E V I D E N C E I N T H E L I T E R AT U R E Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Building Human · Major international · Randomized literature Resources (0.05) health improvements has found a number of (cont.) since 1940s have led aid interventions to be to a larger increase effective in education in population than (see appendix E). incomes, hence there is no evidence that health improvements raised income per capita (Acemoglu and Johnson 2007). Education (0.017) · Educational · Adequate and · Sustained progress toward attainment (Easterly effective delivery of universal basic education, 2001). education, health, and literacy, and more equitable · Primary enrollment social infrastructure access to early child rate (Hanushek and (World Bank 2004e). development program Kimko 2000; Barro and · Education enables services. Sala-i-Martin 2003; people to make · Standards for teacher Doppelhofer, Miller, use of economic preparation, student and Sala-I Martin opportunities created learning, and oversight of 2004; Hanushek and by the growth process private/NGO providers. Woessmann 2008). (Dreze and Sen 2002). · Systematic tracking of · Secondary and · Improving literacy school performance and tertiary education facilitates more pro- student learning outcomes (World Bank 1999; poor growth (Chhibber and feedback to schools Vandenbussche, and Nayyar 2007). and parents. Aghion, and Meghir · Poor educational · Policies for post-basic 2006; Aghion and outcomes reduce education and training Howitt 2009). the poverty-reducing services. impact of growth · Quality, equity of access, (Menezes-Filho and and efficiency of resource Vasconcellos 2004). use. · Investment in education can be used to attack poverty as a method to redistribute to the poor (Besley and Burgess 2003). Human immunodeficiency Mixed evidence: virus/acquired · Greater the immunodeficiency prevalence of malaria, syndrome (HIV/AIDS), lower the per capita tuberculosis, malaria income (Sachs 2003). · Prevention, treatment, care and support of HIV/AIDS, tuberculosis, and malaria. · Track disease prevalence, resources, and program implementation. (continued on next page) 87 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Building Human · Quality and timeliness of · Short-run eradication Resources (0.05) services. of malaria lowers (cont.) · Focus on the poor. per capita income · Cost-effective use of public (because malaria resources. affects mainly young children) whereas it raises per capita income only slightly over long run (Ashraf, Lester, and Weil 2008). · Eradication of tuberculosis raises per capita income slightly in both short and long run (Ashraf, Lester, and Weil 2008). · HIV/AIDS associated with small reduction in per capita income over long-run (Kambou, Devarajan, and Over 1992; Over 1992; Cuddington 1993; Cuddington and Hancock 1994; Haacker 2002). · HIV/AIDs has no impact on per capita income (Bloom and Mahal 1997). · HIV/AIDS has large negative impact on per capita income (Bell, Devarajan, and Gersbach 2006). · HIV/AIDS increases per capita income of survivors (Young 2004). Social Protection Social safety net programs · Getting the labor Child labor may lead and Labor (0.05) (0.01) market right to foster to intergenerational · Social protection programs growth and, at the transmission of poverty: provide income support to same time, protecting "child labor traps" poor and vulnerable groups. people (not jobs) (Barham and others Protection of basic labor through establishing 1995; Ilahi, Orazem, and standards (0.01) social safety nets Sedlacek 2001; Emerson · Ratification and (World Bank 2008a; and Souza 2003; implementation of Ocampo 2003). Edmonds 2007). international core labor standards. 88 A P P E N D I X C : 2 0 0 7 C P I A C R I T E R I A F O R C L U S T E R S A , B A N D C A N D E V I D E N C E I N T H E L I T E R AT U R E Development CPIA criteria/indicators Sustained growth Poverty reduction effectiveness Social Protection Labor market regulations · Empowerment of and Labor (0.05) (0.01) communities as (cont.) · Labor market regulations a determinant of on health and safety, sustained growth working conditions, and (Stern 2001; World hiring and firing. Bank 2000c, 2004e). Community-driven initiatives (0.01) · Encourage and support communities' own development initiatives or local accountability mechanisms. Pension and old-age savings programs (0.01) · Pension and savings programs provide income security to most potentially vulnerable groups. Policies and · Regulations and policies · Debate on whether · Policies reducing Institutions for (and implementation environmental environmental risk Environmental thereof) for pollution and regulation can not necessarily Sustainability natural resources. raise growth rate progressive (Parry and (0.05) · Information widely (Bovenberg and others 2005; Fullerton available. Smulders 1995, 1996; 2008). · Priority setting. Hettich 1998; and · Sector ministries Esty and Porter 2005; incorporate environmental versus Fullerton and concerns. Kim 2006). 89 Water supply in Marracuene, Mozambique. Photo by Eric Miller/World Bank 90 APPENDIX D: LITERATURE REVIEW ON AID EFFECTIVENESS A. Aid Growth with No Conditions Short /Long-term Period of Article (year) Type of aid impact analysis Hansen and Tarp (2000) All aid Short-term (4 years) 1974­1993 Dalgaard and Hansen All aid Short-term (4 years) 1974­1993 (2001) Hansen and Tarp (2001) All aid Short-term (4 years) 1974­1993 Ram (2003) Bilateral Short-term (4 years) 1970­1993 Clemens, Radelet, and Short impact aid (budget support or "program" aid for any Short-term (4 years) 1970­2001 Bhavnani (2004) purpose, and project aid given for real sector investments for infrastructure or to directly support production in transportation, communications, energy, banking, agriculture, and industry). Headey (2005) Multilateral and bilateral, separately Short-term (4 years) 1970­2001 Reddy and Minoiu (2006) Developmental (multilateral and bilateral) Long-term (2 decades) 1960­2000 91 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT B. Aid Growth with Conditions Type of Short /Long-term Period of Article (year) aid impact analysis Conditions Burnside and Dollar (2000) All aid Short-term (4 years) 1970­1993 Index of policies: trade openness (Sachs-Warner), (net flow) fiscal policy (budget surplus), monetary policy (inflation rate). Guillaumont and Chauvet All aid Long-term (12 years) 1970­1993 Index of environment/vulnerability: climatic (2001) shocks (stability of agricultural value added), trade shocks (stability of real value of exports, trend of terms of trade), and structural exposure to these shocks (population size). Collier and Dehn (2001) All aid Short-term (4 years) 1970­1993 Countries suffering sharp price drops in key (net flow) commodity exports. Collier and Dollar (2002) All aid Short-term (4 years) 1974­1997 Country Policy and Institutional Assessment (CPIA) (overall for 20 components). Collier and Hoeffler (2004) All aid Short-term (4 years) 1974­1997 A few years after civil war with good policies (first social, then sectoral, then macro). Dalgaard, Hansen, and Tarp All aid Short-term (4 years) 1970­1993 Countries outside of the tropics. (2003) C. Aid Has No Impact on Growth Article (year) Type of aid Short/Long-term impact Period of analysis Boone (1994) Easterly, Levine, and Roodman All aid Short-term (4 years) 1970­1997 (2004) Easterly (2003) All aid Long-term (12 or 24 years) 1970­1993 Roodman (2007a) All aid Tested Hansen and Tarp (2001) in section A and all six papers in section B above Rajan and Subramanian (2008) All aid Medium (10­20 years) and long-term (30­40 1960­2000 years) D. Aid Has Negative Impact on Growth Article (year) Type of aid Short/Long-term impact Period of analysis Ram (2003) Multilateral Short-term 1970­1993 Reddy and Minoiu (2006) Geopolitical (all aid-developmental aid) Long-term (2 decades) 1960­2000 92 APPENDIX E. EXAMPLES OF POSITIVE IMPACTS OF AID PROJECTS FROM RANDOMIZED EVALUATIONS IN EDUCATION, HEALTH, INFRASTRUCTURE, AND AGRICULTURE Education cash-for-schooling program in Mexico, which Angrist and others (2002) find that lottery winners had health components, also had a major health of vouchers for private schools in Colombia impact. The Bobonis, Miguel, and Puri-Sharma had 0.12­0.16 additional years of schooling, (2006) study on anemia and school participation test scores that were higher by 0.2 standard finds that iron supplements and de-worming deviations, and higher secondary school comple- drugs were effective in increasing children's tion (the last finding was confirmed in a follow-up weight-for-height and weight-for-age scores. study by Angrist, Bettinger, and Kremer (2006). Another area in which randomized evaluations Vermeersch and Kremer (2003) find that a school found success is in preventing or treating infant meals program in preschools in Kenya raised diarrhea (Zwane and Kremer 2007). attendance rates from 21 to 29 percent. Kremer and Holla (2008) find that a merit scholarship Infrastructure for high school girls in Kenya seemed to induce Ashraf, Berry, and Shapiro (2007) find that water greater study effort and raised the girls' test purification tablets in Zambia are an inexpensive scores, and even had some externalities to boys' way of avoiding waterborne illness. Kremer and performance in the same classroom. others (2008) show that investment in protecting naturally occurring springs from contamination More generally, as underlined by Kremer and led to dramatic improvements in water quality in Holla (2008), evidence is accumulating on the rural Kenya. effectiveness of certain school inputs such as extra teachers and textbooks (for example, see Agriculture Banerjee and others 2005; Duflo, Kremer, and Duflo, Kremer, and Robinson (2007) find Robinson 2007; and Glewwe, Kremer, and Moulin that selling vouchers earmarked for fertilizer 2007); provider incentives (Glewwe, Holla, and purchases to the farmers right after harvest Kremer 2008 and Muralidharan and Sundara- solved the problem of the farmers not setting raman 2007); remedial education (Banerjee aside funds for fertilizer for the next season. and others 2007; Duflo, Kremer, and Robinson Duflo, Kremer, and Robinson (2008) find a large 2007; He, Linden, and MacLeod 2007); citizens' positive return to fertilizer use in maize farms report cards; the hiring of contract teachers; in Kenya, whereas too little or too much fertil- or increased oversight of local school commit- izer renders the return unfavorable. Conley tees (Bjorkman and Svensson 2009; and Duflo, and Udry (2007) find that farmers learn how Kremer, and Robinson 2007); and school choice much fertilizer to apply from their successful programs (Angrist, Bettinger, and Kremer 2002, neighbors in a new technology for pineapple 2006; Bettinger, Kremer, and Saavedra 2008). growing in Ghana. Health Gertler (2004) finds that the Programa de Educación, Salud y Alimentación (PROGRESA) 93 Fisherman, Colombia. Photo by Edwin Huffman/World Bank 94 APPENDIX F. REVIEW OF FINANCIAL SECTOR CRITERION A review of the literature indicates that the measures related to non-performing loans financial sector criterion does cover the (NPLs).4 Although NPLs may predict crises to dimensions along which finance is currently some extent, they are typically a lagging indicator, thought to be important: stability, that is, depth with high values suggestive of a problem that has and efficiency; and access. However, not all already crystallized. Hence these NPL measures three finance dimensions are equally important are crude and inadequate even as indicators of for growth and poverty reduction. Further, the current or imminent problems. This is clearly way some of the dimensions are currently being reflected to the fact that NPLs for residential assessed can be strengthened. mortgages did not provide a sensitive early warning system in the recent crisis in advanced Financial Stability economies. There is considerable evidence of a large impact of banking crises on output losses (Hoggarth, The indicator on adherence to Basel Core Princi- Reis, and Saporta 2002)1 as well as on the national ples (BCP) does embody a set of regulatory budget (Laeven and Valenciana 2008; Honohan policies and practices on most points on which 2008b).2 On the other hand, the evidence as to there is broad consensus. which policies work to limit banking crises is ambiguous and controversial (discussed below). However, the econometric evidence between In light of the controversies over what constitute BCP compliance and financial sector perfor- appropriate policies for financial stability, the fact mance is not very strong. that the Country Policy and Institutional Assess- ment (CPIA) assesses intermediate outcomes The indicator on tools and resources for banking in addition to policies and institutions (it is supervision is subject to a lot of controversy. supposed to only assess the latter) is perhaps An extensive data collection and econometric reasonable. exercise (Barth, Caprio, and Levine 2006) fails to uncover any statistically significant relationship There is quite a lot of controversy in the literature between their measures of banking supervision on the current CPIA indicators used for assessing and regulation on the one hand, and financial the stability of the financial system. The indicator sector performance including stability on the on the banking sector's vulnerability to shocks, other. The alternative approach that emphasizes although relevant, is extremely vague. It evokes market discipline (enforced through regulations the concept of stress testing,3 which has not that provide market participants with information been very effective, as clearly demonstrated by and incentives to monitor firms) has not worked the recent wave of systemic failures in mature either as evidenced by the current financial crisis. economies, all of whose prudential regulators were using stress tests. What is clearly missing in the CPIA is explicit attention to the information (for example, The indicator for banking system soundness accounting and disclosure requirements) and specifies two alternative intermediate outcome incentive structures (for example, deposit 95 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT insurance) for market discipline. This is supported The indicator "interest rate spreads" measures by an extensive literature on moral hazard of efficiency of intermediation, which is affected deposit insurance schemes and other crisis by both administrative efficiency as well as the management policies which show that fiscal costs degree of competition in the sector. Policy in of crises are higher in cases where policy design the form of taxation of financial intermediation neglects moral hazard (Honohan and Klingebiel (implicit or explicit) and in the form of adminis- 2003; Demirguc-Kunt, Kane, and Laeven 2006). trative controls (such as holding interest spreads below breakeven rates) can strongly influence Financial Sector Efficiency, Depth, and the spread, so there is no unambiguous link Resource Mobilization between spreads and sustainable growth. There is a convincing literature that financial depth (usually measured as outstanding bank The indicator efficiency of microfinance is vague, credit to the private sector as a share of gross as efficiency presumably relates to the cost of domestic product [GDP] drives economic achieving some goal. Yet the CPIA instructions growth (Honohan 2009). At the same time, there neither specify the goal nor how to judge the is a wide range of policies both in finance and cost. Further, this indicator might fit better under in other sectors that influence the depth and the access dimension. efficiency of finance. In this light, it is perhaps not unreasonable that most of the indicators In addition to these problems with the outcome under this dimension are outcome indicators. indicators, this dimension has also neglected the Yet many of the indicators specified in the CPIA policies that are important for achieving financial are problematic. depth. A recent paper (Tressel and Detragiache 2008) using a large cross-country panel finds The indicator "size and reach of financial markets" that policies that facilitate entry, liberalize pricing is vague, and is already captured by another and the provision of services, and reduce state indicator "private sector credit as a share of GDP." ownership in finance do help deepen finance-- The latter corresponds directly to the measure of although only where complementary deep legal financial depth most commonly used in empiri- institutions protecting private property from cal analyses of the finance-growth link. However, expropriation are present. private sector credit as a share of GDP is not an ideal measure. Rapid increases in financial depth Access to Financial Services are often unsustainable, and yet can be mistaken There are individual success stories of microfi- for finance-supported sustainable growth nance (Littlefield, Morduch, and Hashemi 2003), (Honohan 2004b). Further, the term "reach" but because of the formidable selection biases in normally refers to access, which is covered in the the data, there is little firm evidence of a sizeable third dimension of the CPIA finance criterion. growth or poverty reduction impact of microfi- nance. Indeed, the indications are that overall The indicator "development of capital markets" financial development, as measured by financial is typically measured in terms of overall stock depth, has had a larger and more certain impact market capitalization. But the free-float of shares not just on growth but also on poverty reduction may be a tiny fraction of total capitalization, so than the expansion of microfinance (World Bank some correction needs to be made to address 2008b). this. Also there is little or no empirical evidence that confirms a causal link between market Aside from the lack of strong empirical evidence capitalization and subsequent economic growth. of this dimension on growth and poverty One paper (Levine and Zervos 1998) finds a reduction, the indicators specified under this relationship between stock market turnover and dimension also suffer from a lack of clarity and growth, although this is also not very strongly specificity. The indicator on the payments system supported by the evidence. is unclear. To the extent that it refers to the 96 A P P E N D I X F. R E V I E W O F F I N A N C I A L S E C T O R C R I T E R I O N wholesale payments system, it is not an access Regarding the indicators on access, cross-country issue but rather relates to the cost and security regressions do not suggest any strong influence of a minor part of the banking system (Honohan of household financial access on growth or on 2009). At the same time, however, there is no poverty (taking into account the average level of empirical evidence that a wholesale payments income) (Honohan 2008a). However, access of system failure could trigger a banking collapse. small and medium enterprises (SMEs) to formal To the extent that the payments system refers to sector financial services has been found to be the retail system, it relates to access but it is not important for firm entry, firm growth, innovation, clear what is being assessed in the CPIA. and size distribution of firms (Beck, Demirguc- Kunt, and Levine 2005; Ayyagari, Demirguc-Kunt, The indicator on the credit reporting system is and Maksimovic 2007). relevant, as credit registries are known to have the potential to expand access to finance, particularly The indicator regarding the legal and regula- to small and medium enterprises (SMEs) (Miller tory framework supporting access to finance is 2003; Berger, Frame, and Miller 2005; Djankov, vague. Also, the cross-country empirical litera- McLiesh, and Shleifer 2007). However, the CPIA ture suggests that although legal and political falls short in specifying how this is to be assessed, institutions that protect private property against although the literature has established a standard the state are important for financial depth, the set of good practices. Such good practices include protection of contracts between private agents facilitating entry of private providers of registries is important for determining access (Beck, where possible, including as wide a set of Demirguc-Kunt, and Levine 2005; World Bank information providers as possible, and ensuring 2008b). privacy laws do not unduly constrain reporting of "positive information" (Miller 2003). There is also Recommendations evidence that quality matters in ensuring that a The above review suggests that the CPIA indica- credit registry does expand availability of finance tors for all three dimensions could be strength- (Brown, Jappelli, and Pagano 2009, and Lutoto, ened. Suggestions for such strengthening are McIntosh, and Wydick 2007). presented in appendix box F. 1. 97 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Appendix Box F.1: Suggestions for Strengthening of the Country Policy and Institutional Assessment (CPIA) Indicators for the Financial Sector Each of the three dimensions currently covered by the CPIA and quasi-taxes, inadequate legal protection, exogenous risk financial sector criterion can be revised and strengthened, as and market power), and bank operating costs (for example as follows: a share of total assets). Financial Stability Policy barriers to efficiency and depth. Among the relevant indi- Whether policy creates good incentives for prudential man- cators are: distorting financial sector taxes and quasi-taxes on agement of financial firms. This would include enforcement of intermediation including binding interest rate ceilings; and directed prudential regulations and the safety net policy (including the credit programs. The preconditions for the BCP (which are not nature and extent of deposit insurance). graded) also provide a useful summary of key policy dimensions How good are supervisory powers and effectiveness (including as relevant to depth and efficiency as to stability. the tools and resources for risk assessment)? This would entail assessment of the compliance with Basel Core Principles (BCP) Access (already included in the CPIA), which is a useful summary for Policy. Has policy created an enabling environment for expand- banking. However, non-bank finance also needs to be considered ing outreach of the financial system? For example, is the legal where it is sizeable. framework for debt recovery and insolvency economically and The vulnerability of financial institutions to shocks. This can be administratively efficient (including time taken and cost of in- informed by measures of bank capitalization (for example, whether solvency and debt recovery)? Is there a good credit reporting regulatory minima sufficient for the economy's risk, well measured system? Are there legal or regulatory barriers to creating one? and enforced; whether all banks satisfy regulatory minima; how is Is there an adequate legal basis for the leasing industry? Is capital measured). Non-performing loans (NPLs) (especially net the regulatory framework for microfinance supportive? Is anti- of provision) should be taken into account but supplemented by money laundering and combating the financing of terrorism exposure to foreign exchange risk. Rapid growth of credit should (AML-CFT) policy well adapted to protecting financial service be monitored as a possible warning sign. providers focusing on the poor from undue costs (for example, are microfinance firms able to operate depository and money Financial Depth and Efficiency transmission services without crippling regulatory burden?) Depth. The key measure is the size of private sector credit (Isern and others 2005). as a share of gross domestic product (GDP), adjusted for the country's overall level of development. (However, rapid growth Outcomes. Measures could include: the percentage of the popu- in this indicator should be assessed as negative). Other mea- lation with access to formal financial services; how good the sures are market capitalization (total and free-float) and stock access of small and medium enterprises (SMEs) to financial market turnover. services is (such as from responses to the Investment Climate Assessment surveys); how financially secure the microfinance Efficiency of intermediation. The measures are intermediation industry is; and whether retail money transmission costs (inter- spreads (ideally distinguishing what is attributable to taxes nal and international) are low. Source: Honohan 2009. 98 APPENDIX G: COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT (CPIA) AND LOAN PERFORMANCE This appendix presents the econometric approved and closed between 2004 and 2007. analysis undertaken to estimate the relationship This, in turn, would have reduced the sample between the CPIA and Bank loan performance. size substantially as well as biased the sample The methodology draws on a similar exercise because those loans would have consisted mostly by Dollar and Levin (2005) who find a strong of development policy loans. Estimates based association between institutional quality and loan on data for loans approved since 1998 found a success rate using the property rights/rule of law relatively good correlation between the share measure from the International Country Risk of problem loans and IEG loan outcome ratings Guide and democracy measure from Freedom (the correlation coefficient is 0.63). House as proxies for institutional quality, and (Independent Evaluation Group) IEG's loan The flags for problem loans are based on outcome ratings as a measure of loan perfor- self-evaluation by Bank staff of loans where mance. This analysis uses a very similar specifica- implementation progress or development tion to that of Dollar and Levin, but a different objective is rated unsatisfactory. For this analysis, measure of loan performance (see below) and a loan is identified as a problem if it has been the CPIA for the institutional quality variables. flagged as a problem loan anytime between 2004 and 2007. The sample is restricted to all countries Data that had at least three loans approved during Policy and institutions the period. For each country the share of loans The average overall CPIA rating for the country (approved between 2004 and 2007) that had from 2004­07 was used as the explanatory been flagged as problem loans was used as the variable. The average ratings for each of the four dependent variable.1 CPIA clusters were also used alternatively as explanatory variables. The time period 2004­07 Control variables was selected because of the major restructuring The following control variables are used in the of the CPIA in 2004, and in particular because of econometric analysis:2 the change in the definition of the rating scale, which resulted in a discontinuity in the data that · Average aid as a share of gross domestic prod- year. uct (following Dollar and Levin 2005) which could influence loan performance in opposite Loan performance ways. On the one hand, there could be in- This analysis uses the Implementation Status and creasing returns in the sense that a lot of aid Results Report (ISR) flags for problem loans as may create a better environment of supporting the proxy for loan performance because of the services and resources. On the other hand, need to match the loan performance data with there could be diminishing returns or absorp- the CPIA data which confines the analysis to this tive capacity constraints. The data are from period of 2004­07. Given this restriction, it is not the Organisation for Economic Co-operation possible to use IEG loan outcome ratings because and Development (OECD) Development As- the analysis would have been restricted to loans sistance Committee (DAC) database. 99 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT · Geographic constraints to development as signs are not consistent across specifications--a represented by the share of a country's terri- manifestation of multicollinearity. tory in the tropics that has been found to af- fect economic performance (Dollar and Levin To reduce the multicollinearity problem, specifi- 2005, following Gallup, Sachs, and Mellinger cations were estimated using ratings from all the 1999). possible combinations of two CPIA clusters as · Extent of ethnic fractionalization that has been explanatory variables, in turn. The coefficients found to affect political stability and policies on all the combinations of the two CPIA variables and institutions (see, for example, Mauro are not significant but their signs are consistently 1995; Easterly and Levine 1997; Alesina and negative across specifications. An F-test of the others 2003). joint null hypothesis that the coefficients of both · Regional dummies to control for region-spe- CPIA variables are equal to zero was rejected, cific effects on loan performance. which implies that one or both CPIA ratings have significant negative association with the share Econometric Specification and Results of problem projects. However, their individual Ordinary least squares regressions3 are used for contributions cannot be discerned because of the estimations. The results are presented in their high degree of correlation with each other. appendix table G.1. There is a significant negative Further, an F-test of the null hypothesis that the association between the overall CPIA rating of a coefficient of the CPIA rating for one cluster is country and the percentage of problem loans in not different from the coefficient of the rating of that country. The results are similar when each of the other CPIA cluster cannot be rejected. Similar the four CPIA cluster ratings are used separately results were obtained when the share of problem instead of using the overall CPIA rating. There is projects in a country was regressed on the average some evidence that a higher level of aid as a share ratings of the first three clusters combined and the of GDP is associated with better portfolio perfor- ratings for cluster D. In sum, there is not sufficient mance, and countries with a higher proportion evidence to conclude that the policies and institu- of land in tropics are more likely to have a higher tions measured by one cluster are relatively more proportion of problem loans. However, the important than those measured by the other respective coefficients are not always significant clusters for better project performance. across the different specifications. Conclusions Attempts were made to estimate the relative The analysis finds that higher CPIA ratings importance of the four CPIA clusters on the Bank's are associated with better Bank loan portfo- loan portfolio performance. This was difficult lio performance. This association is found for because of the high degree of correlation among overall CPIA ratings as well as ratings for each the ratings of the four clusters (table G.2). Indeed of the four CPIA clusters. There is insufficient when all four clusters were included as explana- evidence to conclude that any one of the four tory variables along with other control variables, CPIA clusters is more important than the others none of them are found to be significant and their for loan performance. 100 APPENDIX G: COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT (CPIA) AND LOAN PERFORMANCE Table G.1: CPIA and Loan Performance: Ordinary Least Squares Regressions Dependent Variable: Percent of Problem Loans Overall CPIA CPIA Cluster A CPIA Cluster B CPIA Cluster C CPIA Cluster D Independent Variables (1) (2) (3) (1) (2) (3) (1) (2) (3) (1) (2) (3) (1) (2) (3) CPIA ratings ­13.4*** ­16.1* ** ­17.9** ­9.1*** ­9.0** ­10.1*** ­11.4*** ­14.4*** ­16.0*** ­9.7** ­13.7** ­14.4*** ­10.5** ­11.4** ­13.4*** Aid/GDP ­0.5* ­0.4* ­0.4 ­0.3 ­0.5 ­0.4 ­0.4 ­0.3 ­0.4 ­0.3 % Tropical Land 10.5 10.6 12.5 13.6* 10.3 Ethnic 1.7 6.0 2.5 ­0.7 2.8 Fractionalization Africa Dummy ­18.8*** ­18.7** ­16.7*** ­16.5*** ­17.7** ­14.0** ­18.8*** ­19.5** ­17.3*** ­17.7*** ­18.9** ­16.9*** ­16.6*** ­16.3** ­14.4** Europe and Central ­20.6*** ­15.5* ­21.9*** ­19.8*** ­13.5 ­19.8*** ­21.1*** ­14.3 ­22.4*** ­20.5*** ­13.3 ­21.9*** ­22.2*** ­16.9* ­23.6*** Asia Dummy East Asia and Pacific ­25.2*** ­25.7*** ­23.6*** ­21.6*** ­21.6** ­19.5** ­26.5*** ­27.4*** ­26.1*** ­23.7*** ­25.1*** ­23.0*** ­24.2*** ­24.7*** ­22.8*** Dummy Middle East and North ­26.2*** ­16.3 ­28.0*** ­25.0*** ­14.4 ­25.5*** ­25.7*** ­15.2 ­27.6*** ­25.2*** ­13.6 ­27.2*** ­24.2*** ­13.4 ­25.1*** Africa Dummy South Asia Dummy ­22.8*** ­17.2* ­22.5*** ­20.0** ­13.9 ­18.8** ­23.9*** ­16.4 ­24.5*** ­21.5** ­15.8 ­21.7** ­21.1** ­15.2 ­20.4** Constant 85.5*** 90.9*** 104.1*** 71.1 *** 63.1*** 75.9*** 78.9*** 83.6** 98.5*** 71.0*** 79.3*** 89.9*** 71.0*** 68.6*** 82.5*** Observations 92 79 87 92 79 87 92 79 87 92 79 87 92 79 87 R-squared 0.23 0.29 0.26 0.22 0.26 0.22 0.21 0.28 0.24 0.19 0.26 0.22 0.21 0.25 0.23 Source: IEG estimates. Note: *** significant at 1% level; ** significant at 5% level; * significant at 10% level. 101 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Table G.2: Correlations among Average 2004­07 CPIA Ratings for the Four Country Policy and In- stitutional Assessment (CPIA) Clusters CPIA Cluster A Cluster B Cluster C Cluster D Overall CPIA Cluster A 1.00 Cluster B 0.61 1.00 Cluster C 0.66 0.74 1.00 Cluster D 0.74 0.81 0.84 1.00 Overall CPIA 0.86 0.87 0.90 0.94 1.00 Source: IEG estimates. Notes: a. Based on 92 observations. b. All correlations are significant at 1% level 102 APPENDIX H: COMPARING COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT RATINGS BY THE WORLD BANK, AFRICAN DEVELOPMENT BANK, AND ASIAN DEVELOPMENT BANK Comparison of the CPIA ratings (rather than ratings by the AfDB were higher than those by relative rankings implied by the ratings) can the Bank for all 16 CPIA criteria; for 12 of the 16, only be done for the CPIA ratings by the World the ratings were statistically significantly higher Bank, the AfDB, and the ADB. Ratings by these at the 5 percent level. The ratings by the ADB three development banks are on the exact same were higher than those of the Bank for 14 of scale, with each rating having the exact same the 16 CPIA criteria; for 5 of the 14, the ratings meaning, which is not the case with the other were statistically significantly higher. For the indicators. two criteria for which the Bank's ratings were higher than those of the ADB, the differences in Generally speaking, it is found that the 2007 CPIA the ratings were not significantly different (see ratings by the AfDB and ADB are higher than appendix table H.1). the CPIA ratings by the Bank. Specifically, the 103 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Appendix H.1: 2007 CPIA Ratings by the World Bank, AfDB and ADB Average for common World Bank Average for common World Bank and AFDB member countries and ADB member countries Criterion World Bank AfDB World Bank ADB q1 3.64 3.99 3.83 4.00 q2 3.38 3.74 3.44 3.67 q3 3.31 3.67 3.90 4.00 q4 3.58 3.62 3.87 3.65 q5 3.20 3.48 3.17 3.42 q6 3.25 3.38 3.35 3.27 q7 3.24 3.55 3.56 3.62 q8 3.24 3.53 3.58 3.63 q9 3.31 3.45 3.48 3.58 q10 3.01 3.22 3.23 3.29 q11 3.24 3.45 3.13 3.25 q12 2.90 3.23 3.00 3.21 q13 3.16 3.52 3.29 3.60 q14 3.46 3.55 3.35 3.58 q15 3.02 3.27 3.10 3.13 q16 2.85 3.24 2.87 3.06 Sources: World Bank, AfDB, and ADB. Note: Bold figures denote ratings that are significantly different at the 5 percent level between the World Bank and the AfDB, and the World Bank and the ADB. ADB= Asian Development Bank; AfDB= African Development Bank. 104 APPENDIX I: COMPARATOR INDICES OF THE CPIA 105 106 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT CPIA CRITERIA Policies and Institutions Property Quality of Transparency; Business Equity of Social for Environ- Rights and Budgetary Efficiency Quality Accountability, Regulatory Gender Public Building Protection mental Rule Based and Financial of Revenue of Public and Corruption Comparator Financial Environment Equwality Resource Use Human and Labor Sustainability Governance Management Mobilization Administration in the Public Indices Trade (q4) Sector (q5) (q6) (q7) (q8) Resources (q9) (q10) (q11) (q12) (q13) (q14) (q15) Sector (q16) Worldwide Regulatory Rule of Law Government Control of Governance quality effectiveness corruption Indicator Doing Business (5) Getting Doing Business credit (Strength Indicators: of legal rights (1) Starting a index, depth Business of credit (2) Dealing with information construction index). permits (3) Employing workers (6) Protecting investors (7) Paying taxes (10) Closing a business Index of Business Property Rights Freedom from Economic freedom (data Corruption (data Freedom: from Doing from Transparency Heritage Business) International); Foundation property rights International Investment Bureaucracy Corruption Country Risk profile (contract Quality Guide (ICRG): viability; profit PRS Group repatriation; payment delays); law and order Global Gender Gender Gap Gap Report Index (World Economic Forum) Transparency Corruption International Perception Index CPIA CRITERIA Policies and Institutions Property Quality of Transparency; Business Equity of Social for Environ- Rights and Budgetary Efficiency Quality Accountability, Regulatory Gender Public Building Protection mental Rule Based and Financial of Revenue of Public and Corruption Comparator Financial Environment Equwality Resource Use Human and Labor Sustainability Governance Management Mobilization Administration in the Public Indices Trade (q4) Sector (q5) (q6) (q7) (q8) Resources (q9) (q10) (q11) (q12) (q13) (q14) (q15) Sector (q16) Gender Gender Empowerment Empowerment Measure Measure United Nations Development Programme (UNDP) Human Development Report Ibrahim Index Rule of law, of African transparency, and Governance corruption Bertelsmann Private property Social safety Private property Q3.2 Does an Transformation nets independent Index judiciary exist? Q3.3 Are there legal or political penalties for A P P E N D I X I : C O M PA R AT O R I N D I C E S O F T H E C P I A officeholders who abuse their position? Global Enabling Enabling Trade Enabling Trade Trade Report Index: Index: (4) (World (1) Market the business Economic access--tariffs environment-- Forum) and non-tariff regulatory barriers, environment (2) Border administration 2005 Social and Environmental institutional Sustainability capacity Index Global Prevalence of Interest rate Burden of Business impact Property rights Competitive- trade barriers spread (hard government of malaria ness Index data) regulation Trade-weighted Financial market Strength of Malaria Efficiency of tariff rate (hard sophistication auditing and incidence (hard legal framework data) reporting data) standards Burden of Financing Efficacy of Business impact Business costs customs through local corporate boards of tuberculosis of crime and 107 procedures equity market violence (continued on next page) 108 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT CPIA CRITERIA Policies and Institutions Property Quality of Transparency; Business Equity of Social for Environ- Rights and Budgetary Efficiency Quality Accountability, Regulatory Gender Public Building Protection mental Rule Based and Financial of Revenue of Public and Corruption Comparator Financial Environment Equwality Resource Use Human and Labor Sustainability Governance Management Mobilization Administration in the Public Indices Trade (q4) Sector (q5) (q6) (q7) (q8) Resources (q9) (q10) (q11) (q12) (q13) (q14) (q15) Sector (q16) Global Ease of access Protection Tuberculosis Reliability of Competitiveness to loans of minority incidence (hard police services Index (cont.) shareholders' data) interests Soundness of Intensity of local Business impact banks competition of HIV/AIDS Regulation Extent of market Human of securities dominance immunodeficiency exchanges virus (HIV) prevalence (hard data) Legal rights Effectiveness of Infant mortality index (hard anti-monopoly (hard data) data) policy Extent and effect Life expectancy of taxation (hard data) Total tax rate Quality of primary (hard data) education Number of Primary procedures enrollment (hard required to start data) a business (hard data) Time required to Education start a business expenditure (hard (hard data) data) Strength Secondary of investor enrollment (hard protection (hard data) data) Cooperation in Tertiary labor-employer enrollment (hard relations data) Flexibility Quality of the of wage educational determination system Hiring and firing practices Source: IEG. APPENDIX J. NUMBER OF IDA AND IBRD COUNTRIES FOR WHICH EXTERNAL DATA ARE AVAILABLE 109 110 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Bertelsmann Index of Corruption Ibrahim Index Environmental Gender Transformation Economic Perception of African Sustainability Gender Gap Empowerment CPIA ICRG GCI ETI DB Index Freedom WGI Index Governance Index Index Measure q1 IDA IBRD q2 IDA IBRD q3 IDA IBRD q4 IDA 39 37 IBRD 52 48 q5 IDA 39 75 IBRD 52 64 q6 IDA 38 37 75 56 55 75 IBRD 52 48 64 55 59 65 q7 IDA 40 17 IBRD 53 43 q8 IDA IBRD q9 IDA 39 IBRD 52 q10 IDA 56 IBRD 55 q11 IDA 57 IBRD 53 q12 IDA 42 40 56 55 75 IBRD 53 52 55 59 65 q13 IDA IBRD q14 IDA IBRD q15 IDA 42 75 IBRD 53 65 q16 IDA 42 56 55 75 74 38 IBRD 53 55 59 65 61 8 Source: IEG. Note: DB= Doing business; ETI=Enabling Trade Index; GCI= Global Competitiveness Index; IBRD= International Bank for Reconstruction and Development; ICRG= International Country Risk Guide; IDA= International Development Association; WGI= World Governance Indicators. APPENDIX K. COMMENTS BY NETWORK ON REGIONAL CPIA RATING PROPOSALS, 2007 IBRD IDA Total Share of Share of Share of countries countries countries Number of with network Number of with network Number of with network network comments network comments network comments CPIA criterion comments (%) comments (%) comments (%) q1 59 90.8 62 82.7 121 86.4 q2 59 90.8 62 82.7 121 86.4 q3 59 90.8 62 82.7 121 86.4 q4 43 66.2 37 49.3 80 57.1 q5 10 15.4 8 10.7 18 12.9 q6 7 10.8 7 9.3 14 10.0 q7 16 24.6 25 33.3 41 29.3 q8 12 18.5 10 13.3 22 15.7 q9 50 76.9 48 64.0 98 70.0 q10 34 52.3 38 50.7 72 51.4 q11 4 6.2 7 9.3 11 7.9 q12 13 20.0 11 14.7 24 17.1 q13 10 15.4 19 25.3 29 20.7 q14 14 21.5 13 17.3 27 19.3 q15 7 10.8 6 8.0 13 9.3 q16 10 15.4 14 18.7 24 17.1 Total 407 39.1 429 35.8 836 37.3 Source: IEG calculations based on World Bank information. Note: IBRD= International Bank for Reconstruction and Development;; IDA= International Development Association. 111 Elephants, Kenya. Photo by Curt Carnemark/World Bank 112 ENDNOTES Chapter 1 Chapter 2 1. This is an evaluation of the implementation of 1. These are some of the key elements of the the IDA10­12 Replenishment Agreements. so-called Washington Consensus view. 2. The previous name of the Independent Evalua- 2. The point of view of Rodrik and others seems to tion Group (IEG) name was the Operations Evaluation be accepted by the majority of the economists today Department (OED). The name changed from OED to (even if this consensus is obviously quite recent). IEG in December 2005. Relevant documents pertain- Aghion and Howitt (2009) reconcile new growth theory ing to the CPIA and other evaluations from the earlier with what they call "Gerschenkron's views," thereby period may still be catalogued under OED in some addressing development economists' concern that databases. growth theory can only deliver universal, one-size- 3. The external panel consisted of nine academics fits-all policy prescriptions (legal reform to enforce and public officials from developed and developing property rights, investment climate favorable to countries. The panel met at the Bank on February entrepreneurship, education, macrostability, and so 17­18, 2004, and reviewed the coverage of the CPIA on) to maximize the growth prospects of a country system, methodology, database, and cross-country or sector, and does not apprehend structural transfor- comparability. The panel submitted its final report to mations in the process of convergence. (New growth Management on April 2, 2004. See World Bank 2004a. theory calls for better property rights protection and 4. The cluster was renamed from "public finance/ higher education investment in all countries under all civil administration" in 1997. latitudes. Gerschenkron's view is the idea that relatively 5. The criteria on financial stability and financial backward economies could more rapidly catch up with sector depth, efficiency and resource mobilization more advanced economies by introducing appropriate were collapsed into a new financial sector criterion, institutions that are growth enhancing at an early stage and the competitive environment for the private of development but may cease to be so at a later stage). sector and goods and factor markets criteria were More specifically, they analyze some general implica- collapsed into the new business regulatory environ- tions of the notion of "distance-dependent" appropri- ment criterion. ate institutions, by which they mean institutions that are 6. One of these is the criterion on management growth enhancing only for countries at a certain stage and sustainability of the development program, of technological development. In particular, they show which is covered in almost all of the other criteria. The how the failure to adapt institutions to technological other is the criterion on monitoring and analysis of development may generate non-convergence traps poverty outcomes and impacts. This criterion covers whereby a country's average productivity (or per capita the availability of up-to-date household surveys and GDP) remains bounded away from frontier levels. analysis, which is necessary for the criterion on equity 3. The theoretical and empirical analyses of of public resource use, and the criterion on building Vandenbussche, Aghion, and Meghir (2006) suggest human resources. (World Bank 2004d). that countries with productivities far from the techno- 7. There were some changes in the instructions logical frontier should put more emphasis on primary/ for some criteria mainly aimed at reducing overlap secondary education, whereas countries closer to between criteria, and improving consistency. the frontier should put more emphasis on tertiary 8. World Bank (2008e, annex 1, p. 2). education. 113 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT 4. See Aghion and Howitt (2009) for a discussion. from The World Bank's internal database. Because the For the neoclassical and endogenous growth theories, simulations focused on changes in allocations, they it is investment in physical and human capital that did not require data on the overall IDA envelope. drives growth, whereas for the product variety and 13. In fiscal 2009, the post-conflict countries are Schumpeterian theories, it is investment in technology Afghanistan, Angola, Burundi, Democratic Republic (in the form of research). In the hybrid model, invest- of Congo, Republic of Congo, Côte d'Ivoire, Eritrea, ment in capital and technology are both important. Liberia, and Timor-Leste. The re-engaging countries 5. The discussion in this paragraph is drawn from are Central African Republic, Haiti, and Togo. The World Bank (2008a). capped countries are India and Pakistan. 6. Bourguignon (2004) indicated that a serious 14. References are cited in Kanbur (2004). evaluation of Mexico's PROGRESA/Oportunidades and 15. A 1 percent rate of growth in average household Brazil's Bolsa Escola/Bolsa Families, essentially means- income or consumption brought anything from a tested income transfer programs with conditionalities, modest drop in the poverty rate of 0.6 percent to a finds that these programs were effective in raising more dramatic 3.5 percent annual decline. (Ravallion school enrollment rates and health outcomes in 2001). targeted populations. The sources cited for this were 16. This is a term coined by Ghura, Leite, and Skoufias (2001) on PROGRESA, and Bourguignon, Tsangarides (2002). Ferreira, and Leite (2003) on Bolsa Escola; and World 17. Besley and Burgess (2003) underline that Bank (2004g). expanding access to credit for the poor may increase 7. This has especially been the case with East the elasticity between economic growth and poverty Asian countries. See World Bank (1993). reduction, in addition to being a form of redistribution. 8. An early study of Grameen Bank finds that it 18. Honahan (2004b), based on analysis of a cross- generated employment and income for the poor, section of some 70-odd developing countries. especially women (Hossain 1988). The most compre- 19. According to Winters, McCulloch, and McKay hensive impact studies of microfinance, a joint research (2004), these channels are: (i) job opportunities and project of the Bangladesh Institute of Development wages of the poor; (ii) prices that poor consumers Studies and the World Bank find strong evidence that pay for the goods they buy; (iii) government revenues the programs help the poor through asset building and in turn social expenditures; (iv) income instabil- (and consumption smoothing) (Khandker 1998; Pitt ity and workers' chances of becoming poor. Goldberg and Khandker 2003). and Pavcnik (2004) identify three other channels: 9. A very recent paper (Freund and Rocha 2010) (i) participation of household members in the estimates that for African exports, a one-day reduction in labor market; (ii) household consumption; and (iii) inland travel times translates into nearly a 3 percentage household production. point reduction in all importing country tariffs. 20. Porto (2005) shows that reducing such barriers 10. This is in accordance with the so-called Lerner's would reduce poverty from an initial headcount ratio symmetry, whereby taxing imports has the same effect of 48.3 percent to a poverty rate from 43.3 to 45.5 on international trade as taxing exports. percent. In other words, it would lift between 100,000 11. The phenomenon is called the "environmental to 180,000 Moldovan citizens (out of a population of Kuznets curve" whereby pollution levels, initially fairly 3.5 million) out of poverty. low, rise as national income increases up to a certain 21. Notably, this has followed Amartya Sen's work point, then begin to decline with further economic on poverty and freedom (Sen 1997, 1999). growth as cleaner technologies are adopted and 22. It is recognized that economic development environmental sanitation infrastructure investments encompasses more dimensions than just growth, begin to catch up with expanding local needs (IEG although growth is certainly the key dimension, and 2008, chapter 1, footnote 3). moreover the only dimension for which a theoreti- 12. The allocations were calculated using the PBA cal foundation exists. Poverty reduction is another formula. The GNI per capita and population data are dimension of economic development, and it is also from World Development Indicators, and the portfo- recognized that although growth is not sufficient lio performance ratings are constructed using data for poverty reduction, it is necessary. Therefore, the 114 ENDNOTES discussion on the theoretical and empirical literature 30. Such problems include difficulties with: (i) of the impact of aid on growth also pertains to the accounting for endogeneity (in particular, which impact of aid on reduction of income poverty. instruments to use); (ii) the specification of the regres- 23. These are the Harrod-Domar model and its sion equation (including the timing of the impact of extension the Solow growth model. aid on growth); and (iii) the treatment of outliers; data 24. Boone (1994). According to Clemens, Radelet, limitations, among others. and Bhavnani (2004), parts of the analysis of Boone 31. Headey (2005) and Reddy and Minoiu (2006) (1994) were published in Boone (1996) without the find that multilateral or developmental aid has a growth regressions. positive impact on growth, whereas geopolitical and 25. The exception was small countries, where aid bilateral aid do not (except for bilateral aid that is not flows which made up a large share of GDP were found politically motivated). Ram (2003) finds that bilateral to lead to higher investments, which Boone attributed but not multilateral aid has a large and positive impact to the lack of fungibility of aid flows. Boone provided on growth (although this paper has been criticized for an example that in a small country one dam or a large not taking the endogeneity of aid into account). public infrastructure project can represent a sizeable 32. Clemens, Radelet, and Bhavnani (2004) also , portion of GNP and the project is unlikely to be fungible. find that the impact on growth is somewhat larger 26. Other innovations in this later generation of in countries with stronger institutions or longer life empirical literature include: larger datasets that cover expectancies. more countries and more years; accounting for the 33. The concept of such evaluations originates endogeneity of aid (that is, in addition to growth from the scientific/medical realm of randomized trials, being an outcome of aid, aid can also be influenced by whereby the effects of medicines are tested by compar- the growth performances of countries); and allowing ing the group of patients who received the medicine for a non-linear aid-growth relationship (in particular with a group who received the placebo. The selection diminishing returns to aid, or in other words, decreas- of the "comparable" group is therefore key and, in the ing impact of aid on growth after reaching a point of context of aid project evaluations, needs to ensure that maximum impact). the differential effects between the two groups could 27. In Burnside and Dollar (1997, 2000), fiscal not be due to some characteristic of the recipient policy is measured by the budget surplus (following group that is not present in the control group. Easterly and Rebelo 1993), monetary policy by the 34. A request made by the IEG team to the network inflation rate (following Fischer 1993) and trade by on April 23, 2009, for the number of responses to the trade openness (as measured by Sachs and Warner environment questionnaire for 2008 was not answered. 1995). 35. This is constructed by applying the relevant 28. Including the relative size of the donor versus weights to the ratings for market access and border the recipient (the larger the ratio, the greater the administration, respectively, of the Enabling Trade influence, and presumably the larger amounts of aid), Index. commonality of language, and colonial ties (either 36. Although the rank correlation coefficient with current or past). one based on the Global Competitiveness Index (GCI) 29. Roodman (2007a). He performed the tests on will remain unchanged at 0.70. The one based on the Burnside and Dollar (2000), Collier and Dehn (2001), GCI is constructed by applying the relevant weights to Collier and Dollar (2002), Collier and Hoeffler (2004), the combined rating for tariffs and trade barriers (50 Hansen and Tarp (2001), Dalgaard, Hansen, and Tarp percent each) and the rating for burden of customs (2004), and Guillaumont and Chauvet (2001). The administration. tests were designed to minimize arbitrariness and 37. As in the simulation on dropping q8, this were derived mainly from the differences among the simulation is conducted based on a CPIA implied by papers themselves. The tests included: (i) changing the IDA allocation formula--that is, 25 percent weight the control set (using the control sets of other papers given to CPIA clusters A through C, and 75 percent being tested); (ii) redefining aid; (iii) redefining good weight on cluster D. policy; (iv) changing periodization; (v) changing 38. There is room for debate as to the causality outliers; and (vi) expanding the sample. between every banking crisis and output losses, as 115 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT there have certainly been instances in the past where global integrity indicators produced by Global Integrity fiscal or political crises have triggered banking crises, that rate the budget process, and the Open Budget such as Argentina in 1981, in the transition countries, Index produced by the Open Budget Initiative that rates and in several African countries. the transparency of the budget. However, both these 39. At the same time, however, what evidence indicators comprise only part of what is being assessed there is does not suggest that crises systematically under q13, so are not strictly comparable with q13, and worsen the Gini coefficient (Honohan 2004a), that is, hence the team did not include them as comparators. that banking crises have a disproportionate effect on 4. Rank correlations are used instead of pair-wise the poor, although the poor are likely less able than correlations for comparing the different indicators others to absorb adverse shocks. because of the less restrictive assumption underlying 40. One is the stock of NPLs as a share of total rank correlations (they do not require the assump- loans; the other, the "level of capital at risk," is a net tion of a linear relationship between the indica- figure which subtracts provisions already taken against tors). Nonetheless, the IEG team for this evaluation these NPLs. performed both types of correlations and obtained very similar results from them. Chapter 3 5. A better correlation denotes a rank correlation 1. For the criteria on gender (q7), property rights coefficient that is at least 5 percent higher. and rule-based governance (q12), and quality of 6. Ratings from ADB are excluded from this budgetary and financial management (q13), the comparison because only 3 IBRD countries are rated 2007 AfDB questionnaire is identical to the pre-2005 by ADB, whereas 12 are rated by AfDB. Bank CPIA questionnaire that has since been updated, although the differences are not substantial. Further, Appendix F the Bank's ratings for the criterion on policies and 1. There is room for debate as to the causality institutions for environmental sustainability (q11) between every banking crisis and output losses, as are relatively stringent compared with those of the there have certainly been instances in the past where AfDB. For example, the Bank gives a "1" rating on q11 fiscal or political crises have triggered banking crises, if "For both pollution and natural resource issues: such as Argentina in 1981, in the transition countries, regulations and policies are lacking. Environmental and in several African countries. information is not published. Environmental Assess- 2. At the same time, however, what evidence there ment legislation is lacking. No data are available for is does not suggest that crises systematically worsen priority setting. Sector ministries do not incorporate the Gini coefficient (Honohan 2004a), that is, that environmental concerns." A "1" rating is given by AfDB banking crises have a disproportionate effect on the if "For two years or more, government policies have a poor, although the poor are likely less able than others negative effect on environment (for example agricul- to absorb adverse shocks. ture policies that stimulate expansion into marginal 3. Stress tests model the impact of extreme but land or tropical forest; subsidized prices on the exploi- plausible shocks, and measure the capacity of banks to tation of scarce and/or non-renewable resources). absorb the shocks with available liquidity and capital. Government has no environmental action plans or 4. One is the stock of NPLs as a share of total loans; similar national framework, and no institutions to the other, the "level of capital at risk," is a net figure sustainably manage the environment and support the which subtracts provisions already taken against these various dimensions of sustainable development." NPLs. 2. It is difficult to find good comparators for the economic management criteria because both Appendix G outcomes and policies are taken into account in the 1. For loans approved from 1998 onward, the CPIA ratings, whereas other indicators focus only on correlation between the share of problem projects in a outcomes (such as the economic risk index produced country and the share of loans receiving an unsatisfac- by the International Country Risk Guide). tory IEG outcome rating for that country is 0.63. 3. The IEG team examined two indicators that are 2. The level of per capita GDP was not included possible comparators for q13. These are one of the as a control variable due to its high correlation with 116 ENDNOTES the overall CPIA rating. The percent of problem loans and Aid to GDP variable were instrumented using the might also depend on the quality of the Bank staff share of population speaking English, the share of working on those loans; ratings on task team quality population speaking a continental European language, are provided by the Quality Assurance Group (QAG) distance from equator, level of population, and each of in its quality at entry ratings for loans. However, due the above four instruments multiplied by population to limited data availability, the quality of Bank staff was (see Dollar and Levin 2005). The first-stage regressions not included as a control variable. suggest that the instruments are weak (the F-statis- 3. To account for the possible endogeneity of both tics are relatively low). The Durbin-Wu-Hausman the CPIA ratings and the Aid to GDP variable, two-stage test for endogeneity suggests that both variables are least square specifications were estimated. The CPIA exogenous and the OLS estimates are consistent. 117 Women preparing food, Nigeria. Photo by Curt Carnemark/World Bank 118 REFERENCES Acemoglu, Daron, and Simon Johnson. 2007. Development on Convergence: Theory and "Disease and Development: The Effect of Life Evidence." Quarterly Journal of Economics Expectancy on Economic Growth." Journal 120 (1): 173­222. of Political Economy 115 (61): 925­985. Ahmad, Ehtisham, and Nicholas Stern. 1987. Acemoglu, Daron, and James A. Robinson. 2000. "Alternative Sources of Government Revenue: "Political Losers as a Barrier to Economic Illustrations from India, 1979­80." In The Development." American Economic Review Theory of Taxation for Developing Countries. Papers and Proceedings 90: 126­44. David M. G. Newbery, and Nicholas H. Stern Acemoglu, Daron, and Thierry A. Verdier. 2000. (editors), New York: Published for The World "The Choice Between Market Failures and Bank by Oxford University Press. Corruption." American Economic Review 90 Ahrend, Rudiger. 2001. "Press Freedom, Human (1): 194­211. Capital and Corruption." Mimeo. ------. 2001. "The Colonial Origins of Compar- Alesina, Alberto. 1997. "Politics, Procedures and ative Development: An Empirical Investi- Budget Deficits." Background Paper prepared gation." American Economic Review 91 for The World Development Report 1997. (5): 1369­401. World Bank, Washington, DC. Acemoglu, Daron, Simon Johnson, and James A. Alesina, Alberto, and George-Marios Angeletos. Robinson. 2005. "The Rise of Europe: Atlantic 2005. "Corruption, Inequality and Fairness." Trade, Institutional Change, and Economic National Bureau of Economic Research Growth." American Economic Review 95 (3): (NBER) Working Paper No. 11399. 546­79. Alesina, Alberto, and Roberto Perotti. 1996. Adsera, Alicia, Carles Boix, and Mark Payne. 2003. "Income Distribution, Political Instability, and "Are You Being Served? Political Accountabil- Investment." European Economic Review 40 ity and Quality of Government." Journal of (6): 1203­28. Law, Economics, & Organization 19 (2): Alesina, Alberto, and Dani Rodrik. 1994. 445­90. "Distributive Politics and Economic Growth." Agenor, Pierre-Richard. 2004. "Does Globaliza- Quarterly Journal of Economics 109 (2): tion Hurt the Poor?" International Econom- 465­90. ics and Economic Policy 1 (1): 21­51. Alesina, Alberto, and Beatrice Weder. 2002. "Do Aghion, Philippe, and Peter Howitt. 2009. The Corrupt Governments Receive Less Foreign Economics of Growth. Cambridge, Mass.: The Aid?" The American Economic Review 92 (4): MIT Press. 1126­37. Aghion, Philippe, Eve Caroli, and Cecilia García- Alesina, Alberto, Arnaud Devleeschauwer, Peńalosa. 1999. "Inequality and Economic William Easterly, Sergio Kurlat, and Romain Growth: The Perspective of the New Growth Wacziarg. 2003. "Fractionalization." Journal Theories." Journal of Economic Literature of Economic Growth 8 (2): 155­94. 37 (4): 1615­60. Ames, Brian, Ward Brown, Shanta Devarajan, and Aghion, Philippe, Peter Howitt, and David Alejandro Izquierdo. 2001. "Macroeconomic Mayer-Foulkes. 2005. "The Effect of Financial Policy and Poverty Reduction." Prepared by 119 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT The International Monetary Fund and The Balat, Jorge, Irene Brambilla, and Guido Porto. World Bank. 2007. "Realizing the Gains from Trade: Export Anderson, Kym. 2003. "Trade Liberalization, Crops, Marketing Costs, and Poverty." NBER Agriculture, and Poverty in Low-income Working Paper No. 13395. Countries." WIDER Discussion Paper No. Banerjee, Abhijit, Paul Gertler, and Maitresh 2003/25. United Nations University. Ghatak. 2002. "Empowerment and Efficiency: Andvig, Jens, and Odd-Helge Fjeldstad. 2008. Tenancy Reform in West Bengal." The Journal "Crime, Poverty and Police Corruption in of Political Economy 110 (2): 239­80. Developing Countries." Christian Michelsen Banerjee, Abhijit, Shawn Cole, Esther Duflo, and Institute (CMI) Working Paper No. 2008:7. Leigh Linden. 2005. "Remedying Education Angrist, Joshua, Eric Bettinger, and Michael Evidence from Two Randomized Experiments Kremer. 2006. "Long-Term Educational in India." NBER Working Paper No. 11904. Consequences of Secondary School Vouchers: ------. 2007. "Remedying Education: Evidence Evidence from Administrative Records in from Two Randomized Experiments in India." Colombia." American Economic Review 96 Quarterly Journal of Economics, 122 (3): (3): 847­62. 1235­64. Angrist, Joshua, Eric Bettinger, Erik Bloom, Bardhan, Pranab. 1997. "Corruption and Elizabeth King, and Michael Kremer. 2002. Development: A Review of Issues." Journal "Vouchers for Private Schooling in Colombia: of Economic Literature 35 (3): 1320­46. Evidence from a Randomized Natural Experi- ------. 2000. "Social Justice in the Global ment." American Economic Review 92 (5): Economy." International Labor Organiza- 1535­58. tion Social Policy Lecture, University of the Ashraf, Nava, James N. Berry, and Jesse Shapiro. Western Cape, South Africa. September 1­6. 2007. "Can Higher Prices Stimulate Product ------. 2006a. "The Economist's Approach to Use? Evidence from a Field Experiment in the Problem of Corruption." World Develop- Zambia." NBER Working Paper No. 13247. ment 34 (2): 341­48. Ashraf, Quamrul, Ashley Lester, and David Weil. ------. 2006b. "Institutional Economics of 2008. "When Does Improving Health Raise Development: Some General Reflections." GDP?" NBER Working Paper No. 14449. Opening Lecture at CESifo/BREAD Confer- Ayers, Robert. 1997. "Crime and Violence as ence on Institutional Economics at San Development Issues in Latin America and the Servolo, Italy. Caribbean." Paper prepared for a World Bank ------. 2006c. "Institutions, Trade and Develop- Seminar on Urban Criminal Violence, Rio de ment." Paper prepared for UNCTAD, Geneva, Janeiro, March 2­4. Switzerland. Ayyagari, Meghana, Asli Demirguc-Kunt, and Barham, Vicky, Robin Boadway, Maurice March- Vojislav Maksimovic. 2007. "Firm Innovation and, and Pierre Pestieau. 1995. "Education in Emerging Markets: Role of Governance and and the Poverty Trap." European Economic Finance." Policy Research Working Paper No. Review 39 (7): 1257­75. 4157. World Bank, Washington, DC. Barro, Robert. 1999. "Determinants of Democra- Azfar, Omar, Young Lee, and Anand Swamy. 2001. cy." Journal of Political Economy 107 (6) 2. "The Causes and Consequences of Corrup- ------. 2003. "Determinants of Economic tion." Annals of the American Academy of Growth in a Panel of Countries." Annals of Political and Social Science 573: 42­56. Economics and Finance 4: 231­74. Bahl, Roy, and Richard Bird. 2008. "Tax Policy Barro, Robert, and Xavier Sala-i Martin. 2003. in Developing Countries: Looking Back Economic Growth. Cambridge, Mass.: The and Forward." Institute for International MIT Press. Business Working Paper No. 13. Joseph L. Barth, James, Gerard Caprio, and Ross Levine. Rotman School of Management, University 2006. Rethinking Bank Regulation: Till Angels of Toronto. Govern. New York: Cambridge University Press. 120 REFERENCES Beck, Paul, and Michael Maher. 1986. "A Compari- Bettinger, Eric, Michael Kremer, and Juan son of Bribery and Bidding in Thin Market," Saavedra. 2008. "Are Educational Vouchers Economics Letters 20: 1­5. Only Redistributive?" Draft, prepared for the Beck, Thorsten, Asli Demirguc-Kunt, and Ross CESifo/PEPG joint conference "Economic Levine. 2004. "Finance, Inequality, and Incentives: Do They Work in Education?" Poverty: Cross-Country Evidence." NBER Munich, Germany, May 16­17. Working Paper No. 10979. Bird, Richard. 2003. "Administrative Dimensions ------. 2005. "Law and Firms' Access to Finance." of Tax Reform." International Tax Program American Law and Economics Review 7 (1): Paper No. 0302. Joseph L. Rotman School of 211­52. Management, University of Toronto. Becker, Gary, and George Stigler. 1974. "Law ------. 2005. "Value-Added Taxes in Develop- Enforcement, Malfeasance and the Compen- ing and Transitional Countries: Lessons and sation of Enforcers." Journal of Legal Studies Questions." International Tax Program Paper 3 (1): 1­19. No. 0505. Joseph L. Rotman School of Manage- Bell, Clive, Shantayanan Devarajan, and Hans ment, University of Toronto. Gersbach. 2006. "The Long-Run Economic ------. 2008. "Tax Challenges Facing Develop- Costs of AIDS: A Model with an Application ing Countries." Institute for International to South Africa." The World Bank Economic Business Working Paper No. 12. Review 20 (1): 55­89. Bird, Richard, and Duanjie Chen. 1998. "The . Berger, Allen N., W Scott Frame, and Nathan H. Fiscal Framework for Business in Asia." In Miller. 2005. "Credit Scoring and the Availabil- Fiscal Framework and Financial System ity, Price, and Risk of Small Business Credit." in East Asia: How Much Do They Matter?, Journal of Money, Credit, and Banking 37 Wendy Dobson (ed.). Toronto: University of (2): 191­222. Toronto Press. Besley, Timothy. 1995. "Property Rights and Bird, Richard, and Pierre-Pascal Gendron. 2006. Investment Incentives: Theory and Evidence "Is VAT the Best Way to Impose a General from Ghana." The Journal of Political Consumption Tax in Developing Countries?" Economy 103 (5): 903­37. International Tax Program Paper No. 0602. Besley, Timothy, and Robin Burgess. 2000. "Land Joseph L. Rotman School of Management, Reform, Poverty Reduction, and Growth: University of Toronto. Evidence from India." Quarterly Journal of ------. 2007. The VAT in Developing and Transi- Economics 115 (2): 389­430. tional Countries. New York: Cambridge ------. 2002. "The Political Economy of Govern- University Press. ment Responsiveness: Theory and Evidence." Bird, Richard, and Barbara Miller. 1989. "The Quarterly Journal of Economics 117 (4): Incidence of Indirect Taxation on Low-Income 1415­51. Households in Jamaica." Economic Develop- ------. 2003. "Halving Global Poverty." The ment and Cultural Change 37 (2): 393­409. Journal of Economic Perspectives 17 (3): Bird, Richard, and Enid Slack. 2002. "Land and 3­22. Property Taxation around the World: A Besley, Timothy, and Anne Case. 1995. "Does Review." Journal of Property Tax Assessment Electoral Accountability Affect Economic and Administration 7 (3): 31­80. Policy Choices? Evidence from Gubernatorial Bird, Richard, and Eric Zolt. 2007. "Tax Policy Term Limits." Quarterly Journal of Econom- in Emerging Countries." International Tax ics 110 (3): 769­98. Program Paper No. 0707. Joseph L. Rotman Besley, Timothy, Robin Burgess, and Andrea Prat. School of Management, University of Toronto. 2002. "Mass Media and Political Accountabil- Bjorkman, Martina, and Jakob Svensson. 2009. ity." In The Right to Know: Institutions and "Power to the People: Evidence from a the Media, Roumeen Islam (ed.). World Bank, Randomized Field Experiment of Community- Washington, DC. Based Monitoring in Uganda." Policy Research 121 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Working Paper No. 4268. World Bank, simulating Brazil's Bolsa Escola Program." Washington, D.C. World Bank Economic Review 17 (2). Bloom, David E. and Ajay S. Mahal. 1995. "Does Bovenberg, A. Lans, and Sjak Smulders. 1995. the AIDS Epidemic Really Threaten Economic "Environmental Quality and Pollution- Growth?" NBER Working Paper No. 5148. Augmenting Technical Change in a Two-Sector ------. 1997. "Does the AIDS Epidemic Threaten Endogenous Growth Model." Journal of Economic Growth?" Journal of Econometrics Public Economics 57 (3): 369­91. 77 (1): 105­24. ------. 1996. "Transitional Impacts of Environmen- Bobonis, Gustavo J., Edward Miguel, and Charu tal Policy in an Endogenous Growth Model." Puri-Sharma. 2006. "Anemia and School Partic- International Economic Review 37 (4): 861­93. ipation." Journal of Human Resources 41 Braithwaite, Valerie. 2003. Taxing Democracy. (4): 692­721. Understanding Tax Avoidance and Evasion. Bodewig, Christian, and Akshay Sethi. 2005. Aldershot and Burlington, VT: Ashgate. Poverty, Social Exclusion and Ethnicity in Brautigam, Deborah. 2000. Aid Dependence and Serbia and Montenegro: The Case of Roma. Governance. Stockholm: Almqvist & Wiksell. World Bank, Washington, DC. International for the Swedish Ministry of Boex, L., F. Jameson, Jorge Martinez-Vazquez, Foreign Affairs. and Robert McNab. 2000. "Multi-Year Budget- Breen, Richard, and Cecilia García-Peńalosa. ing: A Review of International Practices and 2005. "Income Inequality and Macroeconomic Applications for Developing and Transitional Volatility: An Empirical Investigation." Review Economies." Public Budgeting and Finance of Development Economics 9 (3): 380­98. 20 (2). Brown, Michael, Tulio Jappelli, and Marco Boone, Peter. 1994. "The Impact of Foreign Aid Pagano. 2009. "Information Sharing and on Savings and Growth." Center for Economic Credit: Firm-Level Evidence from Transition Performance Working Paper No. 677, London Countries." Journal of Financial Interme- School of Economics. diation 18 (2): 151­72. ------. 1996. "Politics and the Effectiveness of Brunetti, Aymo, and Beatrice Weder. 1999. "A Free Foreign Aid." European Economic Review 40 Press is Bad News for Corruption." Mimeo. (2): 289­329. Basel, Switzerland: University of Basel. Boskin, Michael. 2006. "Eight Lessons for Public Burgess, Robin, and Rohini Pande. 2005. "Do Finance in Developing Economies." China Rural Banks Matter? Evidence from the Indian Development Forum, State Council of the Social Banking Experiment." American PRC, Beijing, March. Economic Review 95 (3): 780­95. Bourguignon, Francois. 1998. "Crime as a Social Burgess, Robin, and Nicholas Stern. 1993. Cost of Poverty and Inequality: A Review "Taxation and Development." Journal of Focusing on Developing Countries." DELTA, Economic Literature 31 (2): 762­830. Paris. Mimeo. Burnside, Craig, and David Dollar. 1997. "Aid, ------. 1999. "Crime, Violence and Inequita- Policies, and Growth." Policy Research ble Development." Paper prepared for the Working Paper No. 1777. World Bank, Annual World Bank Conference on Develop- Washington, DC. ment Economics, Washington, D.C., April ------. 2000. "Aid, Policies, and Growth." 28­30. American Economic Review 90 (4): 847­68. ------. 2004. "The Poverty-Growth-Inequality ------. 2004. "Aid, Policies, and Growth: Reply." Triangle." Paper presented at the Indian American Economic Review 94 (3): 781­84. Council for Research on International Buvinic, Mayra, and Andrew R. Morrison. 2000. Economic Relations, New Delhi, February 4. "Living in a More Violent World." Foreign Bourguignon, Francois, Francisco Ferreira, and Policy 118: 58­72. Phillippe Leite. 2003. "Conditional Cash Buvinic, Mayra, Andrew R. Morrison, and Michael Transfers, Schooling and Child Labor: Micro- Shifter. 1998. "Violence in Latin America and 122 REFERENCES the Caribbean: A Framework for Action." Paper Review of Recent Literature." United States presented at an Inter-American Develop- Agency for International Development ment Bank (IDB) Seminar on "Building (USAID) Final Report. Peaceful Societies: A Framework for Action." Chhibber, Ajay, and Gaurav Nayyar. 2007. Cartagena, Columbia. "Pro-Poor Growth: Explaining the Cross- Cagé, Julia. 2009a. "Literature Review for a Country Variation in the Growth Elasticity Special Study by the Independent Evaluation of Poverty." Brooks World Poverty Institute Group on the Country Policy and Institutional (BWPI) Working Paper No. 14, The University Assessment of The World Bank." Background of Manchester. Paper for this CPIA evaluation. Chong, Alberto, and Cesar Calderon. 2000. ------. 2009b. "Asymmetric Information, "Causality and Feedback between Institutional Rent Extraction and Aid Efficiency." Agence Measures and Economic Growth." Econom- Française de Développement Working Paper. ics and Politics 12 (1): 69­81. Calderon, Cesar, and Luis Serven. 2003. "The Christiaensen, Luc, Lionel Demery, and Stefano Output Cost of Latin America's Infrastruc- Paternostro. 2002. "Growth, Distribution, and ture Gap." In The Limits of Stabilization: Poverty in Africa: Messages from the 1990s." Infrastructure, Public Deficits and Growth Policy Research Working Paper No. 2810. in Latin America, William Easterly and Luis World Bank, Washington, DC. Serven (editors). Palo Alto, CA: Stanford Clemens, Michael, Steven Radelet and Rikhil University Press, and The World Bank. Bhavnani. 2004. "Counting Chickens when ------. 2004. "The Effects of Infrastructure They Hatch: The Short Term Effect of Aid Development on Growth and Income Distri- on Growth." Center for Global Development bution." Policy Research Working Paper No. Working Paper No. 44. 3400. World Bank, Washington, DC. Cnossen, Sijbren. 2004. "VAT in South Africa: Call, Charles. 2000. "Sustainable Development in What Kind of Rate Structure?" International Central America: The Challenges of Violence, VAT Monitor 15 (1): 19­24. Injustice and Insecurity." Hamburg Institut für Collier, Paul. 2007. The Bottom Billion: Why the Iberoamerika-Kunde, CA 2020 Working Paper Poorest Countries Are Failing and What No. 8. Can Be Done About It. New-York and Oxford: Campos, Edgardo, and Sanjay Pradhan. 1996. Oxford University Press. "Budgetary Institutions and Expenditure Collier, Paul, and Jan Dehn. 2001. "Aid, Shocks Outcomes. Binding Governments to Fiscal and Growth." Policy Research Working Paper Performance." Policy Research Working Paper No. 2688. World Bank, Washington, DC. No. 1646. World Bank, Washington, DC. Collier, Paul, and David Dollar. 2002. "Aid Alloca- Cardenas, Enrique, Jose. Ocampo, and tion and Poverty Reduction." European Rosemary Thorp. 2000. The Export Age: Economic Review 46 (8): 1475­1500. The Latin American Economies in the Late ------. 2004. "Development Effectiveness." Nineteenth and Early Twentieth Centuries. Economic Journal 114 (496): F244­F271. (Vol. 1, An Economic History of Twentieth Collier, Paul, and Anke Hoeffler. 2002. "Aid, Century Latin America). New York: Palgrave Policy and Peace: Reducing the Risks of Civil Press. Conflict." Journal of Defense Economics and Cashin, Paul Anthony, and Rupa Duttagupta. Peace 13 (6): 435­50. 2008. "The Anatomy of Banking Crises." IMF ------. 2004. "Aid, Policy and Growth in Working Paper WP/08/93. Post-Conflict Societies." European Economic Chand, Sheetal, and Karl Moene. 1999. "Control- Review 48 (5): 1125­45. ling Fiscal Corruption." World Development Conley, Timothy G., and Christopher R. Udry. 2007. 27(7): 1129­40. "Learning About a New Technology: Pineapple Chetwynd, Eric, Frances Chetwynd, and Bertram in Ghana." Economic Growth Center Discus- Spector. 2003. "Corruption and Poverty: A sion Paper No. 817, Yale University. 123 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Cuddington, John T. 1993. "Modeling the Dillinger, William. 1992. Urban Property Tax Macroeconomic Effects of AIDS, with an Reform. Guidelines and Recommendations. Application to Tanzania." The World Bank World Bank, Washington, DC. Economic Review 7 (2): 173­189. Di Tella, Rafael, and Ernesto Schargrodsky. 2003. Cuddington, John T., and John D. Hancock. 1994. "The Role of Wages and Auditing during a "Assessing the Impact of AIDS on the Growth Crackdown on Corruption in the City of Path of the Malawian Economy." Journal of Buenos Aires." Journal of Law and Econom- Development Economics 43 (2): 363­68. ics 46 (1): 269­92. Dalgaard, Carl-Johan, and Henrik Hansen. 2001. Dixit, Avinash. 2007. "Evaluating Recipes for "On Aid, Growth and Good Policies." Journal Development Success." World Bank Research of Development Studies 37 (6): 17­41. Observer 22 (2): 131­57. Dalgaard, Carl-Johan, Henrik Hansen, and Finn Djankov, Simeon, Rafael La Porta, Florencio Tarp. 2003. "On the Empirics of Foreign Aid Lopez-de Silanes, and Andrei Shleifer. 2009. and Growth." The Economic Journal 114 "Disclosure by Politicians." NBER Working (496): F191­F216. Paper No. 14703. Dehejia, Rajeev, and Roberta Gatti. 2002. "Child Djankov, Simeon, Caralee McLiesh and Andrei Labor: The Role of Income Variability and Shleifer. 2007. "Private Credit in 129 Access to Credit across Countries." NBER Countries." Journal of Financial Economics Working Paper No. 9018. 84 (2): 299­329. De la Croix, David, and Clara Delavallade. 2006. Dollar, David, and Aart Kraay. 2002. "Growth is "Growth, Public Investment and Corrup- Good for the Poor." Journal of Economic tion with Failing Institutions." Center for Growth 7 (3): 195­225. Operations Research and Econometrics Dollar, David, and Victoria Levin. 2005. "Sowing (CORE) Discussion Paper No. 2006/101. and Reaping: Institutional Quality and Project Demirgüç-Kunt, Asli, Edward J. Kane, and Luc Outcomes in Developing Countries." Policy Laeven. 2006. "Deposit Insurance Design Research Working Paper No. 3524. World and Implementation: Policy Lessons from Bank, Washington, DC. Research and Practice." Policy Research Doppelhofer, Gernot, Ronald Miller, and Working Paper No. 3969. World Bank, Xavier Sala-i Martin. 2004. "Determinants Washington, DC. of Long-Term Growth: A Bayesian Averaging De Nicoló, Gianni, Patrick Honohan, and Alain of Classical Estimates (BACE) Approach." Ize. 2005. "Dollarization of Bank Deposits: American Economic Review 94 (4): 813­35. Causes and Consequences." Journal of Dreze, Jacques, and Amartya Sen. 1989. Hunger Banking and Finance 29 (7): 1697­1727. and Public Action. New York: Oxford Univer- Deolalikar, Anil, Alex Brillantes Jr., Raghav Gaiha, sity Press. Ernesto Pernia, and Mary Racelis. 2002. ------. 1995. India: Economic Development "Poverty Reduction and the Role of Institu- and Social Opportunity. New Delhi: Oxford tions in Developing Asia." Asian Development University Press. Bank Economics and Research Department ------. 2002. India Development and Partici- Working Paper No. 10. pation. New Delhi: Oxford University Press. De Soto, Hernando. 2000. The Mystery of Duncan-Waite, Imani, and Michael Woolcock. Capital: Why Capitalism Triumphs in the 2008. "Arrested Development: The Political West and Fails Everywhere Else. New York: Origins and Socio-Economic Foundations of Random House. Common Violence in Jamaica." Brooks World Devas, Nick, Simon Delay, and Michael Hubbard. Poverty Institute (BWPI) Working Paper 46, 2001. "Revenue Authorities: Are They the University of Manchester, U.K. Right Vehicle for Improved Tax Administra- Due, John. 1988. Indirect Taxation in Develop- tion?" Public Administration and Develop- ing Countries. Baltimore: Johns Hopkins ment 21 (3): 211­22. University Press. 124 REFERENCES Duflo, Esther, Michael Kremer, and Jonathan . Edmonds, E.V 2007. "Child Labor." NBER Working Robinson. 2007. "Why Don't Farmers Use Paper No. 12926. Fertilizer? Experimental Evidence from Eijffinger, Sylvester, and Petra Geraats. 2005. Kenya." Massachusetts Institute of Technol- "How Transparent Are Central Banks?" ogy and Harvard University, Working Paper. European Journal of Political Economy 22 ------. 2008. "How High Are Rates of Return to (1): 1­21. Fertilizer? Evidence from Field Experiments in Emerson, Patrick, and Andre Souza. 2003. "Is Kenya." American Economic Review 98 (2): There a Child Labor Trap? Intergenerational 482­88. Persistence of Child Labor in Brazil." Economic Dyck, Alexander, David Moss, and Luigi Zingales. Development and Cultural Change 51 (2): 2008. "Media versus Special Interests." NBER 375­98. Working Paper No. 14360. Epaulard, Anne. 2003. "Macroeconomic Perfor- Easterly, William. 2001. The Elusive Quest for mance and Poverty Reduction." IMF Working Growth: Economists' Adventures and Paper WP/03/72. Misadventures in the Tropics. Cambridge, Esty, David C., and Michael E. Porter. 2005. Massachusetts: MIT Press. "National Environmental Performance: An ------. 2003. "Can Foreign Aid Buy Growth?" Empirical Analysis of Policy Results and Journal of Economic Perspectives 17 (3): Determinants." Environment and Develop- 23­48. ment Economics 10 (4): 391­434. ------. 2006. "Inequality Does Cause Underde- Evans, Peter, and James Rauch. 1999. "Bureau- velopment: Insights from a New Instrument." cracy and Growth: A Cross-National Analysis Mimeo. of the Effects of `Weberian' State Structures Easterly, William, and Stanley Fischer. 2001. on Economic Growth." American Sociologi- "Inflation and the Poor." Journal of Money, cal Review 64 (5): 748­65. Credit and Banking 33 (2): 160­78. Ferraz, Claudio, and Federico Finan. 2008. Easterly, William, and Ross Levine. 1997. "Exposing Corrupt Politicians: The Effects of "Africa's Growth Tragedy: Policies and Ethnic Brazil's Publicly Released Audits on Electoral Divisions." Quarterly Journal of Economics Outcomes." Quarterly Journal of Economics 112 (4): 1203­50. 123 (2): 703­46. ------. 2003. "Tropics, Germs, and Crops: How Fischer, Stanley. 1991. "Growth, Macroeconom- Endowments Influence Economic Develop- ics and Development." NBER Working Paper ment." Journal of Monetary Economics 50 No. 3702. (1): 3­39. ------. 1993. "The Role of Macroeconomic Easterly, William, Ross Levine, and David Factors in Growth." Journal of Monetary Roodman. 2004. "Aid, Policies, and Growth: Economics 32 (3): 485­512. Comment." American Economic Review 94 Field, Erica. 2002. "Entitled to Work: Urban (3): 774­80. Property Rights and Labor Supply in Peru." Easterly, William, and Sergio Rebelo. 1993. "Fiscal Princeton, N.J.: Princeton University. Policy and Economic Growth: An Empirical Fields, Gary, Paul Cichello, Samuel Freije, Marta Investigation." NBER Working Paper No. 4499. Menéndez, and David Newhouse. 2002. Easterly, William, Carlos Rodriguez, and Klaus "Escaping Poverty: Household Income Schmidt-Hebbel (editors). 1995. Public Sector Dynamics in Indonesia, South Africa, Spain, Deficits and Macroeconomic Performance. and Venezuela." In Pathways Out of Poverty: New York: Oxford University Press. Private Firms and Economic Mobility in Eastwood, Robert, and Michael Lipton. 2001. Developing Countries. Gary S. Fields, and "Pro-Poor Growth and Pro-Growth Poverty Guy Pfefferman (editors). Washington D.C.: Reduction: Meaning, Evidence, and Policy International Finance Corporation. Implications." Asian Development Review 18 Fjeldstad, Odd-Helge, and Mick Moore. 2007. (2): 22­58. The Asian Development Bank. "Taxation and State-Building: Poor Countries 125 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT in a Globalised World." Chr. Michelsen Ghura, Dhaneshwar, Carlos Leite, and Institute Working Paper No. 2007:11. Charalambos Tsangarides. 2002. "Is Growth Fleisig, Heywood. 1995. "The Right to Borrow: Enough? Macroeconomic Policy and Poverty Legal and Regulatory Barriers that Limit Access Reduction." IMF Working Paper WP/02/118. to Credit by Small Farms and Businesses." Glewwe, Paul, Alaka Holla, and Michael Kremer. Public Policy for the Private Sector Note 44. 2008. "Teacher Incentives in the Developing The World Bank, Washington DC. World." Mimeo. Foster, James, and Miguel Székely. 2008. "Is Glewwe, Paul, Michael Kremer, and Sylvie Moulin. Economic Growth Good for the Poor? 2007. "Many Children Left Behind? Textbooks Tracking Low Incomes Using General Means." and Test Scores in Kenya." NBER Working International Economic Review 49 (4): Paper No. 13300. 1143­72. Goldberg, Pinelopi, and Nina Pavcnik. 2004. Foster, Mick, Adrian Fozzard, Felix Naschold, "Trade, Inequality, and Poverty: What Do We and Tim Conway. 2002. "How, When and Why Know? Evidence from Recent Trade Liberal- Does Poverty Get Budget Priority? Poverty ization Episodes in Developing Countries." Reduction Strategy and Public Expenditure NBER Working Paper No. 10593. in Five African Countries." Overseas Develop- ------. 2005. "The Effects of the Colombian ment Institute Working Paper No. 168. Trade Liberalization on Urban Poverty." NBER Fozzard, Adrian, and Mick Foster. 2001. Working Paper No. 11081. "Changing Approaches to Public Expenditure Guillaumont, Patrick, and Lisa Chauvet. 2001. Management in Low-Income Aid Dependent "Aid and Performance: A Reassessment." The Countries." World Institute for Development Journal of Development Studies, Taylor and Economics Research (WIDER) Discussion Francis Journals 37 (6): 66­92. Paper No. 2001/17. Gupta, Sanjeev, Benedict Clements, Emanuele Freund, Caroline, and Nadia Rocha. 2010. "What Baldacci, and Carlos Mulas-Granados. 2005. Constrains Africa's Exports?" The World Bank, "Fiscal Policy, Expenditure Composition, and Washington, DC. Growth in Low-Income Countries." Journal Fullerton, Don. 2008. "Distributional Effects of of International Money and Finance 24 (3): Environmental and Energy Policy: An Introduc- 441­63. tion." NBER Working Paper No. 14241. Gupta, Sanjeev, Benedict Clements, Alexan- Fullerton, Don, and Seung-Rae Kim. 2006. der Pivovarsky, and Erwin Tiongson. 2003. "Environmental Investment and Policy "Foreign Aid and Revenue Response: Does with Distortionary Taxes and Endogenous the Composition of Aid Matter?" IMF Working Growth." NBER Working Paper No. 12070. Paper WP/03/176. Gallup, John L., and Jeffrey D. Sachs, with Andrew Gupta, Sanjeev, Hamid Davoodi, and Rosa Alonso Mellinger. 1999. "Geography and Economic Terme. 1998. "Does Corruption Affect Income Development." Center for International Equality and Poverty?" IMF Working Paper Development Working Paper No. 1, Harvard WP/98/76. University. ------. 2002. "Does Corruption Affect Income Gemmell, Norman, and Oliver Morrissey. 2003. Distribution?" Economic Governance 3 (1): "Tax Structure and the Incidence on the Poor 23­45. in Developing Countries." Centre for Research Gupta, Sanjeev, Luis de Mello, and Raju Sharan. on Economic Development and International 2000. "Corruption and Military Spending." Trade Research Paper No. 03/18. IMF Working Paper WP/00/23. Gertler, Paul. 2004. "Do Conditional Cash Gyimah-Brempong, Kwabena, and Samaria Transfers Improve Child Health? Evidence Munoz de Camacho. 2006. "Corruption, from PROGRESA's Control Randomized Growth, and Income Distribution: Are Experiment." American Economic Review There Regional Differences?" Economics of Papers and Proceedings 94 (2): 336­41. Governance 7 (3): 245­69. 126 REFERENCES Haacker, Markus. 2002. "The Economic Discussion Paper 34. International Food Policy Consequences of HIV/AIDS in Southern Research Institute (IFPRI), Washington, D.C. Africa." IMF Working Paper WP/02/38. He, Fang, Leigh L. Linden, and Margaret MacLeod. ------. 2004. "HIV/AIDS: The Impact on the Social 2007. "Helping Teach What Teachers Don't Fabric and the Economy." In The Macroeco- Know: An Assessment of the Pratham English nomics of HIV/AIDS, Markus Haacker, editor. Language Program." Columbia University. Washington D.C.: The International Monetary Mimeo. Fund. Heady, Christopher. 2001. "Taxation Policy in Hall, Robert, and Charles Jones. 1999. "Why Low-Income Countries." WIDER Discussion Do Some Countries Produce So Much More Paper No. 2001/81. Output per Worker Than Others?" The Headey, Derek. 2005. "Foreign Aid and Foreign Quarterly Journal of Economics 114 (1): Policy: How Donors Undermine the Effective- 83­116. ness of Overseas Development Assistance." Hansen, Henrik, and Finn Tarp. 2000. "Aid Centre for Efficiency and Productivity Analysis Effectiveness Disputed." Journal of Interna- Working Paper Series No. 05/2005, Austra- tional Development 12 (3): 375­98. lia, School of Economics, University of ------. 2001. "Aid and Growth Regressions." Queensland. Journal of Development Economics 64 (2): Heinemann, Alessandra, and Dorte Verner. 547­70. 2006. "Crime and Violence in Development. Hanushek, Eric, and Dennis Kimko. 2000. A Literature Review of Latin America and the "Schooling, Labor-Force Quality, and the Caribbean." Policy Research Working Paper Growth of Nations." American Economic 4041. The World Bank, Washington, DC. Review 90 (5): 1184­1208. Hellman, Joel, Geraint Jones, and Daniel Hanushek, Eric, and Ludger Woesmann. 2008. Kaufmann. 2000. "Seize the State, Seize the "The Role of Cognitive Skills in Economic Day: State Capture, Corruption, and Influence Development." Journal of Economic Litera- in Transition." Policy Research Working Paper ture 46 (3): 607­68. No. 2444. The World Bank, Washington, DC. Harrison, Ann. 2006. "Globalization and Poverty." Hettich, Frank. 1998. "Growth Effects of a NBER Working Paper No. 12347. Revenue-Neutral Environmental Tax Reform." Hashemi, Syed M., Sidney Ruth Schuler, and Journal of Economics 67 (3): 287­316. Ann P. Riley. 1996. "Rural Credit Programs Hoff, Karla, and Joseph Stiglitz. 2001. "Modern and Women's Empowerment in Bangladesh." Economic Theory and Development." In World Development 24 (4): 635­53. Frontiers of Development Economics: The Hausmann, Ricardo, and Michael Gavin. 1996. Future in Perspective. New York: Oxford "Securing Stability and Growth in a Shock- University Press. Prone Region: The Policy Challenges for Latin Hoggarth, Glenn, Ricardo Reis, and Victoria America." In Securing Stability and Growth Saporta. 2002. "Costs of Banking System in Latin America, Ricardo Hausmann and Instability: Some Empirical Evidence." Journal Helmut Reisen (eds.). Paris: Organization for of Banking and Finance 26 (5): 825­55. Economic Cooperation and Development Holmes, K.R., B. T. Johnson, and M. Kirkpatrick. (OECD). 1997. The 1997 Index of Economic Freedom. Hay, Jonathan R., Andrei Shleifer, and Robert New York, New York: The Heritage Founda- W. Vishny. 1996. "Privatization in Transi- tion, Dow Jones and Company, Incorpo- tion Economies, Toward a Theory of Legal rated. Reform." European Economic Review 40 Honohan, Patrick, editor. 2003. Taxation of (3­5): 559­67. Financial Intermediation: Theory and Hazell, Peter, and Lawrence Haddad. 2001. Practice for Emerging Economies. New York: "Agricultural Research and Poverty Reduction." Oxford University Press and Washington, D.C.: Food, Agriculture, and the Environment The World Bank. 127 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Honohan, Patrick. 2004a. Financial Sector Policy International Monetary Fund. 2000. "Mozambique and the Poor. World Bank Working Paper No. Fiscal Transparency Study." Aide Memoire. 43. The World Bank, Washington, DC. Irz, Xavier, Lin Lin, Colin Thirtle, and Steve ------. 2004b. "Financial Development, Growth Wiggins. 2001. "Agricultural Productivity and Poverty: How Close are the Links?" In Growth and Poverty Alleviation." Develop- Financial Development and Economic ment Policy Review 19 (4): 449­66. Growth: Explaining the Links, Charles Isern, Jennifer, David Porteous, Raul Hernandez- Goodhart (ed.). London: Palgrave. Coss, and Chinyere Egwuagu. 2005. "AML/CFT ------. 2008a. "Cross-Country Variation in Regulation: Implications for Financial Service Household Access to Financial Services." Providers That Serve Low-Income People." Journal of Banking and Finance 32 The Consultative Group to Assist the Poor (11):2493­2500. (CGAP) Focus Note 29. ------. 2008b. "Risk Management and the Costs Jamison, Dean, Jeffrey Sachs, and Jia Wang. of the Banking Crisis." National Institute 2001. "The Effect of the AIDS Epidemic on Economic Review 206 (1): 15­24. Economic Welfare in Sub-Saharan Africa." ------. 2009. "Country Policy and Institutional World Health Organization Commission on Assessment (CPIA) Indicators and the Role of Macroeconomics and Health, Working Group Finance in the Development Process." Draft 1, Working Paper No. 13. background paper for The CPIA: An Evalua- Jenkins, Glenn. 1994. "Modernization of Tax Ad- tion, Independent Evaluation Group, The ministration: Revenue Boards and Privatization World Bank. as Instruments of Change." Bulletin for Inter- Honohan, Patrick, and Daniela Klingebiel. 2003. national Fiscal Documentation 48 (2): 75­81. "The Fiscal Cost Implications of an Accommo- Johnson, Simon, John McMillan, and Christo- dating Approach to Banking Crises." Journal pher Woodruff. 2002. "Property Rights and of Banking and Finance 27 (8):1539­60. Finance." The American Economic Review Hossain, Mahabub. 1988. "Credit for Alleviation of 92 (5): 1335­56. Rural Poverty: The Grameen Bank in Bangla- Kalemli-Ozcan, Sebnem, Harl E. Ryder, and David desh." Research Report 65. International Food N. Weil. 2000. "Mortality Decline, Human Policy Research Institute, Washington, DC. Capital Investment, and Economic Growth." Huntington, Samuel. 1968. Political Order in Journal of Development Economics 62 (1): Changing Societies. New Haven: Yale Univer- 1­23. sity Press. Kambou, Gerard, Shantayanan Devarajan, and Ilahi, Nadeem, Peter Orazem, and Guilherme Mead Over. 1992. "The Economic Impact of Sedlacek. 2001. "The Implications of Child AIDS in an African Country: Simulations with Labor for Adult Wages, Income and Poverty: a Computable General Equilibrium Model of Retrospective Evidence from Brazil." The Cameroon." Journal of African Economies 1 International Monetary Fund. Mimeo. (1): 109­130. IEG (Independent Evaluation Group). 2008. Kanbur, Ravi. 2004. "Growth, Inequality and Environmental Sustainability: An Evalua- Poverty: Some Hard Questions." Mimeo. tion of World Bank Group Support. Washing- ------. 2005. "Reforming the Formula: A Modest ton, D.C.: The World Bank. Proposal for Introducing Development Inter-American Development Bank. 1997 Outcomes in IDA Allocation Procedures." "Studies of the Magnitude, Costs and Policies Revised version published in Revue d'Econo- Against Violence in Six Places: Lima, Rio de mie du Developpment. Janeiro, San Salvador, Mexico, Venezuela and Kaufmann, Daniel. 2003. "Rethinking Governance: Colombia." Prepared for the Chief Economic Empirical Lessons Challenge Orthodoxy." Officer of the Inter-American Development Discussion draft. The World Bank. Bank. Inter-American Development Bank, Kaufmann, Daniel, Aart Kraay, and Pablo Zoido- Washington, DC. Lobaton. 1999. "Governance Matters." Policy 128 REFERENCES Research Working Paper No. 2196. The World Knowles, Stephan, Paula Lorgelly, and Dorian Bank, Washington, DC. Owen. 2002. "Are Educational Gender Gaps ------. 2002. "Governance Matters II: Updated a Brake on Economic Development: Some Indicators for 2000/01." Policy Research Cross-Country Empirical Evidence." Oxford Working Paper No. 2772. The World Bank, Economic Papers 54 (1): 118­49. Washington, DC. Krebs, Tom. 2003. "Human Capital Risk and Kaufmann, Daniel, Aart Kraay, and Massimo Economic Growth." Quarterly Journal of Mastruzzi. 2003. "Governance Matters III." Economics 118 (2):709­44. Policy Research Working Paper No. 3106. The Kremer, Michael, and Alaka Holla. 2008. "Pricing World Bank, Washington, DC. and Access: Lessons from Randomized Kerekes, Carrie, and Claudia Williamson. 2008. Evaluations in Education and Health." Center "Unveiling de Soto's Mystery: Property Rights, for Global Development Working Paper No. Capital Formation, and Development." 158. Journal of Institutional Economics 4 (3): Kremer, Michael, Jessica Leino, Edward Miguel, 299­325. and Alix Peterson Zwane. 2008. "Spring Khandker, Shahidur. 1998. Fighting Poverty with Cleaning: A Randomized Evaluation of Source Microcredit: Experience in Bangladesh. New Water Quality Improvement." Harvard Univer- York: Oxford University Press. sity, Department of Economics. Mimeo. Khandker, Shahidur, 2005. "Microfinance and Krusell, Per, and Jose-Voctor Ríos-Rull. 1996. Poverty: Evidence Using Panel Data from "Vested Interests in a Positive Theory of Bangladesh." The World Bank Economic Stagnation and Growth." The Review of Review 19 (2): 263­86. Economic Studies 63 (2): 301­29. Khandker, Shahidur, and Mark Pitt. 2003. Kuznets, Simon. 1955. "Economic Growth and "The Impact of Group-Based Credit on Income Inequality." American Economic Poor Households: An Analysis of Panel Review 45 (1): 1­28. Data from Bangladesh." The World Bank, Landes, David. 1983. Revolution in Time. Washington, DC. Cambridge, MA: Harvard University Press. Kiser, Edgar, and Audrey Sacks. 2007. "Improving Laeven, Luc, and Fabian Valencia. 2008. "The Use Tax Administration in Contemporary African of Blanket Guarantees in Banking Crises." States: Lessons from History." Prepared IMF Working Paper WP/08/250. for The Thunder of History: Taxation in La Porta, Rafael, Florencio Lopez-de-Silanes, Comparative and Historical Perspective. and Andrei Shleifer. 2008. "The Economic Northwestern University, May 4­5. Consequences of Legal Origins." Journal of Klapper, Leora, Luc Laeven, and Raghuram Economic Literature 46 (2): 285­332. Rajan. 2006. "Entry Regulation as a Barrier La Porta, Rafael, Florencio Lopez-de-Silanes, to Entrepreneurship." Journal of Financial . Andrei Shleifer, and Robert W Vishny. 1998. Economics 82 (3): 591­629. "Law and Finance." Journal of Political Klasen, Stephan. 2002. "Low Schooling for Economy 106 (6): 1113­55. Girls, Slower Growth for All?" World Bank ------. 1999. "The Quality of Government." Economic Review 16 (3): 345­73. Journal of Law, Economics and Organiza- Klasen, Stephan, and Francesca Lamanna. tion 15: 222­79. 2003. "The Impact of Gender Inequality in ------. 2000. "Investor Protection and Corporate Education and Employment on Economic Governance." Journal of Financial Econom- Growth: Updated Estimates." Mimeo. ics 58: 3­27. Knack, Stephen, and Philip Keefer. 1995. "Institu- Leff, Nathaniel. 1964. "Economic Development tions and Economic Performance: Cross through Bureaucratic Corruption." The Country Tests Using Alternative Institutional American Behavioural Scientist 8 (3): 8­14. Measures." Economics and Politics 7 (3): Levine, Ross. 1997. "Financial Development 207­27. and Economic Growth: Views and Agenda." 129 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT Journal of Economic Literature 35 (2): Luoto, Jill, Craig McIntosh, and Bruce Wydick. 688­726. 2007. "Credit Information Systems in ------. 1998. "The Legal Environment, Banks, Less-Developed Countries: A Test with and Long-Run Economic Growth." Journal Microfinance in Guatemala." Economic of Money, Credit, and Banking 30 (3): Development and Cultural Change 55 (2): 596­613. 313­34. ------. 2003. "More on Finance and Growth: Maine, Henry. 1980. Popular Government. More Finance, More Growth?" The Federal London: John Murray. Reserve Bank of St. Louis. Martinez-Vazquez, Jorge, and Jameson Boex. ------. 2005. "Law, Endowments and Property 2001. "Russia's Transition to a New Federal- Rights." Journal of Economic Perspectives 19 ism." World Bank Institute, Washington, DC. (3):61­88. Mauro, Paolo. 1995. "Corruption and Growth." Levine, Ross, Norman Loayza, and Thorsten Beck. Quarterly Journal of Economics 110 (3): 2000. "Financial Intermediation and Growth: 681­712. Causality and Causes." Journal of Monetary Mazingo, Christopher. 1999. "Effects of Property Economics 16 (46). Rights on Economic Activity: Lessons from Levine, Ross, and Sara Zervos. 1998. "Stock the Stolypin Land Reform." Massachusetts Markets, Bank and Economic Growth." Institute of Technology. Mimeo. American Economic Review 88 (3): 537­58. McGillivray, Mark, Simon Feeny, Niels Hermes, . Lewis, W Arthur. 1954. "Economic Development and Robert Lensink. 2005. "It Works; It with Unlimited Supplies of Labour." Manches- Doesn't; It Can, But That Depends? 50 Years ter School of Economic and Social Sciences of Controversy over the Macroeconomic 22 (2): 139­91. Impact of Development Aid." World Institute Lien, Da Hsiang Donald. 1986. "A Note on for Development Economics Research Paper Competitive Bribery Game." Economics No. 2005/54. United Nations University. Letters 22: 337­41. McGillivray, Mark, and Oliver Morrissey. Lin, Justin Yifu. 1992. "Rural Reforms and Agricul- 2001a. "Aid Illusion and Public Sector Fiscal tural Growth in China." American Economic Behaviour." The Journal of Development Review 82 (1): 34­51. Studies 37 (6): 118­36. Littlefield, Elizabeth, Jonathan Morduch, and ------. 2001b. "Fiscal Effects of Aid." World Syed Hashemi. 2003. "Is Microfinance an Institute for Development Economic Research Effective Strategy to Reach the Millennium Discussion Paper No. 2001/61, United Nations Development Goals?" CGAP Focus Note 24. University. Loayza, Norman. 1996. "The Economics of the McLure, Charles. 1999 "Tax Holidays and Invest- Informal Sector: A Simple Model and Some ment Incentives: A Comparative Analysis." Empirical Evidence from Latin America." Bulletin for International Fiscal Documen- Carnegie-Rochester Conference Series on tation 53(8­9) 326­39. Public Policy 45 (1): 129­62. Menezes-Filho, Naercio, and Ligia Vasconcellos. Londono, Juan Luis. 1996. "Violencia y Capital 2004. "Operationalising Pro-Poor Growth: A Social." Paper presented at the Second Annual Case Study on Brazil." Prepared as part of the Conference of the World Bank on Develop- Operationalizing Pro-Poor Growth research ment in Latin America and the Caribbean. program. Bogotá, Colombia. Mikesell, John. 2003. International Experiences Lopez de Silanes Florencio, Simon Johnson, with Administration of Local Taxes: A Review Rafael La Porta, and Andrei Shleifer. 2000. of Practices and Issues. School of Public and "Tunnelling." NBER Working Paper No. 7523. Environmental Affairs, Indiana University. Lui, Francis. 1985. "An Equilibrium Queuing Model Miller, Margaret, editor. 2003. Credit Reporting of Bribery." Journal of Political Economics 93 Systems and the International Economy. (4): 760­81. Cambridge, Massachusetts: MIT Press. 130 REFERENCES Mocan, Naci. 2004. "What Determines Corrup- Village Economy Perspective." Asian Develop- tion? International Evidence from Micro ment Review 19 (1): 98­116. Data." NBER Working Paper No. 10460. Over, Mead. 1992. The Macroeconomic Impact of Mokyr, Joel. 1990. The Lever of Riches: Techno- AIDS in Sub-Saharan Africa. Working Paper. logical Creativity and Economic Progress. Population and Human Resources Depart- New York: Oxford University Press. ment. The World Bank, Washington, DC. Montiel, Peter, and Luis Servén. 2006. "Macroeco- Panday, Pranab Kumar. 2008. "Representation nomic Stability in Developing Countries: How Without Participation: Quotas for Women in Much Is Enough? "World Bank Research Bangladesh." International Political Science Observer 21 (2): 151­78. Oxford University Review 29 (4): 489­512. Press. Parry, Ian, Hilary Sigman, Margaret Walls, and Mookherjee, Dilip. 1995. "Reforms in Income Tax Roberton Williams. 2005. "The Incidence of Enforcement in Mexico." IRIS-India Working Pollution Control Policies." Rutgers Univer- Paper No. 6. University of Maryland. sity, Department of Economics, Departmental Mookherjee, Dileep, and Arindam Das-Gupta. Working Paper No. 2005­04. 1995. "Reforming Indian Income Tax Enforce- Perotti, Roberto. 1992. "Fiscal Policy, Income ment." IRIS-India Working Paper No. 3, Distribution, and Growth." Review of University of Maryland. Economics and Statistics 60 (4): 755­76. Moore, Mick. 2004. "Taxation and the Politi- ------. 1993. "Political Equilibrium, Income cal Agenda, North and South." Forum for Distribution, and Growth." Review of Development Studies 31 (1): 7­32. Economic Studies 60 (4): 755­56. Moser, Caroline. 1996. "Urban Poverty and ------. 1996. "Growth, Income Distribution, Violence: Consolidation or Erosion of Social and Democracy: What the Data Say." Journal Capital?" Second Annual World Bank Confer- of Economic Growth 1 (2): 149­87. ence on Development in Latin America, Persson, T., and G. Tabellini. 1994. "Is Inequal- Bogota. ity Harmful for Growth?" The American Moser, Caroline, and Jeremy Holland. 1997. Economic Review 84 (3): 600­21. Urban Poverty and Violence in Jamaica. Porto, Guido. 2005. "Informal Export Barriers and Washington, DC.: The World Bank. Poverty." Journal of International Econom- Muralidharan, Karthik, and Venkatesh Sundara- ics 66 (2): 447­70. raman. 2007. "Teacher Incentives in Develop- Posner, Richard. 1981. "The Economics of ing Countries: Experimental Evidence from Privacy." American Economic Review Papers India." The World Bank, Washington, DC. and Proceedings 71 (2): 405­9. September. Mimeo. ------. 1998. "Creating a Legal Framework for North, Douglas. 1990. Institutions, Institutional Economic Development." The World Bank Change and Economic Performance. New Research Observer 13 (1): 1­11. York: Cambridge University Press. Rajan, Raghuram, and Arvind Subramaniam. Ocampo, Jose. 2003. "Structural Dynamics and 2008. "Aid and Growth: What Does the Cross- Economic Growth in Developing Countries." Country Evidence Really Show?" The Review Mimeo. of Economics and Statistics 90 (4): 643­65. Olken, Benjamin. 2007. "Monitoring Corruption: Rajkumar, Andrew, and Vinaya Swaroop. 2002. Evidence from a Field Experiment in Indone- "Public Spending and Outcomes: Does Gover- sia." Journal of Political Economy 115 (2): nance Matter?" Policy Research Working Paper 200­49. No. 2840. The World Bank, Washington, DC. OED (Operations Evaluation Department). 2001. Ram, Rati. 2003. "Roles of Bilateral and Multilat- "Review of the Performance-Based Alloca- eral Aid in Economic Growth of Developing tion System, IDA10­12." The World Bank, Countries." Kyklos 56 (1):95­110. Washington, DC, February 14. Rauch, James, and Peter Evans. 2000. "Bureau- Otsuka, Keijiro. 2002, "Poverty Reduction Issues: cratic Structure and Bureaucratic Performance 131 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT in Less Developed Countries." Journal of Paper No. 134. Center for Global Develop- Public Economics 75: 49­71. ment, December. Ravallion, Martin. 2001. "Growth, Inequality and Rosegrant, Mark, and Peter Hazell. 2000. Poverty: Looking Beyond Averages." World "Transforming the Rural Asian Economy: Development 29 (11): 1803­15. The Unfinished Revolution." In Study of ------. 2004. "Looking Beyond Averages in the Rural Asia (Vol. 1), Asian Development Trade and Poverty Debate." Policy Research Bank, copublished by the Oxford University Working Paper No. 3461. The World Bank, Press. Washington, DC. Rosenstein-Rodan, Paul. 1943. "Problems of Ravallion, Martin, and Gaurav Datt. 2002. "Why Industrialisation of Eastern and South- Has Economic Growth Been More Pro-Poor in Eastern Europe." The Economic Journal 53 Some States of India than Others?" Journal of (210/211): 202­11. Development Economics 68: 381­400. Rostow, W Walt. 1960. The Stages of Economic . Reddy, Sanjay, and Camelia Minoiu. 2006. Growth: A Non-Communist Manifesto. "Development Aid and Economic Growth: A Cambridge: Cambridge University Press. Positive Long-Run Relation." Working Paper Sachs, Jeffrey. 2003. "Institutions Don't Rule: No. 29. United Nations, Department of Direct Effects of Geography on Per Capita Economics and Social Affairs. Income." NBER Working Paper No. 9490. Reinikka, Ritva, and Jakob Svensson. 2004. ------. 2005. The End of Poverty: Economic "Local Capture: Evidence from a Central Possibilities for Our Time. New York: Penguin Government Transfer Program in Uganda." USA. Quarterly Journal of Economics 119: ------. 2008. Common Wealth: Economics for 679­705. a Crowded Planet. New York: Penguin USA. Rigobon, Roberto, and Dani Rodrik. 2004. "Rule Sachs, Jeffrey D., and Andrew Warner. 1995. of Law, Democracy, Openness, and Income: "Economic Reform and the Process of Global Estimating the Interrelationships." NBER Integration." Brookings Papers on Economic Working Paper No. 10750. Activity. (1): 1­118. Rodrik, Dani. 1999. "Institutions for High-Quality Schneidman, Miriam. 1996. "Targeting At-Risk- Growth: What They Are and How to Acquire Youth: Rationales, Approaches to Service Them." Draft paper prepared for The Interna- Delivery, and Monitoring and Evaluation tional Monetary Fund Conference on Second- Issues." Latin America and Caribbean Human Generation Reforms. Washington, D.C., and Social Development Group Paper Series November 8­9. No. 2. The World Bank, Washington, DC. ------. 2000. "Comments on "Trade, Growth, Sen, Amartya. 1992. Inequality Re-examined. and Poverty" by David Dollar and Aart Kraay Cambridge: Harvard University Press. (editors). Mimeo. ------. 1997. On Economic Inequality. Oxford: ------. 2003. "Growth Strategies." NBER Clarendon Press. Working Paper No. 10050. ------. 1999. Development as Freedom. Oxford: Rodrik, Dani, Arvind Subramanian, and Francesco Oxford University Press. Trebbi. 2002. "Institutions Rule: The Primacy Shavell, Steven. 2003. "Economic Analysis of of Institutions over Geography and Integra- Property Law." NBER Working Paper No. 9695. tion in Economic Development." NBER Shleifer, Andrei, and Robert Vishny. 1993. Working Paper No. 9305. "Corruption." The Quarterly Journal of Roodman, David. 2007a. "The Anarchy of Economics 108 (3): 599­617. Numbers: Aid, Development, and Cross- Skoufias, Emmanuel, Benjamin Davis, and Sergio Country Empirics." The World Bank de la Vega. 2001. "Targeting the Poor in Mexico: Economic Review 21 (2): 255­77. An Evaluation of the Selection of Households ------. 2007b. "Macro Aid Effectiveness into PROGRESA." World Development 29 Research: A Guide for the Perplexed." Working (10): 1769­84. 132 REFERENCES Steets, Julia. 2008. "Adaptation and Refinement of Wei, Shang-Jin. 2000. "How Taxing Is Corrup- the World Bank's Country Policy and Institu- tion on International Investors?" Review of tional Assessment." Global Public Policy Economics and Statistics 82 (1): 1­11. Institute on behalf of the German Federal Weil, David N. 2005. "Accounting for the Effect of Ministry for Economic Cooperation and Health on Economic Growth." NBER Working Development. Paper No. 11455. Stern, Nicholas. 2001. A Strategy for Develop- Welch, Karen Horn, Margaret McMillan, and Dani ment. Washington, D.C.: The World Bank. Rodrik. 2004. "When Economic Reform Goes Stevens, Michael. 1999. "Interview." The World Wrong: Cashews in Mozambique." Brookings Bank, Washington, DC, January 6. Trade Forum 2003. Summers, Lawrence. 2003. "Godkin Lectures." Werlin, Herbert. 1979. "The Consequences Harvard University. Mimeo. of Corruption: Ghanaian Experience." In Tanzi, Vito. 1998. "Corruption around the World: Bureaucratic Corruption in Sub-Saharan Causes, Scope, and Cures." IMF Working Africa, Monday U. Ekpo, editor,. Washington, Paper WP/98/63. D.C.: University Press of America. Todaro, Michael. 1997. Economic Development. White, H., and E. Anderson. 2002. "Growth versus London: Longman. Distribution: Does the Pattern of Growth Topalova, Petia. 2004. "Factor Immobility and Matter?" Development Policy Review 19 (3): Regional Effects of Trade Liberalization: 267­89. Evidence from India." MIT. Mimeo. Williamson, John. 1990. "What Washington Means ------. 2005. "Trade Liberalization, Poverty and by Policy Reform." In Latin American Adjust- Inequality: Evidence from Indian Districts." ment: How Much Has Happened? Washington NBER Working Paper No. 11614. D.C.: Institute for International Economics. Treisman, Daniel. 2000. "The Causes of Corrup- ------. 2004. "The Washington Consensus tion: A Cross-National Study." Journal of as Policy Prescription for Development." A Public Economics 76 (3): 399­457. lecture in the series "Practitioners of Develop- Tressel, Thierry, and Enrica Detragiache. 2008. ment" delivered at The World Bank on January "Do Financial Sector Reforms Lead to Financial 13, 2004. Development? Evidence from a New Dataset." Winters, Alain, Neil McCulloch, and Andrew IMF Working Paper WP/08/265. McKay. 2004. "Trade Liberalization and Van Rijckeghem, Caroline, and Beatrice Weder. Poverty: The Evidence So Far." Journal of 1997. "Corruption and the Rate of Tempta- Economic Literature 42 (1): 72­115. tion: Do Low Wages in the Civil Service Cause Woolard, Ingrid, and Stephan Klasen. 2005. Corruption?" IMF Working Paper WP/97/73. "Determinants of Income Mobility and ------. 2001. "Bureaucratic Corruption and Household Poverty Dynamics in South Africa." the Rate of Temptation: Do Wages in the The Journal of Development Studies 41 (5): Civil Service Affect Corruption, and by How 865­97. Much?" Journal of Development Economics World Bank. 1992. Development and Environ- 65 (2): 307­31. ment. New York: Oxford University Press. Vandenbussche, Jerome, Philippe Aghion, and ------. 1993. The East Asian Miracle: Economic Costas Meghir. 2006. "Growth, Distance Growth and Public Policy. New York and to Frontier and Composition of Human Oxford: Oxford University Press for The World Capital." Journal of Economic Growth 11 Bank. (2): 97­127. ------. 1994. World Development Report, 1994: Vermeersch, Christel, and Michael Kremer. Infrastructure for Development. Washing- 2003. "School Meals, Educational Achieve- ton, D.C.: The World Bank. ment and School Competition: Evidence ------. 1997. World Development Report, 1997: from a Randomized Evaluation." University The State in a Changing World. Washington, of Oxford. D.C.: The World Bank. 133 THE WORLD BANK'S COUNTRY POLICY AND INSTITUTIONAL ASSESSMENT ------. 1998. Beyond the Washington Consen- World Bank. Operations Policy and Country sus: Institutions Matter. New York: Oxford Services. University Press. ------. 2004e. "Reducing Poverty: Sustaining ------. 1999. World Development Report Growth." Scaling up Poverty Reduction. A 1998/99: Knowledge for Development. New Global Learning Process and Conference in York: Oxford University Press. Shanghai, The World Bank, May 25­27. ------. 2000a. Can Africa Claim the 21st ------. 2004f. 2003 Annual Review of Develop- Century? New York: Oxford University Press. ment Effectiveness. The Effectiveness of Bank ------. 2000b. World Development Report Support for Policy Reform. 2000/2001: Attacking Poverty. New York: ------. 2004g. World Development Report 2004: Oxford University Press. Making Services Work for Poor People. New ------. 2000c. Poverty Reduction and Strategy York, NY: Oxford University Press. Paper Source Book. ------. 2004h. Building State Capacity in ------. 2001a. Engendering Development. New Africa: New Approaches, Emerging Lessons. York: Oxford University Press. Washington, D.C.: World Bank Institute, The ------. 2001b. "Enhancing IDA's Performance- World Bank. Based Allocation System." September ------. 2005. Economic Growth in the 1990s: --------.2001c. Poverty Reduction Strategy Learning from a Decade of Reform. The Sourcebook: A Resource to Assist Countries International Bank for Reconstruction and in Developing Poverty Reduction Strategies. Development/The World Bank. Washington, D.C.: The World Bank. ------. 2006a. "Chairman's Summary: IDA 14 ------. 2002a. Globalization, Growth and Mid-Term Review Meeting." November. Poverty. Washington, D.C.: The World Bank. ------. 2006b. World Development Report. New ------. 2002b. "Linking IDA Support to Country York: Oxford University Press. Performance." Third Annual Report on IDA's ------. 2007. "Country Policy and Institutional Country Assessment and Allocation Process. Assessment 2007: Assessment Question- April 30. naire." The World Bank, Washington, DC. ------. 2002c. World Bank Group Work in ------. 2008a. Commission on Growth and Low-Income Countries Under Stress: A Task Development: "The Growth Report. Strate- Force Report. Washington, D.C.: The World gies for Sustained Growth and Inclusive Bank. Development." Discussion Paper. The World ------. 2003a. "Country Policy and Institutional Bank, Washington, DC. Assessment 2003: Assessment Question- ------. 2008b. "Country Policy and Institutional naire." Washington, D.C.: The World Bank. Assessment 2008: Assessment Questionnaire." ------. 2003b. "IDA's Performance-Based Alloca- Operations Policy and Country Services. tion System: Current and Emerging Issues." September 5. World Bank, Washington, DC October. ------. 2008c. Demirguc-Kunt, Asli, Thorsten ------. 2004a. "Country Policy and Institutional Beck, and Patrick Honohan, Finance for All? Assessment: An External Panel Review." The Policies and Pitfalls in Expanding Access. World Bank, April 5. Washington, D.C. ------. 2004b. "IDA's Performance-Based Alloca- ------. 2008d. Environmental Health and Child tion System: IDA Rating Disclosure and Survival: Epidemiology, Economics, Experi- Fine-Tuning the Governance Factor." Septem- ences. Washington, D.C.: The World Bank. ber. ------. 2008e. "IDA: The Platform for Achieving ------. 2004c. "IDA's Performance-Based Alloca- Results at the Country Level." Additions to tion System: Update on Outstanding Issues." IDA Resources: the Fifteenth Replenishment. February. February. --------.2004d. "Disclosing IDA Country ------. 2008f. Poverty and the Environment: Performance Ratings." Washington, D.C.: The Understanding Linkages at the Household 134 REFERENCES Level. Washington, D.C.: The World Bank. Zwane, Alix Peterson, and Michael Kremer. 2007. World Economic Forum. 2008. Global Competi- "What Works in Fighting Diarrheal Diseases tiveness Report 2008­2009. in Developing Countries? A Critical Review." Young, Alwyn. 2004. "The Gift of the Dying: The NBER Working Paper No. 12987. Tragedy of AIDS and the Welfare of Future African Generations." NBER Working Paper No. 10991. 135 IEG PUBLICATIONS Annual Review of Development Effectiveness 2009: Achieving Sustainable Development Addressing the Challenges of Globalization: An Independent Evaluation of the World Bank's Approach to Global Programs Assessing World Bank Support for Trade, 1987­2004: An IEG Evaluation Books, Building, and Learning Outcomes: An Impact Evaluation of World Bank Support to Basic Education in Ghana Bridging Troubled Waters: Assessing the World Bank Water Resources Strategy Climate Change and the World Bank Group--Phase I: An Evaluation of World Bank Win-Win energy Policy Reforms Debt Relief for the Poorest: An Evaluation Update of the HIPC Initiative A Decade of Action in Transport: An Evaluation of World Bank Assistance to the Transport Sector, 1995­2005 The Development Potential of Regional Programs: An Evaluation of World Bank Support of Multicountry Operations Development Results in Middle-Income Countries: An Evaluation of World Bank Support Doing Business: An Independent Evaluation--Taking the Measure of the World Bank­IFC Doing Business Indicators Egypt: Positive Results from Knowledge Sharing and Modest Lending--An IEG Country Assistance Evaluation 1999­2007 Engaging with Fragile States: An IEG Review of World Bank Support to Low-Income Countries Under Stress Environmental Sustainability: An Evaluation of World Bank Group Support Evaluation of World Bank Assistance to Pacific Member Countries, 1992­2002 Extractive Industries and Sustainable Development: An Evaluation of World Bank Group Experience Financial Sector Assessment Program: IEG Review of the Joint World Bank and IMF Initiative From Schooling Access to Learning Outcomes: An Unfinished Agenda--An Evaluation of World Bank Support to Primary Education Hazards of Nature, Risks to Development: An IEG Evaluation of World Bank Assistance for Natural Disasters How to Build M&E Systems to Support Better Government IEG Review of World Bank Assistance for Financial Sector Reform An Impact Evaluation of India's Second and Third Andhra Pradesh Irrigation Projects: A Case of Poverty Reduction with Low Economic Returns Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population Improving the Lives of the Poor through Investment in Cities Improving Municipal Management for Cities to Succeed: An IEG Special Study Improving the World Bank's Development Assistance: What Does Evaluation Show: Maintaining Momentum to 2015: An Impact Evaluation of Interventions to Improve Maternal and Child Health and Nutrition Outcomes in Bangladesh New Renewable Energy: A Review of the World Bank's Assistance Pakistan: An Evaluation of the World Bank's Assistance Pension Reform and the Development of Pension Systems: An Evaluation of World Bank Assistance The Poverty Reduction Strategy Initiative: An Independent Evaluation of the World Bank's Support Through 2003 The Poverty Reduction Strategy Initiative: Findings from 10 Country Case Studies of World Bank and IMF Support Power for Development: A Review of the World Bank Group's Experience with Private Participation in the Electricity Sector Public Sector Reform: What Works and Why? An IEG Evaluation of World Bank Support Small States: Making the Most of Development Assistance--A Synthesis of World Bank Findings Social Funds: Assessing Effectiveness Sourcebook for Evaluating Global and Regional Partnership Programs Using Knowledge to Improve Development Effectiveness: An Evaluation of World Bank Economic and Sector Work and Technical Assistance, 2000­2006 Using Training to Build Capacity for Development: An Evaluation of the World Bank's Project-Based and WBI Training The Welfare Impact of Rural Electrification: A Reassessment of the Costs and Benefits--An IEG Impact Evaluation World Bank Assistance to Agriculture in Sub-Saharan Africa: An IEG Review World Bank Assistance to the Financial Sector: A Synthesis of IEG Evaluations World Bank Group Guarantee Instruments 1990­2007: An Independent Evaluation The World Bank in Turkey: 1993­2004--An IEG Country Assistance Evaluation World Bank Engagement at the State Level: The Cases of Brazil, India, Nigeria, and Russia All IEG evaluations are available, in whole or in part, in languages other than English. For our multilingual section, please visit http://www.worldbank.org/ieg. ISBN 978-0-8213-8427-5 SKU 18427