56808 FAST TRACK BRIEF June 30, 2009 The IEG report "The World Bank's Country Policy and Institutional Assessment," was discussed by CODE on June 30, 2009 The World Bank's Country Policy and Institutional Assessment -- an Evaluation The World Bank's Country Policy and Institutional Assessment (CPIA) assesses the conduciveness of a country's policy and institutional framework to poverty reduction, sustainable growth, and the effective use of development assistance. It plays an important role in the country performance ratings that have been used for allocating resources from the International Development Association (IDA) to eligible countries since 1980. The CPIA consists of 16 criteria grouped into four clusters--economic management, structural policies, policies for social inclusion and equity, and public sector management and institutions-- weighted equally to derive the overall CPIA rating. Since the beginning of FY09, IDA has made transparent the weights of the clusters used in the IDA allocation formula--24 percent on the first three CPIA clusters combined and 68 percent on the fourth (governance) cluster (with the remaining 8 percent weighted on portfolio performance). In other words, the governance cluster has eight and a half times the weight of each of the other three clusters in the formula. This has also made transparent the weak link between the overall CPIA index and IDA allocations, with a country's governance performance (particularly relative to its performance in the other clusters) being more important in the latter. The content of the CPIA broadly reflects the determinants of growth and poverty reduction identified in the economics literature, but some criteria need to be revised and streamlined and one criterion added. The literature offers no evidence to justify any particular set of weights on the four clusters used for IDA allocation, or the way the criteria are clustered (such as having social sectors and environment in one cluster). The literature offers only mixed evidence regarding the relevance of the content of the CPIA for aid effectiveness broadly defined--that is, that it represents the policies and institutions important for aid to lead to growth. However, the CPIA is associated with aid effectiveness defined more narrowly--the better performance of Bank loans. But there is insufficient evidence to conclude that the most heavily weighted CPIA cluster associates better with loan performance than the other three clusters. The CPIA ratings are in general reliable and correlate well with similar indicators, but it is difficult to establish an empirical link between the CPIA and growth outcomes. Having Network reviewers validate ratings helps guard against potential biases in having Bank staff rate countries on which their work programs depend. The CPIA ratings correlate better with similar indicators for IBRD than for IDA countries. This could in part be because more information is available on IBRD countries, and in part because the CPIA ratings are meant to take into account the stage of development, which is more pertinent for IDA countries, and which means ratings for these countries are more subject to judgment than those for IBRD countries. This is exacerbated by the different practices with respect to accounting for the stage of development, as none of the Regional reviewers of the CPIA do this, while Network reviewers vary in their practices. IEG makes four recommendations. First, disclose the ratings for IBRD countries in the interest of accountability and transparency. Second, remove accounting for the stage of development in the rating exercise to reduce subjectivity. Third, undertake a thorough review of the adequacy of each criterion, including a review of experience and the literature, and revise as necessary, based inter alia on the findings of this evaluation. Fourth, consider not producing an overall CPIA index while continuing to produce and publish the separate CPIA components. has an impact on growth, it can be surmised that the way the Overview CPIA enters the formula for the allocation of IDA funds is T his evaluation takes the premise that beyond driven much more by fiduciary and possibly other concerns informing IDA allocation, the CPIA is useful as a of donors than by the objectives of achieving sustained broad indicator of development effectiveness. It growth and poverty reduction. reviews the appropriateness of the CPIA as an indicator that assesses the conduciveness of a country's policies and The CPIA strives to allow for country specificity--that institutions to fostering poverty reduction, sustainable different sets of policies and institutions can achieve growth, and the effective use of development assistance. It similar outcomes--but there are some pitfalls. The assesses the relevance of the content of the CPIA through a CPIA instructions to staff indicate that outcomes should be review of the economics literature. It also assesses the taken into account when assessing policies and institutions, reliability of CPIA ratings in two ways--through comparing which helps to account for country specificity. Indeed, CPIA ratings with similar indicators, and through reviewing outcome indicators are included in the assessment of some the CPIA ratings generation process. Based on these criteria; they could also be added to other criteria, in assessments, the evaluation derives recommendations for particular trade. enhancing the CPIA. The trade criterion does not adequately allow for country specificity. The specification of particular tariff rates for Relevance of CPIA different ratings reflects a one-size-fit-all approach to trade liberalization that is not supported by country experience. The contents of the CPIA are largely relevant for growth Including export performance (an outcome indicator) in the and poverty reduction. The CPIA criteria map well with assessment would help to allow for country specificity. the determinants--policies and institutions--of growth and poverty reduction identified in the literature, although some The trade criterion also does not reflect the importance criteria can usefully be revised and streamlined and one can of complementary institutions for successful be added (see recommendations). liberalization. The two-thirds weight on trade restrictiveness and one-third weight on trade facilitation is not The evidence is mixed regarding the relevance of the supported by country experience that shows that at moderate content of the CPIA for aid effectiveness as defined tariff levels (which practically all countries currently have), (broadly) in the literature. The review of the literature complementary factors (macroeconomic stability and trade indicates there is little consensus on the impact of aid on facilitation) are more important than further tariff reduction growth itself and on the conditions under which aid can have to promote integration into the global economy. a positive impact on growth. The CPIA is missing an assessment on disadvantaged However, the CPIA is associated with aid effectiveness socioeconomic groups other than gender. Currently, only in a narrower sense--that is, the performance of World gender is being assessed with respect to equality, yet country Bank loans. Empirical analysis finds that the overall CPIA evidence indicates that social exclusion of other marginalized ratings are negatively associated with the share of problem groups could have severe poverty and growth implications. loans that in turn is correlated with loan outcomes. Important linkages among certain criteria are not Empirical analysis indicates there is insufficient reflected in the CPIA. Except for the three economic evidence to conclude that the governance cluster management criteria, all the CPIA criteria are assessed associates better with loan performance than the other independently, which could be problematic in two instances. clusters. Based on this finding, as well as the lack of First, the assessment of trade liberalization needs to take into consensus in the literature on the conditions under which aid 2 account the extent of intersectoral labor mobility because the time for IDA countries (they prevailed more often--86 former in the absence of the latter could exacerbate poverty. percent of the time--for IBRD countries). However, these Second, fiscal policy needs to be assessed in conjunction with instances made up only 6 percent of the ratings for IDA the quality of budgetary and financial management to ensure countries and 5 percent of the ratings for IBRD countries; that the fiscal condition of the country in its entirety is hence, there does not seem to be a strong upward bias in realistically captured. ratings for either group of countries. Reliability of CPIA Ratings Recommendations The Bank has made efforts over time to improve the Based on its findings, IEG has derived recommendations to definition of the CPIA rating scale to enhance the enhance the CPIA as an indicator of policies and institutions reliability of the ratings. These efforts have aimed to reduce that are important for growth, poverty reduction (or welfare staff discretion in providing ratings. more broadly), and the effective use of development assistance. The CPIA ratings correlate well with similar indicators in terms of relative rankings of countries and direction of Adoption of these recommendations could result in a change. For each of the 16 criteria, the rank correlation discontinuity in the CPIA ratings, which Bank management coefficients of CPIA ratings with similar indicators average has been trying to avoid. However, it is important that the between 0.7 and 0.8. Other indicators correlate better with CPIA reflect the latest thinking in development as well as the Bank's CPIA ratings than with those of the African lessons learned (both of which are intentions stated by the Development Bank and the Asian Development Bank, the Bank). It would also provide the opportunity to address an closest comparators to the Bank as they use almost exactly issue that some Network reviewers have raised regarding the the same CPIA guidelines. quality of the ratings for some criteria because of what they perceive as inflated baseline ratings from a few years ago. CPIA ratings correlate better with similar indicators for The recommendations are as follows. IBRD than IDA countries. This could be due in part to the greater amount of information available on IBRD than IDA First, disclose the ratings for IBRD countries. Disclosure countries, which increases the likelihood of different is important for accountability and transparency and would institutions having similar assessments on IBRD countries. It further enhance the quality of the ratings. could also be due in part to the fact that the CPIA rating exercise takes into account the stage of development Second, remove accounting for the stage of development (introduced since 2004). This is more pertinent for IDA from the CPIA exercise. If this cannot be done, at the very countries, and hence would subject ratings of those countries least it is important to clarify and justify in the guidelines to more judgment in an exercise that is already centered on which criteria should take into account the stage of staff judgment. development and how the adjustments should be made. Accounting for the stage of development in the CPIA Third, undertake a thorough review of the CPIA and ratings is problematic. In addition to the judgment revise the criteria as necessary. It is recommended that involved, accounting for the stage of development is also the review entail an in-depth literature review for each problematic because of the different practices adopted across criterion and reflect the latest thinking on development and the Bank. Regional reviewers do not take this into account, lessons learned. The criteria should reflect an appropriate while Network reviewers vary in their practices. Further, balance between liberalization and regulation. The review accounting for the stage of development means that the should also examine whether the clustering of criteria is CPIA is no longer an index in the true sense of the word. appropriate. In particular, it will examine the appropriateness of clustering the social sectors together with the environment, The review process for the CPIA, which gives the which limits the emphasis accorded to these aspects. Networks responsibility for validating the ratings, helps Guideposts for assessing the criteria need to be reviewed at to guard against potential biases in ratings, although the same time. It is also recommended that the following be there are exceptions. A major advantage of the CPIA taken into account in the review and revisions: exercise is having well-informed staff's professional judgment as the central determinant of the ratings. At the same time, · Revision of the trade criterion to include a however, having staff rate the countries on which their work subcomponent on exports that evaluates programs depend could lead to rating biases. Analysis of the performance as well as policies and institutions to 2007 review process indicates that for instances where the reduce anti-export bias. This sub-component and Networks challenged the Regions' initial proposals of a rating those on trade restrictiveness and trade facilitation increase from 2006, the Networks prevailed 73 percent of the 3 should all get equal weights. The trade Producing the different components of the CPIA without restrictiveness sub-component should be revised to assigning weights to them to arrive at an aggregate index reflect country experience that at moderate levels of would allow for different weights to be applied according to tariffs (which almost all countries have), further country contexts and use. reduction is less important than complementary factors for global integration. · Dropping or reformulating the criterion on equity of About Fast Track Briefs public resource use, as much of its content is already covered by other CPIA criteria (property rights, Fast Track Briefs help inform the World Bank Group (WBG) access to education and to credit, income transfers) managers and staff about new evaluation findings and recom- or information is lacking for an adequate assessment mendations. The views expressed here are those of IEG and (the progressivity or regressivity of taxes). should not be attributed to the WBG or its affiliated organiza- tions. Management's Response to IEG is included in the pub- · Addition of an assessment of other disadvantaged lished IEG report. The findings here do not support any general socioeconomic groups to the CPIA. This could inferences beyond the scope of the evaluation, including any infe- either replace the criterion on equity of public resource rences about the WBG's past, current or prospective overall use or be added to that criterion if it were to be performance. reformulated. · Revision of the financial sector criterion. This needs to entail: (a) revision of the weights for the three The Fast Track Brief, which summarizes major IEG evalua- subcomponents--stability, depth and efficiency, and tions, will be distributed to selected World Bank Group staff. If you would like to be added to the subscription list, please email access--in light of the importance of financial us at ieg@worldbank.org, with "FTB subscription" in the stability as reflected by recent global evidence, and subject line and your mail-stop number. If you would like to the mixed evidence on the importance of micro stop receiving FTBs, please email us at ieg@worldbank.org, finance; (b) adding assessment of policies, with "FTB unsubscribe" in the subject line. regulations, and institutions for fostering an enabling environment for the financial sector taking into account lessons learned, notably from the current crisis; and (c) strengthening the assessment of financial stability. Contact IEG: · Combining the assessment of tax policy with fiscal Director-General, Evaluation: Vinod Thomas policy. Director: Cheryl Gray (IEG-WB) Manager: Ali M. Khadr (IEGCR) · Streamlining the assessment of judicial independence Task Manager: Helena Tang (IEGCR) and the assessment of corruption in the public sector management and institutions cluster, as they are currently assessed in more than one criterion in the cluster. Copies of the report are available at: · Strengthening the assessment of the environment http://www.worldbank.org/ieg/ criterion while making the process more efficient-- IEG Help Desk: (202) 458-4497 currently, staff need to answer 85 questions even E-mail: ieg@worldbank.org though there is only one rating. · Reporting only one consolidated rating for the three economic management criteria to avoid confusion. Fourth, consider not producing an overall CPIA index while continuing to produce and publish the separate CPIA components. The overall CPIA index is not used as such for the allocation of IDA funds. With respect to the broader use of the CPIA as an index of policies and institutions, country specificity implies that the appropriate weights of the different clusters could be different depending on a country's initial conditions and stage of development. 4