278 privatesector P U B L I C P O L I C Y F O R T H E NUMBER NOTE 2004 Donor Performance OCTOBER Tim Harford and What Do We Know, and What Should We Know? Michael Klein It would be hard to disagree with the objectives of the international Tim Harford aid industry. But how much are donors contributing to their (tharford@ifc.org) is an economist at the World achievement? Despite recent progress, we still know surprisingly Bank and International little. We know that some donors give much more than others Finance Corporation (IFC). Michael Klein relative to income. We also know that donors are focusing aid less on (mklein@worldbank.org) PRESIDENCY poor countries and more on countries with strong institutions or is chief economist at IFC good policies. And we know that there appears to be no tradeoff VICE and joint World Bank­IFC vice president, here: the countries that give the most aid also target poor countries private sector and those with good policies. Yet we are still in the dark about which development. donors, or which projects, are achieving the best results. DEVELOPMENT This Note is part of a series exploring trends Much commentary on the aid industry focuses What determines the effectiveness of aid? in the aid industry, SECTOR purely on the volume of aid. While the size of Analysts of donor performance have variously including patterns of aid flows, competition, and aid flows matters, there is surely more to be said. made three types of assumption. The first is that the effectiveness of Not all dollars of aid are equivalent; some help aid does more good if it goes to poor countries. PRIVATE different types of aid. the poor, and some do not. Researchers are This is clearly plausible, holding other things therefore paying growing attention to the qual- equal. Yet there may be reasons that aid flowing ity of the aid given by donors (sometimes in an to poor countries does not help the poor. Many attempt to provoke better practices). But there poor people live in countries that are not them- GROUP is a problem: before measuring the average selves very poor. And poorer countries tend to quality or effectiveness of aid, we need to know have less ability to use aid effectively. BANK what makes aid high quality or effective. We The second assumption, then, is that aid have only a vague idea of this, and tentative early does more good if given to countries with good research has revealed little except the paucity of policies or good institutions. This claim is based the data. As a result, researchers have tended to on a widespread view that aid works well if the WORLD base their work on plausible assumptions rather recipient country does the right things with it-- than solid research. thus the famous claim that "aid works in a good THE D O N O R P E R F O R M A N C E W H A T D O W E K N O W , A N D W H A T S H O U L D W E K N O W ? policy environment." The paper that kicked off Figure The big donors, and the generous ones the debate, Burnside and Dollar (2000), is too 1 specific to support such a blanket claim. France UN target: Burnside and Dollar use a narrow definition of Largest Japan 0.7% "a good policy environment" (a particular com- donors United States bination of low inflation, fiscal rectitude, and an open economy) and "aid works" (GDP growth in the recipient country must be higher in the Denmark four-year period following the four-year period Highest Norway 2 in which the aid was received). Subsequent effort Sweden analysis has shown that the conclusions change 0 1 dramatically if the specification of the model is ODA as a percentage of GNI, 2002 changed or the data set expanded. Never- theless, since the basic idea that "good govern- Source: World Bank 2004. ments" use aid better than "bad governments" remains plausible, it seems right that it should Monetary Fund--have claimed that aid is more play a role in evaluating aid quality. effective in the form of grants rather than loans. The third assumption is that how aid is given Once more, no evidence is given. matters. For example, David Roodman (2004) Since the claims about what makes aid effective and other members of the Center for Global are plausible but rest on modest evidence or none Development have separately claimed that at all, we need a realistic view of what the quanti- donors should give support for central govern- tativeindicatorsofaidqualityactuallyshow.These ment budgets (program aid) if governments are indicators are suggestive but hardly conclusive. strong and benign, but should switch to support for particular projects (project aid) if govern- What indicators are out there? ments are weak or corrupt. While this claim is Thethreemainassumptionsaboutwhatmakesaid plausible, Roodman does not support it with evi- effective underlie several indicators of aid quality: dence. The World Bank's Global Monitoring Poverty focus. McGillivray (1989) constructed Report 2004 (2004) argues that aid is more effec- an income-weighted index running back to tive in flexible forms like cash rather than as debt 1969. Aid to poorer countries receives a relief, emergency aid, or technical cooperation. higher weight, with aid to a recipient with per This claim too is plausible but unsupported. capita income halfway between richest and Other prominent figures--such as Ken Rogoff, poorest given a weight of 0.5. Roodman former chief economist of the International (2004), in a composite index, also gives a higher weight to aid going to poorer coun- Table Ranking donors on aid effort and quality tries. Dollar and Levin (2004) measure the poverty "elasticity" of aid--how much aid 1 Aid quality indexes flows vary depending on the poverty of the Poverty Policy recipient country. elasticity elasticity Composite Policy or institutional selectivity. Dollar and (Dollar and (Dollar and (Roodman Levin (2004) also construct a policy elasticity Donor country Aid efforta Levin 2004) Levin 2004) 2004) measure, based on the Burnside-Dollar Denmark 1 1 1 3 (2000) composite of good policy. Roodman United Kingdom 3 3 2 1 (2004) weights aid depending on whether it Norway 2 2 3 5 France 4 5 6 2 goes to countries with good governance as United States 6 4 5 4 measured by the popular Kaufmann, Kraay, Japan 5 6 4 6 and Zoido-Lobatón index (World Bank 2003). He adjusts the weights depending on Note: Rankings are based on 2002 data. a. Aid effort is measured by official development assistance as a percentage of the donor country's gross national income (GNI). The whether the aid is project aid, said to be more data are from World Bank (2004). appropriate for poorly governed countries, Figure Little sign of greater poverty focus . . . 2 Income-weighted aid index Poverty elasticity of aid flows (inverted scale) 100 ­2.0 ­1.5 80 ­1.0 60 ­0.5 40 ­0.0 1969 1974 1979 1984 1984­89 1990­94 1995­99 . . . but donors may be becoming more selective Policy elasticity of aid flows Composite aid quality index 5.0 0.6 4.0 3.0 2.0 0.4 1.0 0.0 1984­89 1990­94 1995­99 ­1.0 0.2 ­2.0 1995 1997 1999 2002 Scandinavia Japan Multilateral Total EU-4 United States Bilateral Note: The income-weighted aid index includes several measures of aid quality, including poverty focus. The composite aid quality index also includes several measures of aid quality, including targeting of well-governed countries. Source: For income-weighted aid index, McGillivray (1989); for poverty and policy elasticity indexes, Dollar and Levin (2004); and for composite aid quality index, Roodman (2004). or program aid, thought to be suited to measure of aid quality without better theories better-governed countries. and evidence about what makes aid effective. Restrictions and other measures of effectiveness. Roodman (2004) penalizes technical assis- Which donors give high-quality aid? tance and aid that is tied. (Technical assis- Using several of these measures to rank the four tance must usually be purchased from the largest donors and the two with the highest aid donor country, and tied aid must be spent on effort leads to one immediate conclusion: meas- the donor country's products and services. ures of aid quantity and aid quality seem to be cor- Both typically reduce the purchasing power related (table 1). The United States and Japan of the aid.) Roodman's attempts to make not only have the lowest effort among major such adjustments are reasonable but ad hoc. donors (figure 1); they also do relatively little tar- For example, in a previous version of his geting of aid to poor countries and to those with annual paper he subtracted administrative good policy (as defined by Burnside and Dollar). costs, but he abandoned the idea in the 2004 This result is surprising, because there is no a pri- edition after recognizing that high adminis- ori reason to suspect that quantity, a focus on trative costs might lead to better projects. poverty, and a focus on good policy should be cor- Clearly, it is difficult to settle on a convincing related. Indeed, Steven Radelet (2003) makes the D O N O R P E R F O R M A N C E W H A T D O W E K N O W , A N D W H A T S H O U L D W E K N O W ? logical point that efforts to increase the volume Relative to the 1980s, for any given income of aid will make it harder to be selective. There level of recipient, aid is increasingly aimed at are several explanations for the apparent corre- countries with better policies. lation. One is that it is easy to argue for more aid But aid is not aimed more at poor countries. when the aid itself seems to be well aimed. Indeed, it seems to be targeted less to poor viewpoint Another is that when countries are committed to countries than in the 1980s. development, they perform well in several ways. While a tradeoff might be expected between Still, the indexes sometimes agree less than giving large quantities of aid and maintain- is an open forum to might be expected--a warning to avoid reading ing the quality of aid, the reverse seems to be encourage dissemination of a glib, intuitive message into an indicator that true: the countries that give the most aid rel- public policy innovations for depends greatly on the technical details. ative to their income also score well on dif- private sector­led and ferent measures of aid quality. market-based solutions for Are donors more focused on poor countries? Yet the caveats are as important as the con- development. The views There is no sign that aid is becoming more clusions, because the indexes are based either published are those of the focused on poor countries. McGillivray's income- on narrow empirical research or on no research authors and should not be weighted index shows very slight improvements at all. One consequence is that indexes that attributed to the World in 1969­84 for the largest EU economies (the would intuitively be expected to show similar Bank or any other affiliated EU-4: Germany, France, the United Kingdom, things produce very different messages. organizations. Nor do any of and Italy) and for the multilateral agencies, but The real value of this embryonic work on aid the conclusions represent an overall fall in the poverty focus of the United quality is to highlight the importance of the official policy of the World States and no clear trend for Japan (figure 2). issue and the scale of our ignorance. Hopefully, Bank or of its Executive Dollar and Levin find that aid from all major future measures of aid quality and donor per- Directors or the countries donor groups became less responsive to poverty formance will be able to draw on rigorous meas- they represent. between the late 1980s and the late 1990s. ures of aid effectiveness. To order additional copies Are donors more selective? contact Suzanne Smith, For those who believe that aid works well in a good References managing editor, policy environment--now the consensus in the Room F 4K-206, Burnside, Craig, and David Dollar. 2000. "Aid, Policies development community--there is more hope The World Bank, and Growth." American Economic Review 90 (4): 847­68. 1818 H Street, NW, that donors are improving their aid targeting. Dollar, David, and Victoria Levin. 2004. "The Washington, DC 20433. Dollar and Levin find that the responsiveness of Increasing Selectivity of Foreign Aid, 1984­2002." Policy aid flows to good policy increased for most donor Research Working Paper 3299. World Bank, Development Telephone: groups between the late 1980s and the late 1990s Research Group, Washington, D.C. 001 202 458 7281 (see figure 2). The United States (from a low McGillivray, Mark. 1989. "The Allocation of Aid among Fax: base) and Scandinavia (from a high base) showed Developing Countries: A Multi-Donor Analysis Using a Per 001 202 522 3480 particular improvements on this measure. Capita Aid Index." World Development 17 (4): 561­68. Email: Roodman's composite aid quality index, which Radelet, Steven. 2003. "Bush and Foreign Aid." Foreign ssmith7@worldbank.org rewards donors for several important considera- Affairs (September­October): 104­17. tions--directing aid to poor countries and to well- Roodman, David. 2004. "An Index of Donor Produced by Grammarians, governed ones, giving aid in an untied form, Performance." Working Paper 42. Center for Global Inc. choosing appropriately between project and pro- Development, Washington, D.C. (Updated annually.) gram finance--shows no clear trend in 1995­2002, World Bank. 2003. "Governance Research Indicator Printed on recycled paper however, except perhaps a convergence between Country Snapshot (GRICS): 1996­2002." [http://info the better donors and the worse ones. .worldbank.org/governance/kkz2002/]. ------. 2004. Global Monitoring Report 2004: Policies and Conclusion Actions for Achieving the Millennium Development Goals and Researchers are at last beginning to recognize Related Outcomes. Washington, D.C. that the quality of aid is as important as its quan- tity and to compute indexes accordingly. Those indexes suggest three observations: T h i s N o t e i s a v a i l a b l e o n l i n e : h t t p : / / r r u . w o r l d b a n k . o r g / P u b l i c P o l i c y J o u r n a l