21154 WORLD BANK OPERATIONS EVALUATION DEPARTMENT EVALUATION CAPACITY DEVELOPMENT ECD WORKING PAPER SERIES NUMBER 7: FEBRUARY 2000 Sub-Saharan Africa: Lessons from Experience in Supporting Sound Governance Mark Schacter 49 W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N T Monitoring and Evaluation Capacity Development in Sub-Saharan Africa Lessons from Experience in Supporting Sound Governance Mark Schacter 2000 The World Bank www.worldbank.org/html/oed Washington, D.C. OEDPK: Evaluation Capacity Development Evaluation Capacity Development (ECD) helps build sound governance in countries—improving transparency and building a performance culture within governments to support better management and policymaking, including the budget process—through support for the creation or strengthening of national/ sectoral monitoring and evaluation systems. OED aims to identify and develop good-practice approaches in countries, and to share the growing body of experience with ECD efforts. ii Contents Page I. Introduction 1 II. Understanding Monitoring and Evaluation and Governance 3 III. Issues and Experiences 5 IV. Governance Support in Sub-Saharan Africa 7 V. Promising Signs 11 VI. Links Between Governance Support and M&ECD 15 Bibliography 19 Endnotes 23 OEDPK Group Manager: Osvaldo Feinstein ECD Task Team Leader: Keith Mackay This paper was published by the Dissemination and Outreach Unit, OEDPK, directed by Elizabeth Campbell-Pagé (task team leader), including Caroline McEuen (editor), Kathy Strauss (graphics and layout), Barbara Yale (editorial assistant), and Juicy Qureishi-Huq (administrative assistant). DISCLAIMER: ECD Working Papers are produced by the World Bank Operations Evaluation Department, Partnerships and Knowledge Group (OEDPK), Outreach and Dissemination Unit. The views in this paper are those of the author and should not be attributed to the World Bank, its affiliated organizations, or its Executive Directors. iii 49 I. Introduction The World Bank is supporting monitoring and evaluation capacity development (M&ECD) in Sub-Saharan Africa. It has linked its M&ECD efforts to other governance activities in Sub-Saharan Africa, and is endeavoring to strengthen its work by building on the experience of previous work in support of governance in the Region. With this in mind, the Partnerships and Knowledge Pro- • Opportunities to maximize synergies between M&ECD grams Group of the Operations Evaluation Department and broader governance capacity building efforts in (OED) asked the Institute on Governance (IOG) to prepare a Sub-Saharan Africa. short report addressing the convergence between work to support sound governance and that on behalf of M&ECD in Overview Sub-Saharan Africa. The report was to be based on a The paper seeks to provide a basis for identifying common narrow literature review,1 as well as the IOG’s own direct issues and operational synergies between M&ECD and experience. This report seeks to provide an overview of governance in Sub-Saharan Africa. But it is necessary to issues and experiences from other areas of governance- first understand key issues and experiences in these fields. related support in Sub-Saharan Africa, and to draw out operational linkages with M&ECD. The report summarizes: Implications of Experience with Governance Support for M&ECD Efforts • Experiences and lessons from governance support in Chapter VI draws out issues and experiences that are the Region, and their implications for M&ECD work common to both governance and M&ECD, and suggests there implications for M&ECD, as well as opportunities for • Common issues and modalities between governance synergies across the two types of work. support and M&ECD in the Region 1 II. Understanding Monitoring and Evaluation and Governance For the purposes of this paper, the term monitoring and This definition goes beyond “government” to include the evaluation (M&E) will be understood as described in a role of actors from civil society and the private sector. This recent OED publication (Mackay 1999: 3). It covers a range paper focuses on the effort to strengthen public sector of tools used in the public sector to address performance management—which has been the locus of Bank support measurement, including: for sound governance—and makes some observations about the important role to be played by civil society in • Ongoing performance monitoring creating demand for better governance. • Ongoing, real-time evaluation, at the project, program/ sector, or country level, supporting continuous learning M&E can support sound governance in several ways. First, • Ex post evaluation the information produced by M&E can be an important • Performance audits input for government decisionmaking and prioritization, • Financial audits. particularly in the budget process. Second, M&E assists managers by revealing the performance of ongoing activi- Use of the term governance is derived from the IOG’s own ties—producing valuable information for planning new working definition: activities—at the project or sector level. Thus, M&E is a management tool that promotes future learning and Governance is the art of steering societies and improvement (that is, results-based management). Similarly, organizations. Governance occurs through interac- M&E information can be used to assess the performance of tions among structures, processes, and traditions that organizations and institutional reform processes. Third, determine how power is exercised, how decisions M&E contributes to accountability mechanisms, which hold are taken, and how citizens or other stakeholders managers and governments accountable for their perfor- have their say. Governance is about power, relation- mance (Mackay 1998: 8). ships, and accountability: who has influence, who decides, and how decisionmakers are held account- able (Plumptre 1999: 3). 3 III. Issues and Experiences M&ECD Governance Support Substantial M&E achievements on the ground are rare in The Patrimonial State in Sub-Saharan Africa Sub-Saharan Africa (Bratton and others 1998; Mackay 1998). Issues and experiences concerning governance support in The binding constraint would appear to be insufficient Sub-Saharan Africa are shaped by the Region’s difficult demand for M&ECD (ADB 1999; Mackay 1998, 1999; World governance environment. Characteristics of a public service Bank 1999). Few leading bureaucrats and politicians in Sub- taken for granted in the OECD countries—it is apolitical, Saharan Africa accept the value of an evaluation culture that rule-based, well-trained, adequately paid, professional, supports fact-based administrative and political account- merit-based, and dedicated to the provision of public goods ability. Demand for M&ECD does not flourish in a dysfunc- and services to citizens—simply do not apply in many Sub- tional governance environment, in which the public admin- Saharan Africa countries. Development professionals istration is seen as a vehicle to achieve personal gain and within and outside the Bank have written at length on the nurture patrimonial patron-client networks. challenges to public administrative reform in Sub-Saharan African countries posed by the “patrimonial state.” Its Other key issues and experiences related to M&ECD in Sub- features include (Dia 1996; Hope 1997; Partnership for Saharan Africa are a result or a manifestation of the demand Capacity Building in Africa 1996; Polidano and Hulme 1997; problem. They include: Sandbrook and Oelbaum 1997; World Bank 1989, 1997): • Insufficient Supply. There are simply too few people in • Predominance of personalized informal institutions most Sub-Saharan African countries capable of (including family, ethnic groups, political groups) over designing and implementing M&E activities. Many of formal institutions (for example, constitutionally the best people have left—part of the “brain drain” mandated entities such as elections, legislatures, afflicting much of Sub-Saharan Africa (Ul Haque and judiciaries, civil services) in prescribing and enforcing Aziz 1998). Training programs to raise the skills of norms related to gaining and exercising power those who remain have produced disappointing results. • Private appropriation of the state’s “public” authority • Donor Agencies in the “Driver’s Seat.” Lack of demand • Distribution of state-generated benefits by ruling elites for M&E in the Region means that much of the M&E to personal and political followers activity has occurred through donor-driven initiatives. • Selection of public officials based on personal ties M&E often addresses donors’ concerns for the account- • Relatively low levels of control and accountability ability of project inputs and outputs, rather than local applied to rulers’ use of the state’s coercive powers and concerns related to broader development issues. The management of its wealth disproportionate element of donor initiative reduces • Unwillingness by ruling elites to distinguish between the local commitment to, and ownership of, M&E efforts. personal and the public domains. • Absence of a “Learning Culture.” The notion of continu- ous learning, in which the results of performance Patrimonial norms and practices are anathema to the ideals monitoring and program evaluation are fed back into of public administration that Sub-Saharan African states the design of new programs, or the redesign of existing formally endorse and donor agencies seek to support. The ones, is poorly understood and rarely implemented. To public administration in a patrimonial state is built on a the extent that performance information is available, it dysfunctional incentive system that rewards personal is not systematically incorporated into the affinities and opportunism rather than professionalism, policymaking process. hard work, and unbiased service to the public. The average 5 public servant is poorly trained, poorly paid, and poorly • Training of civil servants managed. Public servants see little evidence of a link • Support for budget management between performance, on the one hand, and reward, on the • Development of multiyear investment budgets other, nor are they made to feel accountable for their use of • Support for debt management and tax administration public resources or their interaction with the public. • Support for economic policy formulation Talented people tend to leave the public administration for • Legal and regulatory reform (particularly to support the private sector or international agencies where the work private sector development). is more remunerative and the environment more profes- sional. Capacity thus flees the public sector; the remaining This approach to public administrative reform in the capacity is underused and not oriented toward economic Region was, on balance, a failure (see Nunberg 1996; Olowu and social development. 1999; Partnership for Capacity Building in Africa 1996; Schacter 1995; Schiavo-Campo and others 1997). It is The result is that many Sub-Saharan African countries have difficult to point to one public administration in Sub- public administrations with only minimal capability to design Saharan Africa that is significantly more capable of design- and implement public programs. The sustainability of Bank- ing and implementing public programs today than it was at funded projects in the Region,2 as evaluated by OED, is poor. the beginning of the 1980s. Indeed, many observers believe Only 37 percent of Bank operations in Africa that exited the that despite decades of donor assistance, the capacity and portfolio in FY98 (weighted by disbursements) were judged effectiveness of public administrations in the Region have likely to be sustainable, well below the Bankwide average of 58 actually declined since independence (Partnership for percent. The record in previous years, going back to 1990, is Capacity Building in Africa 1996; Schiavo-Campo 1997). equally troubling (Buckley 1999). Moreover, Bank-supported efforts achieved only minimal Bank-Supported Governance Work in Sub-Saharan Africa success, even when judged against their own limited For the Bank, governance support in Sub-Saharan Africa has objective of containing the cost (as opposed to improving meant public administrative reform. Efforts began in the early the effectiveness) of the public service. A Bank evaluation 1980s as an adjunct to structural adjustment lending, which led observed that “civil service pay and employment reforms to a strong bias toward reducing the size and cost of African have seen only limited achievements . . . especially in Africa” public administrations—a focus on efficiency rather than (Nunberg 1996: 5). effectiveness. Continuing into the early 1990s, Bank-supported public administrative reform programs in Sub-Saharan Africa Only recently have Bank-supported activities paid system- reflected the agenda of fiscal austerity. “Downsizing” was the atic attention to deeper institutional issues at the root of the period’s major theme. It was often accompanied by project dysfunctional patrimonial state—issues related to leader- components of a highly technical nature, closely tied to ship, incentives, and human capacity deficits. Yet the structural adjustment conditionalities, such as: hallmarks of patrimonialism—corruption, cronyism, and critically ineffective service delivery—remain embedded in • Formal restructuring of government agencies the fabric of governance. • Reviews of staffing and remuneration practices 6 IV. Governance Support in Sub-Saharan Africa Lessons local institutions to implement them. Hence, implementa- tion depended upon continued inputs of external technical The major lesson from two decades of governance support in resources. Sub-Saharan Africa is the failure of the blueprint approach to reform. The Bank treated “good governance” as something to It nurtured the tendency of most Sub-Saharan African be “installed” in an African country, much as a Bank infrastruc- governments to overly centralize. Political and administrative ture project might install a bridge or a power dam. The structures in the Region are often highly concentrated in the engineering formulas that underpin the blueprints of a dam or a capital city, while field administrations and local govern- bridge are independent of local circumstances. Similarly, Bank ment suffer acute shortages in quantity and quality of staff and external experts arrived with ready-made blueprints personnel and equipment. The blueprint approach, nor- for solving governance problems, assuming that governance had mally delivered through relatively short missions by external a level of technical specificity similar to that of a civil engineer- experts, lends itself to dealings with the capital city, rather ing project.3 The blueprint approach to governance reform than excursions to dispersed, subnational centers. This failed for a number of reasons. serves to widen the capacity gap between the center and the periphery. It did not take a sufficiently broad view of the problem. Cost containment, managerial efficiency, and organizational The Way Ahead for Support restructuring in the public sector are necessary and impor- tant, but cannot be successfully addressed in isolation from The failure of past approaches to governance support in underlying features of the governance environment. The Sub-Saharan Africa is instructive. It demonstrates that approach ignored the reality that governance is about future interventions, if they are to have a reasonable chance politics and power, institutions and incentives, habits and of success, must pay careful attention to: attitudes—factors that are only partly susceptible to technical fixes and quantitative specification. • The quality of local leadership for reform • Local capacity to design and implement reform It was supply-driven, neither generating nor building on domestic programs political will for reform. Many national leaders had a vested • Features of the local incentive and accountability interest in maintaining an unreformed public administration environment, particularly as they relate to the level of because—as is the case in a patrimonial state—they were the corruption in the public sector and the quality of public main beneficiaries of the dysfunctional characteristics that service delivery Bank-supported programs sought to correct. Even in cases • Capacity-building needs of decentralized as well as where leaders favored fundamental reform, Bank-supported centralized forms of governance blueprints were seen as externally imposed and alien. Reform- • Forces external to the public service that support minded leaders were left little or no role in designing Bank- governance reform. supported reform programs. They consequently felt no sense of ownership, and embraced the programs with little enthusiasm, Local Leadership, Ownership, and Commitment often agreeing only to satisfy a precondition for receiving Because governance reform goes to the heart of how power substantial Bank financial support. is exercised, how decisions are made, and how government interacts with citizens, meaningful reforms must have the It ignored the critical shortage of local institutional capacity. support of the highest leaders in the land. The reform Bank-supported reforms were rarely sustainable because message, once endorsed by the supreme leadership, needs to project designs failed to accurately gauge the capacity of be carried into the ranks by committed champions through- 7 out the public administration. Absence of strong local this question over the past five years, given its pivotal role in leadership of, and commitment to, governance reform has affecting development outcomes, and a variety of interven- been perhaps the single most important constraint to efforts tions have been launched in Sub-Saharan Africa since the to build sound governance in Sub-Saharan Africa. mid-1990s to strengthen policy capacity at this level (Bratton 1998; Schacter 1999). Local leaders must not only demonstrate a strong personal commitment to governance reform, but also believe that An emerging lesson is that the dearth of local capacity is not they are the owners of the programs implemented to only a cause but also a consequence of poor governance and produce it. When the local perception is that reform failed approaches to governance support. A feature of the programs have been designed and imposed by external patrimonial state is that qualities such as responsiveness to agencies, there will be no ownership. Conversely, when local citizens and attention to public service delivery are not actors take the lead in both recognizing the need for systematically valued or encouraged. There is little motive, governance reform and in designing the interventions to therefore, either to invest in building capacity to design and achieve it, the sense of ownership will be high. deliver public services and programs or to retain and reward individuals who already have such capacity. The Bank cannot create the levels of leadership, ownership, and commitment necessary to make reform work. But Furthermore, donors’ approaches to development support through judicious provision of financing, technical advice, (including governance support) have, perversely, impeded and general encouragement, it can play an important role in capacity building in public administrations in the Region. supporting the efforts of leaders who are already committed The donor response to capacity shortages in Africa has, too to governance reform. It can also nurture local ownership often, been to substitute their own expertise, or hired by refraining from playing a leading role in defining the expertise in the form of expatriate consultants. While this reform agenda and designing reform programs. has served donors’ immediate internal requirements for timely design and implementation of development projects, Local Capacity it has done little or nothing for the long-term goal of Low salaries and adverse working conditions make it building local capacity. Indeed, this approach is seen as difficult for public administrations in Sub-Saharan Africa to having stifled local capacity building by hindering the attract and retain well-qualified personnel. Moreover, talent development of a market for locally based technical exper- appears to have flowed out of the Region at a significant tise (World Bank 1996). Public administrations have been pace since independence in the 1960s. According to one left in the ironic situation of experiencing a domestic “brain estimate, middle- and high-level managers have emigrated drain,” while simultaneously receiving substantial flows of at a rate of more than 10,000 each year (UNCTAD estimate, foreign technical assistance.4 cited in Ul Haque and Aziz 1998). A severe shortage of the local human capacity needed to design, manage, and Incentives and Accountability implement public programs is widely recognized as a key Economics has taught us to look to the incentive environment constraint to improved governance in Sub-Saharan Africa to explain or predict human behavior. This is as relevant to the (Ul Haque and Aziz 1998; World Bank 1996). public sector in Sub-Saharan Africa as it is to any other setting. One need not dig too deeply to discover that the structure of A highly specialized—but particularly important—area of both financial and nonfinancial incentives in most public governance weakness in the Region is the capacity of the administrations in the Region is at odds with an ethos of central government (at the cabinet level) for policy analysis professionalism and service to the public. and policymaking. Donors have paid increased attention to 8 Financial Incentives. Public service salaries, with the Support for Decentralized Forms of Governance possible exception of some Franc-zone countries, are widely Central governments in Sub-Saharan African states will accepted as being too low, particularly at higher grade levels, always be indispensable actors in governance reform. The to attract and motivate a high level of dedication and capacity limitations of the central government, however, professionalism (Nunberg 1996; Schiavo-Campo 1997; Ul combined with obstacles imposed by underdeveloped Haque and Aziz 1998). It is normal in many Sub-Saharan transportation and communications infrastructure, mean African countries for public servants—faced with salaries that the effective reach of the central administration may that do not permit an acceptable standard of living—to not extend far from the capital city. “The state stops at PK indulge in petty corruption (demanding side-payments for 12”—a saying heard often in Bangui, capital of the Central the performance of routine duties such as processing files) African Republic—describes the situation well (Bierschenk and to devote much of their official working day to private and De Sardan 1997: 441).7 business. The impact of governance reform at the central level will Nonfinancial Incentives. Key nonfinancial incentives to not, therefore, be felt far from the capital city unless it is performance in public administrations in the Region relate accompanied by reforms at the subnational level. Acknowl- to the accountability regime. In many public services, there edgment of this limitation by donors and governments led is virtually no system of accountability for performance. to the launching of a wave of decentralization programs Public servants see little link between good performance across the Region during the 1990s. Despite much fanfare, and reward or recognition. Promotions and salary increases many of these programs have been limited to “paper tend to be based on seniority. Although there are formal exercises”—preparation of laws and regulations to create regulations and sanctions for various forms of misbehavior, the formal framework for decentralization. Implementation they are rarely enforced. The most obvious symptom of the has been stymied by local administrations’ lack of capacity failed accountability regime is the widespread presence of to assume their formal responsibilities. Few newly decen- corruption. Five of the 15 most corrupt countries identified tralized governments are able to handle basic functions of in Transparency International’s worldwide “Corruption personnel management, financial management, service Perceptions Index” are from the Sub-Saharan Africa Region delivery, revenue mobilization, and tax administration.8 (Transparency International 1999).5 External Forces Favoring Governance Reform Technical capacity for auditing and accounting—the Donor-supported efforts for public administrative reform backbone of an accountability system—is precariously in Sub-Saharan Africa tend to focus directly on the public weak (Johnson 1995, 1996). In many governments of the sector. But an important element of the accountability Region, there are critical shortages of accountants and regime in any country is the existence of forces outside the auditors,6 as well as an absence of recognized accounting public sector that exert pressure on public officials to and auditing standards. Audited accounts of government behave honestly, effectively, and efficiently. Meaningful expenditure, an important check on corrupt or inappropri- public administrative reform in Sub-Saharan Africa is ate use of public funds, are rarely submitted on time; when unlikely to occur solely as a result of public officials’ own finally submitted, they are often inaccurate. Some Sub- initiative and energy. The habits of the patrimonial state die Saharan African countries have gone for more than five hard; public administrations in Sub-Saharan Africa cannot years without publishing audited accounts. Accounting be relied upon to reform themselves. The normal resistance systems are often fragmented and uncoordinated, with to change—found in public and private sector organiza- individual public agencies using different systems that are tions throughout the world—is compounded in Sub- difficult to reconcile. As a result, financial reports are Saharan African countries by a deep vested interest in typically late and unreliable. maintaining the dysfunctional status quo. 9 A vicious circle is at work: government officials or by bribery (Center for Institutional (a) Public administrations have performed poorly for a Reform and the Informal Sector 1996). very long time, doing little to improve the lives of ordinary citizens. Parliament. Parliaments and national assemblies, as repre- (b)Citizens have grown to expect little from their govern- sentatives of all citizens, should be an important source of ments. pressure on government, both to minimize corruption and (c) Citizens, expecting little or nothing, place few demands to improve service delivery. The reality in many countries is on the public administration. that national assemblies lack the capacity to function as (d)The public administration, sensing little demand from effective watchdogs over the public sector. To date, few citizens for improved service, delivers little. legislatures in Sub-Saharan Africa have played a meaningful role in monitoring the activity of the executive branch, or in And so it goes . . .. exerting pressure for cleaner and more effective government performance. Elected representatives are often less edu- Breaking the vicious circle requires that actors outside the cated or less experienced than senior public servants. Few public sector exert concerted pressure on the public have the capacity to critique budget proposals or audited administration to do better. The wave of democratization accounts—hence, the ineffectiveness of many parliamen- that has swept the Region since the beginning of the 1990s tary Public Accounts Committees. provides an opening.9 But the promise of democratization is limited by the weakness and ineffectiveness of groups Civil Society. There is great untapped potential in Sub- outside the executive branch of government. Saharan Africa for citizens to act in an organized fashion to exert pressure on the public sector for improved perfor- Judiciary. A potentially critical counterweight to corruption mance. At the moment, civil society organizations tend to in the public administration, the judiciary (and the legal be weak and fragile. Few have been created explicitly to act system in general), suffers from many of the same ills as the as watchdogs over public sector corruption or the quality of broader public sector. Judges and other legal personnel are public service delivery. Many are heavily dependent on poorly paid and operate under difficult working conditions external support. Their dialogue with government is (lack of proper office space and office supplies, absence of hindered by suspicion on both sides: governments often see systems for recording and disseminating laws and legal civil society organizations as enemies rather than potential decisions, and the like). The independence of the legal partners in improving the efficiency and effectiveness of the system is also compromised in many Sub-Saharan African public sector, and civil society organizations are wary of governments. The judiciary is tainted by corruption, with working closely with the government because they fear judges sometimes influenced either by directives from being co-opted by the state. 10 V. Promising Signs Although the record of governance reform in Sub-Saharan corruption and poor performance by public servants. The Africa is poor and the challenges daunting, there are promising government has been able to increase public sector salaries indications of real progress on the ground. Bank operations substantially (although average remuneration for public have adjusted (perhaps more slowly than they should have) to servants remains below the level determined to be a the lessons of failure. Operations from the mid-1990s onward minimum living wage). Civil service wages increased by have shown increasing attention to (i) local leadership and approximately 50 percent yearly between 1990 and 1994, a ownership; (ii) involving actors outside the public administra- rise made possible by a sustained effort to reduce public tion in the process of reform; (iii) placing more emphasis on sector staff levels—staffing was cut by over 50 percent building local capacity (as opposed to substituting Bank staff or (from 320,000 to 148,000) during the first half of the 1990s. imported expert technical advisers); (iv) directly addressing questions of corruption and service delivery; and (v) supporting The government has sought to address systemic corruption in decentralization. the public sector by reshaping the accountability and nonfinan- cial incentive environment in the public administration. This chapter reviews a few highlights of promising interven- Reforms relate both to increasing the effectiveness of surveil- tions in governance reform. The selection of cases is lance and enforcement measures and altering attitudes toward deliberately nonrepresentative and subjective; it is based on corruption through interventions to raise awareness both the author’s personal biases and experiences. Other equally within and outside the government. A significant recent appropriate examples could have been chosen. The purpose development has been a constitutionally mandated strengthen- is simply to provide a few practical examples of the general ing of the Office of the Inspector General of Government (IGG), points discussed above, with emphasis on issues that a public agency authorized to “take necessary measures for the overlap questions relevant to M&ECD. Although there is a detection and prevention of corruption in public offices.” temptation to refer to these cases as best practices, it is too While the IGG’s mandate had initially been limited to making early to label them as such. They do, however, represent recommendations to the president regarding cases of corrupt approaches that are worth watching. practices in the public sector, Uganda’s 1995 Constitution granted the IGG the powers to arrest and prosecute. Its reports, Uganda which had been sent in confidence to the president, are now submitted to Parliament and made public. IGG investigations Uganda is widely cited, both within and outside the World have led to the dismissal of public officials found to be involved Bank, as the clearest example of a promising beginning to in corruption. Some political leaders have also been relieved of public administrative reform in Sub-Saharan Africa. The office following IGG investigations (Langseth and Stapenhurst key factor in Uganda’s success is local leadership and 1997). ownership. President Yoweri Museveni has made reform of the public administration a top personal priority since The government, with support from the Bank and the coming to power in 1986. Moreover, the Ugandans have international anticorruption NGO Transparency Interna- steadfastly maintained leadership of the reform program, tional, also undertook a variety of initiatives to raise ensuring that it did not become donor-driven. The Bank awareness to the issue, to change attitudes about corruption, has built on this leadership, and worked with the govern- and to involve a broad cross-section of society in the fight ment to achieve significant gains, as detailed below. against it. These have included: Incentives and Accountability • A series of high-level Integrity Workshops for senior Uganda’s extremely low public sector salaries—among the policymakers aimed at reaching consensus on action lowest in the Region—are acknowledged as a cause of plans for fighting corruption 11 • A cabinet retreat on issues related to corruption the issue head-on by dismissing long-standing cabinet • Workshops for journalists on investigative reporting ministers regarded as having participated in massive fraud related to corruption in the public sector under previous governments. Early the following year, he • Workshops for legislators aimed at sensitizing them to commissioned a report on corruption that singled out the role of Parliament in fighting corruption corrupt ministries and departments, corrupt business • A survey of citizens to uncover users’ perceptions about people, and corrupt officials and politicians. Publication of levels of corruption in selected public services. the report led to at least one ministerial resignation. Involving Actors Outside the Public Administration The first significant Bank-supported intervention was a The government, with support from the Bank, undertook a consciousness-raising exercise aimed at helping build a series of service delivery surveys—systematic surveys to broad-based, high-level consensus around an action plan for obtain data on ordinary citizens’ perceptions of the quality fighting corruption. The National Integrity Workshop of of public service delivery.10 The intent of the surveys was 1995 brought together leading Tanzanians from government, not only to provide public officials with useful management business, the judiciary, academia, and the media. The information, but also to generate pressure to improve the workshop was widely covered by the local media. Soon quality of public service delivery. The notion of direct afterward, Mkapa’s Presidential Commission on Corruption accountability of public servants to the public is novel in sought the Bank’s support in designing and implementing a Sub-Saharan Africa; a goal of the service delivery surveys is service delivery survey to assess citizens’ perceptions of to make this idea more generally understood and accepted. corruption in the police, judiciary, lands, and revenue Results of the surveys are disseminated widely, both within services. and outside the public sector. The Bank’s approach to capacity building in Sub-Saharan Decentralization Africa, like its approach to corruption, has undergone Uganda is implementing, with support from the World Bank, significant evolution over the past five years. The two a sweeping decentralization program that gives new examples cited below focus on a particular aspect of resources and broadened responsibilities to the country’s 39 capacity building for improved governance: working with district governments for the management and provision of actors both inside and outside the government to build their basic public services. capacity to make or influence public policy. As will be discussed in the final section of this paper, there is an Tanzania obvious and potentially powerful synergy between this type of governance support and M&ECD. Tanzania, like Uganda, is perceived to have one of the world’s most corrupt public administrations. State corrup- Zambia tion is perhaps its most urgent and debilitating governance problem. As with Uganda, Tanzania is an example of a In Zambia the Bank supported a comprehensive program country where the Bank built on local leadership and aimed at building the capacity of the Cabinet Office, which commitment in tackling the difficult issue of corruption in manages the business of the cabinet and the cabinet the public sector. An opportunity was created by the committees. The Cabinet Office in Zambia plays a key role election of Benjamin Mkapa as President in 1995. Widely in ensuring that cabinet deliberations produce policies that perceived as an honest leader amid a culture of corruption, are founded on sound information, and based on thorough Mkapa almost immediately signaled his intention to address consultation across government departments. Few cabinet 12 offices function well in Sub-Saharan Africa. They typically Ghana exercise little control over the quality or quantity of infor- mation flowing to the cabinet, or the management of As noted above, the potential of Parliament to act as a check cabinet’s agenda. The Zambian capacity building program on the executive branch is rarely realized in Sub-Saharan appears to have produced significant and sustainable results. Africa. The Bank is working in several countries to build It restructured the Cabinet Office, creating the first institu- the capacity of Parliaments to be more effective players in tional “home” for some of the best and brightest policy the governance process. Ghana is one recent example. The analysts in the country. It also led to the development of a Bank has supported several workshops for the Public thoroughly revised Cabinet Handbook that regularized Accounts Committee of Parliament, aimed at developing its procedures for the development and submission of docu- capacity to influence the budget-making process and to ments for cabinet consideration. The impetus behind the comment on the efficiency and effectiveness of public work was firm local leadership and commitment—in this expenditures. case, a strong-willed and influential cabinet secretary. The Zambian case has come to be regarded as a model for cabinet office reform in Sub-Saharan Africa. 13 VI. Links Between Governance Support and M&ECD The foregoing analysis suggests that strengthened account- Fundamental governance reform will not occur without a ability between the state and citizens is a cornerstone of profound change in the attitude or mindset of politicians, improved governance in Sub-Saharan Africa. Governance public officials, and citizens about the role of the public failures are often attributable in whole or in part to: sector and the meaning of public service. External inputs of money, hardware, and technical expertise will always play a • Governments’ unwillingness to make themselves role in realizing governance reform. But the experience of accountable to the citizens they are supposed to serve. the past 20 years has shown that they will have little impact • The inability of citizens and of the organizations and as long as political and bureaucratic elites regard the public institutions that are assumed to represent their interests administration as a vehicle for extracting resources from (legislatures, the judiciary, the news media, NGOs) to citizens, rather than serving them, and for pursuing private impose accountability on governments. enrichment, rather than public gain. Such a fundamental change of perspective can never be donor-led or induced Accountability provides a critical link between governance primarily by the application of external resources. It must and M&ECD in Sub-Saharan Africa. M&E is a vital element come from within. of a country’s accountability infrastructure, and therefore of its governance regime, because it provides governments and There is a clear parallel with M&ECD. The literature, as citizens with information on the effectiveness, efficiency, noted, shows that the key constraint to successful M&ECD in and quality of government programs. It provides a basis for Sub-Saharan Africa is lack of demand. Lack of demand is making public servants accountable to their administrative rooted in the absence of a strong evaluation culture, which superiors, political representatives, and the general public, stems from the absence of performance-orientation in the and provides the information to evaluate the performance public sector (ADB and World Bank 1999: 28, 40). Where of political leaders. It also provides both the possibility and prevailing attitudes in the public sector pay little or no the incentive, for public servants and politicians alike, for attention to the government’s performance in providing continuous learning. M&E is thus an element of the broader services to the public, there will naturally be little perceived governance framework. The link between governance and need for M&E. M&E is a profound one. The implication for M&ECD programming is to heed the In more particular terms, the description and analysis above difficult lesson learned by the Bank over two decades of suggests several major issues common to both governance governance work: invest up-front in activities that may lead support and M&ECD, including the following. to changes in the fundamental dysfunctional attitudes that prevail about governance and the public sector. Avoid (i) Financial and technical resources—the stock-in-trade of significant material inputs in the absence of indications that most Bank-supported operations—are usually not the attitudes are indeed changing. binding constraint to solving either governance or M&E problems. Be cautious as well about providing training for M&E staff when the governance environment in the public sector is In Tanzania, where failed governance manifests itself as not conducive to the effective use of skilled personnel. One widespread public sector corruption, a prominent parlia- experienced commentator has cited the example of failed mentarian once observed that “corruption is not as impor- training efforts in a closely related area, the training of tant as the mentality of our people. They must change from policy analysts in Sub-Saharan Africa: what they were to what they want to be” (Economist 1996). 15 A study of Gambians who returned to their home Genuine demand for M&E, like genuine demand for fundamen- institution after earning degrees from abroad tal governance reform, will only emerge from committed showed that almost none were placed in positions political and administrative leaders who understand and accept where they could apply their new skills. Despite their the risks (as well as appreciate the benefits) of improved new credentials, they were assigned similar duties to capacity for M&E. those they had before going on training . . . Simi- larly, in virtually all African countries, series after This suggests the need for the Bank to be selective in its series of management-related training courses have M&E interventions. It should focus on countries where been provided to countless public servants at all there are already committed high-level champions who are levels, yet little evidence of tangible change has willing and able to push M&ECD through layers of adminis- resulted. What is missing is conscious attention to trative and political resistance. The recent case of Cabinet integrating newly gained knowledge, practices and Office capacity building in Zambia, mentioned earlier, is skills into everyday use in the trainees’ organizations relevant. It succeeded largely because of the commitment (Koenen-Grant 1999: 7). and energy of Zambia’s top-ranking public servant, the Secretary to the Cabinet, who appreciated the need for and The earlier examples of Uganda and Tanzania are instructive. In importance of strengthening the Cabinet Office. He also had these cases, the Bank worked intensively with the govern- the stature within the government to be able to carry the ment—through high-level workshops, news media sensitiza- message about Cabinet Office reform persuasively to other tion, and citizen surveys—to launch a process aimed at creating important actors in the system. new beliefs and attitudes about public service among a variety of stakeholders. The Bank might consider introducing the notion of M&ECD to influential champions who have already emerged in other areas There are obvious opportunities for synergy between this of governance reform. If they perceive the value to be added to kind of consciousness-raising governance support and the governance agenda by M&ECD, they may champion its M&ECD. Discussion of the role and value of M&E, and of the cause as well. need for M&ECD, would be a useful addition to the kinds of governance workshops and seminars the Bank has under- The Bank might also pay attention to making it easier for taken in Tanzania and Uganda, or to the types of work the leadership to emerge in favor of interventions in difficult Bank has been doing with parliamentarians in countries governance areas. This relates to ways that reforms are such as Ghana. The M&E agenda in these countries could described and understood. Reform proposals are likely to benefit from the momentum already created on the broader be met with apathy or resistance if they are seen as not governance front. facilitating the work of people who will be affected by them or, worse, if they are seen as threatening people’s jobs or (ii) Local leadership matters. careers. Governance is about politics, power, and influence over public In contrast, proposed reforms may be met with enthusiasm if matters. It would be naive to think that such a potent subject they are viewed as both useful and nonthreatening. M&ECD, could be addressed in the absence of strong local leadership, because it may be used for both accountability and organiza- ownership, and commitment. A similar argument applies to tional learning, could be perceived by different actors in the M&ECD. M&E, by delving into the quality, efficiency, and system as representing either a threat or an opportunity. effectiveness of government programs, generates information Potential local champions for M&ECD might find it easier to that governments might view as destabilizing or capable of play a leadership role if they were able to frame M&ECD, at least shifting the balance of power between citizens and government. at its initial stages, in a way that emphasized its less-threatening 16 aspects. Again, the experience of Zambian efforts to build the problems and designing interventions to solve them. These capacity of the Cabinet Office is instructive. Positioned as the ingrained operational habits have limited the impact of gatekeeper to the cabinet, the Cabinet Office could have chosen Bank-supported activities in capacity building at the local to flaunt its position and operate in an obstructive way with the level. ministries that wished to put proposals to the cabinet. Instead, it chose to position itself as offering a service to ministries by The implications for M&ECD are identical to those for helping them submit well-prepared proposals (Koenen-Grant support across the full range of governance activities. The 1999). The lesson that “the more these offices demonstrate value Bank should: added to the process . . . the more legitimacy they enjoy” (Koenen-Grant 1999: 7) is as applicable to units responsible for • Let its local partners take the lead in developing and M&E as it is to cabinet offices. designing M&ECD interventions. The preparation process is, in itself, an important tool for capacity (iii) Work with outside actors to achieve changes within the development; it is an opportunity to expand and public administration. develop demand for local capacity to analyze M&ECD problems and design appropriate interventions to A powerful lesson from work to support sound governance in address them. Sub-Saharan Africa is that public administrations will not • Encourage and facilitate the maximum possible use of reform themselves. A push is required from agents outside the local experts for M&ECD interventions. public sector. A similar challenge applies to building the evaluation culture needed to support sustained M&ECD. Left to (v) Coordinate governance capacity-building interventions. themselves, public officials may have little reason or desire to take M&E seriously. It is important to raise the awareness of A major lesson from governance support in Sub-Saharan Africa external stakeholders, such as parliamentarians, members of the concerns the need to take a holistic view of governance news media, and representatives of civil society organizations, problems. In a similar vein, M&ECD interventions should about the value of M&E, and to expose them to the information exploit, to the greatest possible extent, potential synergies with it yields. This is a critical first step to creating the demand for other capacity building activities in related governance fields. an evaluation culture. Three types of related activities—two of these were discussed above—present obvious potential for a high degree of synergy To help build momentum outside the public administration with M&ECD: in favor of M&ECD, there are opportunities for synergy with other governance-related initiatives that are similar to those • Policymaking at the center of government. An important described under point (i) above. Discussion of M&E could current weakness of the policymaking process at the easily be added to consciousness-raising work with the center of government is that it is rarely informed by news media, civil society, and parliamentarians, as supported sound analysis, based on lessons learned from evalua- by the Bank in countries such as Ghana, Tanzania, and tion of completed programs and the monitoring of Uganda. ongoing ones (Bratton and others 1998: 25). Countries such as Zambia, Mali, and Ghana, which are in varying (iv) Donor support must be provided in ways that promote stages of designing or implementing programs related local capacity development. to capacity building for policymaking, should therefore be prime targets for M&ECD initiatives. Bank-supported governance interventions have relied • Results-based management (RBM). This occurs in a heavily on inputs of expatriate technical expertise, and Bank variety of forms. Countries such as Mali, Uganda, and staff have often taken the lead in defining local governance Tanzania have implemented service delivery surveys 17 aimed at laying the basis for a more client-focused others 1998: 4). Program evaluation is, typically, an attribute of public service. Countries such as Ghana have imple- mature management systems in which policies are reliably mented a form of performance contracting within the carried through to the end. This is not the case in most Sub- public administration that requires the top public Saharan African countries, where problems of program servant (Chief Director) in each department to commit implementation are severe (Bratton and others 1998).11 to specific departmental results for the coming year. Focusing on evaluation in an environment where relatively few Effective M&E machinery is a necessary complement to approved programs are ever implemented would seem an RBM. There is a strong case for moving ahead with inefficient approach, compared with the alternative of develop- M&ECD in countries that have shown interest in RBM. ing capacity to monitor ongoing programs. • Rational management of public expenditures. Countries such as Guinea, Ghana, and Malawi have been working (vii) Attend to the capacity building requirements of on the design and implementation of Medium-Term decentralization. Expenditure Frameworks (MTEF). Their objective is a more rational and sustainable allocation of public For a variety of reasons, having to do with factors ranging resources and a more reliable flow of resources to from the Bank’s Articles of Agreement to the Bank’s opera- government departments, contributing to smoother tional “culture,” its governance interventions have tended to program implementation. As MTEFs become well target the central government. As noted above, many established, demand for well-run M&E systems should governments across Sub-Saharan Africa are in varying increase, because annual adjustments in MTEFs will stages of designing and implementing decentralization need to be based on a continuous flow of information programs aimed at devolving responsibility to subnational about program quality and impact. authorities for financing, designing, and implementing the delivery of basic public services. The implication for M&E (vi) Tailor the intervention to the real nature of the problem is that M&ECD interventions must, at the very least, be on the ground. designed with an awareness of the rapidly growing capacity needs related to M&E found in public administrations at the The failure of the blueprint approach to governance reform was district and municipal levels. the result of donor-imposed solutions being poorly matched to the problems they were supposed to address. There is a risk of The need to address M&E capacity deficiencies is also M&ECD falling into the same syndrome of “solutions in search urgent at the central level. Given the need to prioritize M&E of problems.” This is particularly true in the Sub-Saharan interventions, a strong case can be made for focusing at African environment in the case of ex post evaluation—the present on M&ECD at the central government level. How- backward-looking analysis of the impact and efficiency of ever, even if M&ECD interventions remained targeted, for completed programs—as opposed to monitoring—ongoing the time being, on the central government, their design surveillance of the performance of current programs. Capacity should take into account the central government’s role in for both evaluation and monitoring is needed in Sub-Saharan overseeing the decentralization process and in supporting Africa, but in many countries in the Region, the more urgent the building of M&E capacity in decentralized public priority will be to build capacity for monitoring (Bratton and administrations. 18 Bibliography ADB (African Development Bank) and World Bank. 1999. Evaluation Capacity Development in Africa. Washington, D.C.: African Development Bank and the World Bank. Bierschenk, Thomas, and Jean-Pierre Olivier de Sardan. 1997. “Local Powers and a Distant State in Rural Central African Republic.” The Journal of Modern African Studies 35 (3): 441–68. Bratton, Michael, and others. 1998. Executive Offices and Policy Management in Africa’s New Democracies. IPC Monograph No. 7. Washington, D.C.: USAID. Buckley, Robert. 1999. 1998 Annual Review of Development Effectiveness. OED. Washington, D.C.: World Bank. Center for Institutional Reform and the Informal Sector. 1996. Governance and the Economy in Africa: Tools for Analysis and Reform of Corruption. College Park, MD: IRIS Center. CIET International. nd. “Service Delivery Survey. Corruption in the Police, Judiciary, Revenue and Lands Services.” Dia, Mamadou. 1994. A Governance Approach to Civil Service Reform in Sub-Saharan Africa. World Bank Technical Paper No. 225. Washington, D.C. _____. 1996. Africa’s Management in the 1990s and Beyond: Reconciling Indigenous and Transplanted Institutions. Washing- ton, D.C. Economist. 1996. “Tanzania. A Land of Potential.” Nov. 2. _____. 1998. “Kenya. At the End of a Long Reign.” April 18. _____. 1999. “Africa’s Democratic Joys and Tribulations.” June 5. Hope, Ronald, Sr. 1997. “The Political Economy of Policy Reform and Change in Africa. The Challenge of the Transition from Statism to Liberalization.” Regional Development Dialogue 18 (1): 126–38. Johnson, Sonia R. 1995. Quality Review Schemes for Auditors. Their Potential for Sub-Saharan Africa. World Bank Technical Paper Number 276, Africa Technical Department Series. Washington, D.C. _____. 1996. Education and Training of Accountants in Sub-Saharan Anglophone Africa . World Bank Technical Paper Number 305, Africa Technical Department Series. Washington, D.C. Koenen-Grant, Julie. 1999. “Challenges to Policy Management: The African Context.” Photocopy. Langseth, Petter, and others. 1997. Good Governance in Africa. A Case Study from Uganda . EDI Working Paper. Washington, D.C.: World Bank. 19 Langseth, Petter, and Rick Stapenhurst. 1997. National Integrity System Country Studies. EDI Working Paper. Washington, D.C.: World Bank. Mackay, Keith. 1999. “Evaluation Capacity Development. A Diagnostic Guide and Action Framework.” ECD Working Paper Series No. 6, OED. Washington, D.C.: OED, World Bank. Mackay, Keith, ed. 1998. Public Sector Performance—The Critical Role of Evaluation. OED. Washington, D.C.: OED, World Bank. Nunberg, Barbara. 1996. “Re-thinking Civil Service Reform: An Agenda for Smart Government.” Washington, D.C.: World Bank. Olowu, Bamidele. 1999. “Redesigning African Civil Service Reforms.” The Journal of Modern African Studies 37 (1): 1–23. Partnership for Capacity Building in Africa. 1996. Strategy and Program of Action. Washington, D.C.: World Bank. Plumptre, Tim. 1999. “Information and Communications Technologies and Governance: A Framework Study.” Ottawa, Institute On Governance. Photocopy. Polidano, Charles, and David Hulme. 1997. “No Magic Wands. Accountability and Governance in Developing Countries.” Regional Development Dialogue 18 (2): 1–16. Sandbrook, Richard, and Jay Oelbaum. 1997. “Reforming Dysfunctional Institutions Through Democratization? Reflections on Ghana.” The Journal of Modern African Studies 35 (4): 603–46. Schacter, Mark. 1995. “Recent Experience with Institutional Development Lending in the Western Africa Department (World Bank).” Paper presented to Workshop on Civil Service Reform in Anglophone Africa, Cape Town, South Africa. _____. 1999. “Cabinet Decision-Making: Lessons from Canada, Lessons for Africa.” Prepared by the Institute on Gover- nance for the Capacity Building Unit, Africa Region, World Bank. Photocopy. Schacter, Mark, and others. 1996. “New Ways of Doing Business: Reforming Africa Region Products and Practices to Enhance Impact on Capacity Building.” Background paper, Partnership for Capacity Building in Africa. World Bank, Washington, D.C. Photocopy. Schiavo-Campo, Salvatore, and others. 1997. “Government Employment and Pay. A Global Perspective.” World Bank Policy Research Working Paper No. 1771. Washington, D.C. Schick, Allen. 1998. “Why Most Developing Countries Should Not Try New Zealand Reforms.” Presented to PREM Seminar Series. World Bank, Washington, D.C. 20 Transparency International. 1999. “The Corruption Perceptions Index 1998.” http://www.transparency.de/documents/cpi/ index.html Ul Haque, Nadeem, and Jahangir Aziz. 1998. “The Quality of Governance: ‘Second-Generation’ Civil Service Reform in Africa.” IMF Working Paper. Washington, D.C. United Nations Special Initiative on Africa. 1998. “Accountability and Transparency in Africa. A Concept Paper.” Paper presented to Second African Governance Forum in Accra, Ghana. World Bank. 1989. The Long-Term Perspective Study of Sub-Saharan Africa. Washington, D.C. _____. 1997. World Development Report 1997—the State in a Changing World. New York, N.Y.: Oxford University Press for the World Bank. 21 Endnotes Mark Schacter is a director of the Institute on Governance, a not-for-profit public policy think-tank based in Ottawa, Canada. He was a staff member of the World Bank from 1990 to 1997, working for much of that time on governance and capacity-building issues in Sub-Saharan Africa. 1. For the purposes of this report, the World Bank provided the author with all background material on M&ECD in Sub- Saharan Africa. The author undertook his own separate, limited review of literature related to broader governance support in Sub-Saharan Africa. 2. Sustainability is a useful proxy for the capacity and development orientation of a public administration, because it reflects the degree to which a government is able and willing to carry on with a development intervention after the donor agency has withdrawn its support. 3. Respected observers such as Allen Schick still find it necessary to remind the Bank of the hazards of the blueprint approach. The Bank’s recent work in public administrative reform has been influenced by “new public management” (NPM) reforms designed in many OECD countries. Schick has cautioned that some NPM-style reforms—particularly the “government by contract” model implemented in New Zealand—may not be readily exportable to some developing countries, where the prerequisites for NPM may not be in place (see Schick 1998). A case in point would be Ghana (in 1996–97), where as a result of NPM-style reforms to the civil service, Ghanaian “chief directors” (equivalent to perma- nent secretaries) were being asked to enter into quasi-contracts for the improvement of departmental performance. Chief directors regarded this to a significant degree as an empty ritual, because the absence of central budgeting systems to ensure a reliable flow of financial resources to the departments severely limited the validity of departmental performance commitments. 4. Externally supplied technical assistance to Sub-Saharan Africa is estimated to cost $4 billion a year (Ul Haque and Aziz 1998, citing World Bank sources). 5. The index, based on surveys of business people, rates 85 countries on a scale from 1 to 10; lower scores indicate higher perceived levels of corruption. The five Sub-Saharan African countries at the bottom of the list, and their scores, are: Uganda (2.6), Kenya (2.5), Nigeria (1.9), Tanzania (1.9), and Cameroon (1.4). Cameroon had the lowest score of all countries surveyed. An additional six Sub-Saharan African countries had scores below 5.0. 6. In Ghana, for example, the Controller and Accountant-General’s Department has a staff complement of 4,000, of whom only 25 are qualified accountants. Among the 150 auditors and accountants in the government of The Gambia, only two have professional qualifications. 7. “PK 12” is a kilometer post located 12 kilometers from Bangui. 8. The Central African Republic, Ghana, and Mali are examples of this phenomenon. 9. At the end of the 1980s there were only four functioning multiparty democracies in Sub-Saharan Africa: Botswana, The Gambia, Mauritius, and Senegal. During the 1990s, a further 42 countries in the Region held elections. 10. The first survey covered over 5,000 households in 40 communities. 11. A study carried out in Zambia concluded that 75 percent of cabinet decisions were never implemented. Studies in Guinea-Bissau and Ghana found comparably high levels of nonimplementation. 23 Other Papers in This Series Keith Mackay, Lessons from National Experience. Stephen Brushett, Zimbabwe: Issues and Opportunities. Alain Barberie, Indonesia’s National Evaluation System. Keith Mackay, The Development of Australia’s Evaluation System. R. Pablo Guerrero O., Comparative Insights from Colombia, China, and Indonesia. Keith Mackay, Evaluation Capacity Development: A Diagnostic Guide and Action Framework. Other Recommended Reading 1. Keith Mackay (ed.), Public Sector Performance—The Critical Role of Evaluation. 2. The World Bank, Building Evaluation Capacity, Lessons & Practices no. 4, 1994.* 3. The World Bank, Designing Project Monitoring and Evaluation, Lessons & Practices no. 8, 1996.* 4. The World Bank, Evaluating Development Outcomes: Methods for Judging Outcomes and Impacts, Lessons & Practices no. 10, 1997.* 5. The World Bank, Assessing Development Effectiveness.* * These publications are soon to be available in both French and Spanish. 25 Operations Evaluations Department Publications The Operations Evaluation Department (OED), an indepen- project info). dent evaluation unit reporting to the World Bank’s Executive Directors, rates the development impact and performance E-mail: pic@worldbank.org of all the Bank’s completed lending operations. Results and Fax number: (202) 522-1500 recommendations are reported to the Executive Directors Telephone number: (202) 458-5454 and fed back into the design and implementation of new policies and projects. In addition to the individual opera- The World Bank InfoShop tions and country assistance programs, OED evaluates the Bank’s policies and processes. The World Bank InfoShop serves walk-in customers only. The InfoShop is located at: Operations evaluation studies, World Bank discussion papers, and all other documents are available from the 701 18th Street, NW World Bank InfoShop. Washington, DC 20433, USA Summaries of studies and the full text of the Précis and All other customers must place their orders through their Lessons and Practices can be read on the Internet at local distributors. http://www.worldbank.org/html/oed/ Ordering via E-Mail How to Order OED Publications If you have an established account with the World Bank, you Documents listed with a stock number and price code may may transmit your order via electronic mail on the Internet be obtained through the World Bank’s mail order service or to: books@worldbank.org Please include your account from its InfoShop in downtown Washington, DC. For information on all other documents contact the World Bank InfoShop. Ordering World Bank Publications Customers in the United States and in territories not served by any of the Bank’s publication distributors may send publication orders to: The World Bank P.O. Box 960 Herndon, VA 20172-0960 Fax: (703) 661-1501 Telephone: (703) 661-1580 The address for the World Bank publication database on the Internet is: http://www.worldbank.org (select publications/ 26 OED Study Series 1999 Annual Review of Development Effectiveness (1999) Lending for Electric Power in Sub-Saharan Africa (1996) Bangladesh: Progress Through Partnership (1999) Industrial Restructuring: World Bank Experience, Future Challenges Developing Towns and Cities: Lessons from Brazil and the Philippines (1996) (1999) Social Dimensions of Adjustment: World Bank Experience, 1980-93 Investing in Health: Development Effectiveness in the Health, Nutrition, (1996) and Population Sector (1999) 1994 Evaluation Results (1996) Nongovernmental Organizations in World Bank–Supported Projects: Ghana Country Assistance Review: A Review of Development Effective- A Review (1999) ness (1995) Evaluation and Development: The Institutional Dimension (1998) Evaluation and Development: Proceedings of the 1994 World Bank India: The Dairy Revolution (1998) Conference (1995) The World Bank’s Experience with Post-Conflict Reconstruction (1998) Developing Industrial Technology: Lessons for Policy and Practice (1995) Financial Sector Reform: A Review of World Bank Assistance (1998) The World Bank and Irrigation (1995) Rebuilding the Mozambique Economy: Assessment of a Development Partnership (1998) 1993 Evaluation Results (1995) Mainstreaming Gender in World Bank Lending: An Update (1997) Structural and Sectoral Adjustment: World Bank Experience, 1980-82 (1995) Agricultural Extension and Research: Achievements and Problems in National Systems (1997) Gender Issues in World Bank Lending (1995) Fiscal Management in Adjustment Lending (1997) The World Bank’s Role in Human Resource Development in Sub-Sa- haran Reforming Agriculture: The World Bank Goes to Market (1997) Africa: Education, Training, and Technical Assistance (1994) Paddy Irrigation and Water Management in Southeast Asia (1997) 1992 Evaluation Results (1994) Poland Country Assistance Review: Partnership in a Transition New Lessons from Old Projects: The Workings on Rural Development Economy (1997) in Northeast Brazil (1993, contains summaries in French, Portuguese, 1995 Evaluation Results (1997) and Spanish) Zambia Country Assistance Review: Turning an Economy Around World Bank Approaches to the Environment in Brazil (1993, contains (1997) summaries in French, Portuguese, and Spanish) The Aga Khan Support Program: A Third Evaluation (1996) Rapid Appraisal Methods (1993) 27