Assessing and Reforming Public Financial Management A New Approach Richard Allen Salvatore Schiavo-Campo Thomas Columkill Garrity Assessing and Reforming Public Financial Management A New Approach Richard Allen Salvatore Schiavo-Campo Thomas Columkill Garrity THE WORLD BANK © 2004 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, N.W. Washington, D.C. 20433 Telephone 202-473-1000 Internet www.worldbank.org E-mail feedback@worldbank.org All rights reserved. 1 2 3 4 07 06 05 04 The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant per- mission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978- 750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 H Street, N.W., Washington, D.C. 20433, USA, fax 202-522-2422, e-mail pubrights@world bank.org. Library of Congress Cataloging-in-Publication Data Allen, Richard, 1944 Dec. 13- Assessing and reforming public financial management: a new approach/Richard Allen, Salvatore Schiavo-Campo, Thomas Columkill Garrity. p. cm. Includes bibliographic references and index. ISBN 0-8213-5599-6 1. Finance, Public. 2. Expenditures, Public. I. Schiavo-Campo, Salva- tore. II. Garrity, Thomas Columkill, 1967­ III. Title. HJ141.A454 2003 352.4--dc22 2003061163 Contents Preface vii Acknowledgments ix Executive Summary xi Abbreviations and Acronyms xix ASSESSMENTS OF PUBLIC EXPENDITURE MANAGEMENT: RATIONALE AND CONTEXT 1 What Is Public Expenditure Management? 2 This Study's Purpose and Objectives 2 Genesis of Assessments of Public Expenditure Management and Financial Accountability 3 FIDUCIARY RISK AND FINANCIAL ACCOUNTABILITY 9 Risk 10 Accountability 12 Operational Implications 13 THE MAIN INSTRUMENTS FOR ASSESSMENTS 15 World Bank Public Expenditure Reviews 17 World Bank Country Financial Accountability Assessments 18 World Bank Country Procurement Assessment Reports 19 IMF Reports on the Observance of Standards and Codes of Fiscal Transparency 20 iv Contents World Bank­IMF Public Expenditure Tracking Assessments and Action Plans for Heavily Indebted Poor Countries 21 EC Audits 22 DFID Assessments of Fiduciary Risk 23 Questionnaires, Checklists, and Other Tools 23 METHODOLOGY AND MAIN FINDINGS 27 Methodology and Sources 27 Coverage of the Instruments 34 Similarities 35 Differences 40 Institutional and Governance Content 41 Summary 44 RECOMMENDATIONS AND ISSUES FOR FURTHER CONSIDERATION 49 Increasing Integration and Improving Coordination and Cooperation 49 Institutional and Governance Considerations 63 Follow-up and Performance Monitoring 65 Who Assesses the Assessors? 66 Developing a Programmatic and Modular Approach 67 A CONCLUDING WORD 75 ANNEX 1 SCOPE AND APPLICATION OF THE MAIN INSTRUMENTS 77 World Bank Public Expenditure Reviews 77 World Bank Country Financial Accountability Assessments 80 World Bank Country Procurement Assessment Reports 83 IMF Reports on the Observance of Standards and Codes of Fiscal Transparency 85 World Bank­IMF Public Expenditure Tracking Assessments and Action Plans for Heavily Indebted Poor Countries 87 EC Audits 90 DFID Assessments of Fiduciary Risk 94 EC and OECD Support for Improvement in Governance and Management in Central and Eastern Europe 95 Contents v UNDP CONTACT Guidelines 99 ANNEX 2 MEASURING PERFORMANCE IN PUBLIC FINANCIAL MANAGEMENT--GUIDANCE FROM THE DEVELOPMENT ASSISTANCE COMMITTEE, OECD 101 Key Issues 101 Purpose 103 Guiding Principles 105 Good Practices in Diagnostic Work 107 Good Practices in Performance Measurement 110 ANNEX 3 TECHNICAL MAP OF THE ASSESSMENT INSTRUMENTS 113 REFERENCES 123 ABOUT THE AUTHORS 127 INDEX 129 BOXES 1 The role of assessments in promoting dialogue between donors and recipient governments, and reform 16 2 Components of public expenditure management 28 3 Multiple assessments result in myriad recommendations in Burkina Faso 39 4 How an integrated approach can strengthen assessments 60 A2.1 What is a diagnostic review? 102 A2.2 Current diagnostic tools 104 A2.3 Improving public oversight of public expenditures 109 A2.4 Partners support government-led diagnostic process in Tanzania 110 A2.5 Elements of a performance measurement framework 111 vi Contents FIGURES 1 Assessment instruments' coverage of the main components of public expenditure management 32 2 A new public expenditure assessment framework 69 TABLES 1 Features of questionnaires and checklists used by various assessment instruments 25 2 Alternative approaches to public expenditure diagnostics and reform 70 A2.1 Indicators of good practice in measuring performance in public financial management 112 A3.1 Technical mapping of the assessment instruments 115 Preface Over the past 15 years donors seeking to advance development and abate poverty have placed growing emphasis on the need for effective public expenditure management and financial accountability systems. Numerous trends explain this evolution, including the realization that aid resources are fungible, the shift toward policy-based adjustment lending, the need to strengthen the links between policymaking and budget preparation, and the recognition of corruption's destructive effects. As a result many donors have introduced new diagnostic instruments and reports that describe and assess public expenditure and financial accountability laws, systems, and proce- dures in countries that receive international aid and technical assistance. These diagnostic instruments contain information on the fiduciary risks facing aid donors and recipients, and are often required by donors before aid is provided. Such information is also valuable to a recipient country as a foundation on which it can craft sustainable reforms in public expenditure and budgeting, and build institutional capacity. Drawing on a technical mapping of current assessment instruments' coverage, a review of staff guidelines and sample assessment reports, and interviews with experts from donor agencies and recipient governments, this study recommends a new approach to assessing and reforming public expenditure management. This approach has several goals: · Streamlining instruments to avoid unnecessary duplication and fill gaps in coverage. vii viii Assessing and Reforming Public Financial Management · Enhancing collaboration and promoting information-sharing between donors, recipient governments, and other stakeholders. · Providing more complete, accurate, and timely assessments of fiduciary risk. · Monitoring improvements in public expenditure management using appropriate indicators and benchmarks. · Increasing the development impact of public expenditure assessments and reforms. · Developing a standardized assessment--one that synthesizes informa- tion using a common format, including key performance indicators-- that is accepted by all donors as a basis for measuring and monitoring fiduciary risk when providing budget support. Some efforts are already under way to strengthen collaboration on pub- lic expenditure work between the World Bank and the International Mon- etary Fund (IMF), among multilateral development banks, between the Bank and IMF and the European Commission, and between multilateral and bilateral donors. These harmonization efforts--recognized in the Feb- ruary 2003 Rome Declaration on Harmonization--are being supported by organizations such as the OECD's Development Assistance Committee and the Public Expenditure and Financial Accountability (PEFA) program. Boards of directors of donor organizations, the European Parliament, and the European Court of Audit are also exerting considerable influence. But such initiatives are only just beginning. Considerable effort will be required to sustain and advance them, supported by changes in operational procedures and incentives in the agencies concerned. In addition, recipient governments must take a stronger leadership role in this work--particular- ly in developing and implementing strategic action plans to build capacity and manage reform. This report is designed to foster and further such devel- opments, drawing on global experiences to strengthen assessment instru- ments and improve public expenditure management around the world. Acknowledgments The analysis in this study was carried out under the Public Expenditure and Financial Accountability (PEFA) program, a partnership established in December 2001 involving the World Bank, IMF, European Commission, Strategic Partnership with Africa, and several bilateral donors (France, Norway, Switzerland, and the United Kingdom). PEFA's mandate is to support integrated, harmonized approaches to the assessment and reform of public expenditure, procurement, and financial accountability, focusing on the use of diagnostic instruments. Many such instruments have been developed in recent years. In 2002 PEFA conducted a research project that mapped the technical scope and coverage of the instruments and identified areas of overlap as well as areas inadequately cov- ered. The project also investigated the methods and procedures used by donor agencies in undertaking these diagnostic reviews, the extent to which the work was done in collaboration with other donors and recipient govern- ments, and its likely impact on development. This work resulted in a PEFA report issued in the spring of 2003 that has been adapted into this book. The authors are grateful to Serif Sayin and David Steedman for valuable contributions to initial work on the PEFA research project; to the PEFA Secretariat--particularly Mike Boniakowski, Odile Keller, and Nicola Smithers--for advice and technical input; to members of the PEFA Steer- ing Committee--Armando Araujo, Ivor Beazley, Paul Bermingham, Pamela Bigart, Jim Brumby, Jack Diamond, Simon Gill, Cheryl Gray, Hen- ix x Assessing and Reforming Public Financial Management rik Harboe, Gilles Hervio, Jean-Louis Lacube, Sanjay Pradhan, Gradimir Radisic, and Helen Sutch--for support and encouragement; to Anand Rajaram (PERs), David Shand (CFAAs), Pamela Bigart (CPARs), Taryn Parry (Fiscal ROSCs), Bill Dorotinsky and Jim Brumby (HIPC AAPs), and Gradmir Radisic (EC audits) for advice on the specific instruments; and to many colleagues in the World Bank, IMF, European Commission, United Nations Development Programme, OECD Support for Improvement in Governance and Management (SIGMA) program, U.K. Department for International Development (DFID), and other organizations for insights and helpful comments at various stages in preparing this book. The study also benefited from comments on a draft presented to the financial man- agement and accountability subgroup of the OECD Development Assis- tance Committee's Task Force on Donor Practices and to a financial man- agement working group of the Strategic Partnership with Africa. Finally, the authors are indebted to the editorial and production team at the World Bank--Santiago Pombo-Bejarano, Stephenie DeKouadio, Paul Holtz (consultant), Nancy Lammers, and Mary Fisk--for enormous help in preparing the final manuscript for publication. Executive Summary This study is intended to help underpin a more coordinated, effective approach to assessing and reforming systems for public expenditure, pro- curement, and financial accountability in developing countries--especially countries that receive international aid for budget support. Such support, also known as adjustment lending, has become far more important in recent years. At the World Bank, for example, it increased from less than 10 per- cent of total assistance in the 1980s to about 50 percent in fiscal 2002. Many other development agencies are also increasing aid for budget support. This support has been accompanied by--and reflects--widespread recognition that aid is fungible and that resources can be transferred, so that aid intended for one project can effectively be used to finance another. Thus, efforts to safeguard the integrity of donor resources mean little with- out safeguards on the use of government resources. Moreover, growing awareness of the destructive effects of corruption--emphatically under- scored by the East Asian financial crisis of 1997­99--has given new urgency to donors' need to ensure that aid is not diverted to private ends or misallocated to activities not conducive to fostering growth and reducing poverty. For all these reasons it is important, for donors and recipient gov- ernments alike, that the strengths and weaknesses of national budget sys- tems be well understood and that governments implement reforms where needed, especially in high-risk areas. xi xii Assessing and Reforming Public Financial Management Development agencies use a variety of instruments to assess these sys- tems; this study focuses on World Bank Public Expenditure Reviews (PERs), Country Financial Accountability Assessments (CFAAs) and Coun- try Procurement Assessment Reports (CPARs), International Monetary Fund (IMF) Reports on the Observance of Standards and Codes of Fiscal Transparency (Fiscal ROSCs), IMF­World Bank Public Expenditure Tracking Assessments and Action Plans for Heavily Indebted Poor Coun- tries (HIPC AAPs), EC audits of public expenditure management systems, and U.K. Department for International Development (DFID) assessments of fiduciary risk. The study's review of these assessment instruments raises many impor- tant issues. Accordingly, its findings and recommendations are intended to generate discussion and debate between development agencies, recipient governments that are (or may become) the subject of such assessments, and other stakeholders. If the findings and recommendations are accepted, they will need to be translated into changes in the operational rules and practices of the agen- cies concerned. Organizational structures, management processes, staffing requirements, training programs, and internal incentives will need to be reviewed, as will arrangements for improving cooperation and coordination between development agencies, recipient governments, and other stake- holders. The World Bank, IMF, European Commission, and other development agencies are already considering important reforms to their approaches to public expenditure work, including ways of increasing their collaboration. But increased efforts are needed--and it is hoped that this study will con- tribute to these and other emerging reforms. MAIN FINDINGS At the core of this study is a mapping exercise that compares the main fea- tures and focuses of donor instruments for assessing public expenditure management. This comparison, complemented by interviews with govern- ment and donor officials and by reviews of assessment guidelines and sam- ple reports, shows that the wide variety of assessment instruments has evolved in an uncoordinated way. As a result these instruments often impose high transaction costs on recipient governments and development agencies. Executive Summary xiii The instruments have a range of objectives, including gauging fiduciary risk, supporting development goals, defining action plans, and monitoring progress on implementing those plans. Sometimes these objectives are combined in a single instrument, as with CFAAs, CPARs, and EC audits. This mix of objectives--both within and across instruments--often inhibits clear, coherent assessment work. Moreover, though the instruments reviewed in this study are often referred to as assessments of public expen- diture management, some provide limited coverage of a broader set of issues--including the forecasting of government revenue, and the manage- ment of public debt, of the government's assets (physical and financial), and of the procedures for maintaining records of its financial business and transactions. This broader concept should be used in rationalizing the use of these instruments. Though efforts are being made to strengthen it, collaboration on assess- ments between the World Bank, IMF, European Commission, and other donors remains relatively weak. As a result there is substantial overlap between some of the instruments' coverage of public expenditure manage- ment--especially between CFAAs and Fiscal ROSCs. In addition, there is overlap between CFAAs and PERs on "upstream" (preparation and pro- gramming) and especially "downstream" (execution, accounting, control, reporting, monitoring and evaluation) issues. But in most cases this overlap is manageable because CFAAs focus on budget comprehensiveness, realism, and classification, and draw on analysis from PERs whenever possible. Guidelines for EC audits were recently revised, while those for HIPC AAPs were only recently developed. Both instruments have different approaches and objectives from the others, and as such add value to assess- ment efforts. EC audits include "compliance tests," which use audit tech- niques to provide reasonable assurance that public expenditure manage- ment systems and procedures are implemented consistently, in line with relevant rules and regulations. HIPC AAPs provide comprehensive sum- maries of public expenditure management systems and performance-- including benchmarks and indicators that allow for monitoring over time. Although efforts are needed to integrate HIPC AAPs with other diagnos- tic work, work on HIPC AAPs is genuinely collaborative, relying on joint teams of Bank and IMF staff. None of the instruments provides a comprehensive analysis of govern- ment management of taxes and revenues--though revenue issues are part- ly covered in PERs, and recently updated CFAA guidelines recommend xiv Assessing and Reforming Public Financial Management addressing accounting and control issues. There are also gaps in coverage of asset management, aid and debt management, management of govern- ment financial records, and public finance issues related to subnational gov- ernments, public enterprises, and off-budget public agencies. Analysis of institutional and governance issues--including corruption-- is also insufficient. Better understanding of the political, cultural, and insti- tutional underpinnings of the budget process would enhance dialogue with recipient governments, strengthen risk assessments, and improve the development and implementation of action plans. Finally, the extent and speed of reform for assessment instruments may be subject to donor-driven constraints, including statutory obligations, operational and scheduling issues, requirements of management boards and oversight bodies, and institutional turf, cultures, and incentives. MAIN RECOMMENDATIONS Assessments and reforms of public expenditure management should be country-led but supported by donors and based on a coherent, integrated medium-term strategy. To that end, efforts are needed to streamline the coverage of assessment instruments (to avoid unnecessary duplication), enhance collaboration between donors, governments, and other stakehold- ers, provide more complete, accurate, timely assessments of fiduciary risk, and increase the development impact of assessment work. But reforms could go further--which is why this study also recommends developing a programmatic, modular framework for assessments. Streamlining coverage · The Bank, IMF, European Commission, and other development agen- cies should adjust their instruments to reduce unnecessary overlap. · Though the measures being considered to strengthen collaboration between CFAAs and Fiscal ROSCs should address the overlap between the two, it would be desirable for the Bank to first provide an authorita- tive clarification of the boundaries between CFAAs and PERs. · The Bank, IMF, European Commission, and other agencies should con- sider how to fill the gaps in coverage, whether by supplementing current Executive Summary xv instruments or developing new questionnaires and toolkits containing specific information. · The Bank should streamline its internal arrangements and operational practices for planning and conducting assessments, drawing on PEFA's 2002 report on integrating PERs, CFAAs, and CPARs. · The European Commission should develop its policy and methodology for carrying out compliance tests and annual audits to facilitate the inte- gration of such work with CFAAs, Fiscal ROSCs, HIPC AAPs, and other instruments. · Staff guidelines for assessments should be harmonized to facilitate inte- grated efforts and encourage collaboration. · Steps should be taken to make assessment reports more consistent and readable. For example, templates should be developed for PERs and CFAAs. Enhancing collaboration To increase their collaboration with donors, governments should be given complete access to staff guidelines, assessment work plans, schedules, and reports, and other information--for example, through the Country Analyt- ic Work Website being developed by the Bank and other development agen- cies. The Bank and IMF would continue to take the lead in conducting most assessments of public expenditure management and would make them avail- able as primary sources of information. Though governments and other donors would use these assessments, they would still be fully responsible for making their own judgments of fiduciary risks and development needs. · Cooperation, coordination, and collaboration between agencies should be enhanced--especially between the Bank and IMF, drawing on their recent joint paper on strengthening collaboration on public expenditure work (World Bank and IMF 2003). · Steps should be taken to increase the participation of bilateral donor agen- cies in discussions with governments on public expenditure management and in follow-up efforts to implement recommendations. Coordination with regional institutions and initiatives--such as regional multilateral development banks, the Strategic Partnership with Africa, and the New Partnership for Africa's Development--should also be strengthened. xvi Assessing and Reforming Public Financial Management · All assessment reports should include standardized executive summaries, providing core information to facilitate analysis, dissemination, and sharing of information between agencies, governments, and other stakeholders. · Agencies should consider establishing quality assurance procedures for donors participating in multidonor assessments, perhaps building on the internal procedures used by the World Bank's Quality Assurance Group. · Common definitions and terminology should be used in assessment work. Evaluating fiduciary risk and contributing to development goals · Governments and donors should agree on how to define fiduciary risk. · The role of assessments in evaluating fiduciary risk and contributing to long-term development goals should be clarified. · Consideration could be given to splitting the fiduciary and development aspects of assessments into separate processes and reports--and to cre- ating a more independent process for assessing risk, with some element of joint ownership by donors and external quality control and validation. This approach would allow donors to develop stronger partnerships with countries, reduce possible conflicts of interest, and make assess- ments more focused on development and capacity building. Increasing the development impact of assessments and reforms · New approaches to assessing and reforming public expenditure manage- ment should take full account of the views of governments (and other local stakeholders), because governments play the main role in design- ing and implementing reform strategies. · High-quality analysis and advice on public expenditure, procurement, and financial accountability requires seeing all aspects of public expen- diture management as parts of an interrelated whole--not separating such efforts based on the most efficient "division of labor." Thus, strengthening the substance and quality of analysis requires increasing collaboration within and between agencies. · Because governance, corruption, and cultural and institutional factors are often crucial to reforms of public expenditure management, better guidelines are needed on their definition, scope, and importance and on how to integrate them with assessments. Executive Summary xvii · More attention should be paid to how recommendations and action plans are followed up and how changes in public expenditure manage- ment can be monitored and evaluated. · Priority should be given to developing a robust, internationally accept- ed framework for benchmarking and measuring the performance of public expenditure management, building on the HIPC AAP approach. EC compliance tests might also provide useful data. Such work should be linked to the streamlining of assessment instruments. Developing a "programmatic," "modular" approach The recommendations above focus on improvements in the scope and application of current assessment instruments. Though those changes would be welcome, this study's ultimate aim is to foster more far-reaching reform of work on public expenditure management--developing an approach that is both "programmatic" and "modular." In this context it is encouraging that a working group, led by the World Bank and IMF and supported by the PEFA program, was established in summer 2003 to exam- ine ways of making such work more robust, relevant, and cost-effective. The new approach to public expenditure work proposed in this study has several key features, some of which are new. First, it would emphasize the importance of recipient governments participating in--and ideally, leading-- the design and implementation of public expenditure reforms. Second, it is a strategic approach, taking a medium- or long-term perspective that focuses on the steps that should be taken to move from a diagnostic assessment to the design and implementation of a sustainable program of reform and capacity building. Third, the approach would be "programmatic" in that it would be based not on the delivery of standardized products--such as PER, CFAA, or CPAR reports--but on a coordinated, sequenced program of diagnostic and capacity building work agreed between recipient governments and develop- ment agencies based on extensive dialogue. Such a work program would comprise a series of "modules," each dealing with a different aspect of the budget process. Fourth, the work program would be designed to fit with a country's Poverty Reduction Strategy Paper and the associated financial and technical support provided by donors. Finally, progress in improving public expenditure management would be measured by donors and recipient gov- ernments, using an agreed set of performance indicators. An important component of the programmatic approach could be a standardized overview of information obtained through assessments, xviii Assessing and Reforming Public Financial Management including performance indicators, similar to those used in HIPC AAPs, that could be updated periodically. Standardized assessment overviews would be useful to both donors and recipient governments, providing a concise analysis of public expenditure issues and assessments of fiduciary risk, highlighting areas requiring reform, and providing a tool for monitor- ing progress. Each overview could be subject to a joint quality review involving donors and the recipient government. STRUCTURE OF THE STUDY Following an introductory section and a discussion of fiduciary risk and accountability in public expenditure management, the study summarizes the scope and content of the main assessment instruments. It then describes the methodology and sources and presents the findings from the mapping of the instruments' coverage. The last section offers recommendations and identifies issues meriting further consideration. In addition, there are annexes describing the main assessment instruments in greater detail, dis- cussing good practices for measuring performance in public expenditure management (an OECD-DAC study), and presenting detailed results of the mapping exercise. Abbreviations and Acronyms CAS Country Assistance Strategy (World Bank) CFAA Country Financial Accountability Assessment (World Bank) CONTACT Country Assessment in Accountability and Transparency (UNDP) CPAR Country Procurement Assessment Report (World Bank) DFID Department for International Development (U.K.) EC European Commission EDF European Development Fund EU European Union Fiscal ROSC Report on the Observance of Standards and Codes of Fiscal Transparency (IMF) HIPC AAP Assessment and Action Plan for a Heavily Indebted Poor Country (World Bank­IMF) IFAC International Federation of Accountants IGR Institutional and Governance Review (World Bank) IIA Institute of Internal Auditors IMF International Monetary Fund INTOSAI International Organization of Supreme Audit Institutions OECD Organisation for Economic Co-operation and Development PEFA Public Expenditure and Financial Accountability Program xix xx Assessing and Reforming Public Financial Management PER Public Expenditure Review (World Bank) PREM Poverty Reduction and Economic Management Network (World Bank) PRSC Poverty Reduction Support Credit PRSP Poverty Reduction Strategy Paper QAG Quality Assurance Group (World Bank) SIDA Swedish International Development Authority SIGMA Support for Improvement in Governance and Management in Central and Eastern European Countries (EC and OECD) SPA Strategic Partnership with Africa UNDP United Nations Development Programme Assessments of Public Expenditure Management: Rationale and Context This study--carried out under the auspices of the Public Expendi- ture and Financial Accountability (PEFA) Program, a multi-donor partnership--is intended to help international development agencies generate more coordinated, effective instruments and procedures for assessing and strengthening public expenditure, procurement, and financial accountability systems in developing countries--especially countries that receive international aid for budget support.1 It ana- lyzes instruments and approaches used by the World Bank, Interna- tional Monetary Fund (IMF), European Commission (EC), U.K. Department for International Development (DFID), and other agencies, identifies gaps and overlaps by mapping each instrument's coverage of the various elements of expenditure management, and provides recommendations for strengthening and integrating the instruments. The main assessment instruments reviewed are World Bank Public Expenditure Reviews (PERs), Country Financial Accountability Assessments (CFAAs), and Country Procurement Assessment Reports (CPARs), IMF Reports on the Observance of Standards and Codes of Fiscal Transparency (Fiscal ROSCs), IMF­World Bank Public Expen- diture Tracking Assessments and Action Plans (AAPs) for Heavily Indebted Poor Countries (HIPCs), EC audits of public financial man- agement systems, and DFID assessments of fiduciary risk. 1 2 Assessing and Reforming Public Financial Management WHAT IS PUBLIC EXPENDITURE MANAGEMENT? Public expenditure management includes all the components of a country's budget process--both "upstream" (preparation and programming) and "downstream" (execution, accounting, control, reporting, monitoring and evaluation)--including the legal and organizational framework and arrangements for: · Forecasting revenues and expenditures. · Formulating medium-term expenditure frameworks. · Linking the budget to policymaking. · Preparing the budget. · Managing cash and monitoring expenditures. · Performing internal control and audits. · Accounting and reporting. · Procuring public goods and services and managing assets. · Assessing performance. · Conducting external audits. · Ensuring oversight by the legislature and other bodies. The broad objectives of public expenditure management are to achieve fiscal discipline, allocate resources to uses that reflect government policy priorities, and deliver public services efficiently and effectively. The terms public financial management and public expenditure management are often used interchangeably. But for the purposes of this study, public financial management has a narrower definition, involving issues related mainly to the downstream phase of the budget cycle, and the term public expenditure management is used more often. THIS STUDY'S PURPOSE AND OBJECTIVES The main concerns driving this study include the scope and coverage of the assessment instruments, adequacy of the institutional and governance analy- sis in the assessments, attention paid to fiduciary risk and its relationship Assessments of Public Expenditure Management: Rationale and Context 3 with development objectives, availability of questionnaires, checklists, and other tools to support assessment work, adequacy of review and quality con- trol procedures, and fit between the experience and skills of teams that con- duct assessments and the technical and institutional issues being assessed. A consultative approach was used to prepare the study, including with staff of the development agencies concerned--especially task leaders--and with government officials in some recipient countries. This approach was vital because balanced conclusions in this complex area require the informed judgments of those involved in actual assessments. To understand the context for both the PEFA program and this study, the next section reviews the evolution in approaches to assessing public expenditure, procurement, and financial accountability, protecting the integrity of financial resources (by containing fiduciary risk), and achieving development objectives. The study then provides a conceptual framework for assessing fiduciary risk, financial accountability, and public expenditure management, summarizes the coverage and content of various instruments that do so, describes available questionnaires and other tools, explains the study's methodology and information sources, presents findings from the technical mapping of the instruments' coverage, the review of staff guide- lines and sample reports, and interviews with experts from donor agencies and government officials, and offers recommendations and identifies issues meriting future attention. GENESIS OF ASSESSMENTS OF PUBLIC EXPENDITURE MANAGEMENT AND FINANCIAL ACCOUNTABILITY Until the late 1970s most external assistance to developing countries focused on individual projects, including efforts to increase financial integrity and aid effectiveness. Moreover, the literature on development made a clear distinction between development and nondevelopment spend- ing, with development spending usually identified as investment and non- development as current spending. This approach to aid corresponded to the "golden rule" of budgeting, which calls for all government borrowing to be used for investment and for current spending to be fully financed by domestic receipts. This approach was considered essential to assessing whether the debt service incurred by new investment would be more than offset by the increase in debt servicing capacity it made possible--both measured in foreign currency terms. 4 Assessing and Reforming Public Financial Management But over the past 25 years four developments have produced far-reach- ing changes in approaches to and perspectives on external assistance. First, it was recognized that all resources are fungible--especially convertible resources. Thus aid ostensibly earmarked for a well-prepared, well-super- vised donor project could result in government resources being used to finance another project about which the donor knew nothing. (The notion of resource fungibility was first formulated by de Vries 1967 and subse- quently elaborated by Schiavo-Campo and Singer 1970; Devarajan and Swaroop 2000 recently reviewed these arguments.) Moreover, safeguards to ensure the integrity of donor resources pro- vided to a project meant little in the absence of safeguards on the use of government resources effectively released by the aid. Thus the World Bank and other donors began to consider it necessary, when financing a large share of a country's public investment, to appraise the entire invest- ment portfolio rather than just aid-financed projects. The Bank's result- ing Public Investment Reviews were also useful to other development agencies in the context of their project-centered assistance. In addition, because Public Investment Reviews required identifiable public invest- ment portfolios, they led to the provision of technical assistance in for- mulating public investment programs--which quickly became standard in most developing countries.2 Second, in line with the global shift away from central planning and mounting evidence of the importance of social factors for development, the distinction between development and nondevelopment spending lost mean- ing. It became accepted that new teachers and new textbooks are just as cru- cial for development as new schools, and thus that current spending and investment form (or should form) an integrated whole. Hence, the World Bank's Public Investment Reviews expanded into Public Expenditure Reviews (PERs)--though this is often a misnomer because most such reviews cover only government spending, and usually only central government spending. (PERs are discussed later in this study along with other donor instruments used to assess public expenditure and financial accountability.) Third, the dominant influence of economic and social policies on proj- ect effectiveness came to be more widely understood. After the early 1980s it was no longer possible to believe that project-centered aid could be effec- tive for development if macroeconomic policies were badly flawed. Accord- ingly, untied lending conditional on policy reforms began to complement project assistance. This gradual shift to policy-based adjustment lending-- originally permissible only to finance foreign exchange gaps, then from the Assessments of Public Expenditure Management: Rationale and Context 5 mid-1990s justifiable as pure budget support--made increasingly clear the need to periodically assess the recipient government's budget policies and budget management. Yet for much of the 1980s most donors, still in the grip of Harrod- Domar thinking (which argues that growth is a function only of the amount of physical investment and has no relation to the quality of governance or public sector management), remained focused on aggregate spending, its compatibility with the macroeconomic framework, and its sectoral and eco- nomic allocations. Not until the early 1990s were the links between good public sector management and development sufficiently understood.3 It became increasingly accepted that effective development outcomes require not only sufficient resource transfers and sound macroeconomic and social policies, but also efficient spending. Efficient spending, in turn, depends on strong systems for budgeting, financial management, and accountability. The need for donors to get involved in budgeting, accounting, auditing, and control could no longer be ignored.4 Finally, the recognition from late 1996 of the destructive effects of cor- ruption--official and private, and highlighted emphatically by the East Asian crisis of 1997­99--gave new urgency to donor agencies' need to assure their constituencies that aid resources were not being diverted for private ends or misallocated to activities not conducive to promoting growth and reducing poverty. To that end the World Bank adopted an anti- corruption policy in 1997, the Asian Development Bank followed suit in 1998 (see http://www.adb.org), OECD countries negotiated an Anti- Bribery Convention in 1999, and other agencies placed corruption at the fore of their concerns. In addition, because lack of transparency had per- mitted deep-seated financial and governance problems to fester until they erupted in the 1997 East Asian crisis, the IMF developed Standards and Codes of Fiscal Transparency addressing similar issues (see IMF 2001). (IMF Fiscal ROSCs are examined in greater detail later in the study.) This evolution in development policies and practices has raised the pres- sure on governments and donors to understand the public expenditure man- agement and financial accountability environment in which all budget funds are used. Thus, donors have created instruments that share many objectives, processes, analytical methods, and development goals, but that are grounded in each agency's specific concerns. As noted, later sections of this study exam- ine these instruments and assess their coverage in detail. But underlying all these instruments are basic notions of fiduciary risk and accountability and concepts of public expenditure management--the subject of the next section. 6 Assessing and Reforming Public Financial Management NOTES 1. Strengthening such instruments is one of the two main activities of the Public Expenditure and Financial Accountability (PEFA) program; the other is supporting country assessments and reforms. PEFA is a joint pro- gram of the World Bank, European Commission, International Monetary Fund (IMF), development agencies from France, Norway, Switzerland, and the United Kingdom, and the Strategic Partnership with Africa (SPA). The program was established in December 2001 and functions through a secre- tariat based in World Bank headquarters under the guidance of a steering committee representing all the partner institutions. PEFA was established out of concern that the instruments used by different development agencies to assess public expenditure, procurement, and financial accountability are insufficiently integrated, impose unnecessarily high transaction and com- pliance costs on recipient governments, and may be on a divergent course. Moreover, with increasing aid for budget support--where the fungibility of the assistance precludes a direct link between the aid and desired out- comes--these instruments have become more important, especially in terms of strengthening financial accountability. 2. First-generation public investment programs suffered from a number of weaknesses, were often formalistic documents produced as wish lists for consultative groups or other donor meetings, and sometimes even pro- duced adverse budget outcomes over the long term. More recent second- generation public investment programs preserve the advantages of sound medium-term investment programming while avoiding the mistakes of ear- lier years (Schiavo-Campo and Tommasi 1999). 3. These links were first emphasized in World Bank (1989a) and subse- quently formalized in a policy paper (World Bank 1992) and progress report (World Bank 1994). Other international organizations issued similar policy statements (such as Asian Development Bank 1995), as did bilateral donor agencies. The general consensus is that good governance revolves around four pillars: accountability, transparency, rule of law, and popular participation. The World Bank's public sector and governance strategy was first discussed by its Board in 1991; a progress report was issued in 1994, and a revised strategy was issued in October 2000. 4. Budget support also avoids the suggestion of donor interference and micromanagement inherent in some forms of earmarked assistance. For example, the World Bank's first loan, in May 1947, was for $250 million and Assessments of Public Expenditure Management: Rationale and Context 7 went to France. France had difficulty accepting the Bank's unprecedented policy of specific certification for the use of every dollar of the loan and the negative pledge clause in the Bank's Articles of Agreement. French author- ities considered the requirements "a derogation of the dignity of [their] country." During the loan negotiations a Bank staff member noted that "the requirement for specific designation of the use of all the proceeds meant that every contract for purchase of equipment and materials must be sub- mitted and approved by the staff of the Bank against certified bills of the suppliers. We agree to station a staff member in Paris to facilitate the nec- essary approvals and the system which we set up has been followed by the Bank." See World Bank (1987) and the Bank's online archives at www.worldbank.org. Fiduciary Risk and Financial Accountability Efforts to increase accountability in the context of untied budget support call for appropriate definitions of the interrelated notions of risk and accountability--both to interpret the technical mapping of the instruments reviewed in this study and to underpin its eventual recommendations. Although these terms are widely used, their meaning is not always clearly defined or understood. The discussion in this section is not meant to suggest that the diagnos- tic instruments under review are intended only to assess public expenditure and financial accountability systems from the viewpoint of risk and accountability. Indeed, the instruments have much broader objectives, including: · Describing the rules (formal and informal) and operational procedures underpinning public expenditure management systems. · Assessing the strengths and weaknesses of these systems. · Offering recommendations for improving the systems' rules and proce- dures, taking into account a country's political, legal, and cultural tradi- tions and its capacity--and political willingness--to reform. · Supporting the development and implementation of country assistance and poverty reduction programs. 9 10 Assessing and Reforming Public Financial Management · Informing the sponsoring agencies and their development partners about fiduciary risk. RISK The notion of risk is inherently different and more ambiguous in public sec- tor activity, including development aid, than in private enterprise. In com- mercial lending the biggest risk is that loans will not be repaid. But the risk of nonpayment is inapplicable to development grants and very remote for soft loans--especially for international financial institutions that do not allow loan rescheduling. Risk is generally defined as "exposure to the chance of injury or loss" (Merriam-Webster 1997), and fiduciary as a "person or entity in a position of trust, obliged to act in the interest of another" (Walk- er 1980). How such "interest" is defined thus becomes the central question. A distinction is often made between fiduciary risk and development risk. Fiduciary risk refers to the possibility that funds provided will be misused or stolen. But in the context of development aid even a narrow conception of fiduciary risk must also include the possibility that actual expenditures will diverge from authorized expenditures (as reflected in the borrowing country's budget), whether because of misappropriation or misallocation. This is the definition of risk used in the World Bank's Country Financial Accountability Assessments (World Bank 2002b). To avoid fiduciary risk, the borrowing country's budget must be reasonably comprehensive, its fis- cal situation should be on a path to sustainability, and the borrower and the provider of funds must agree on how expenditures should be allocated. (All three conditions are also standard requirements of adjustment lending.) This definition of risk also has a strong governance underpinning. In rep- resentative governments no funds can be mobilized from citizens or spent except through official and public sanction by their elected representatives. Thus unauthorized discrepancies between budgeted and actual spending place a cloud over the legitimacy of the entire public expenditure apparatus. The above construction of fiduciary risk does not include an obligation to ensure "value for money" or any other efficiency objective. But notions of efficiency and effectiveness are integral to development assistance. Hence development risk adds to the risk of misappropriation and misallocation the risk that resources will be wasted as a result of inefficient institutions and organizational practices. This broader definition of risk is used, for example, by the U.K. DFID (though the agency still refers to it as fiduciary risk). Fiduciary Risk and Financial Accountability 11 Assessing performance--let alone quantifying it--is exceedingly complex and fraught with potential misinterpretations and misapplications. But DFID's implementation criteria for its "value for money" requirement are generic enough not to raise the risk of micromanagement or misplaced con- creteness. (For more details on the DFID approach, see annex 1.) Thus the fiduciary objective is met by reasonable assurance that aid money will not be stolen or used for guns instead of butter. The develop- ment objective requires, in addition, some indication that money will be spent efficiently and effectively. These objectives share a kinship with the three classic levels of public expenditure management: overall expenditure control, strategic allocation, and operational management (Campos and Pradhan 1996). Except in severely dysfunctional systems--where expenditure control is paramount--these three levels are interrelated. Improvements at one level facilitate, and are not sustainable without, improvements at the other two. For example, imposing a hard ceiling on sector spending during budget preparation without linking that ceiling to sector policies and programs will likely result in underfunding of economically valuable activities, because such activities tend to offer less potential for rent seeking. Conversely, efforts to strengthen the links between policies and budgets will fail if fiscal discipline is weak. Similarly, operational management cannot be improved without fiscal discipline and sound resource allocation--for which good management is crucial. Over time, fiduciary and development risks tend to merge, and lasting improvements in expenditure management and accountability also should increase operational effectiveness because they reinforce the links between policies and budgets and strengthen fiscal discipline. Moreover, if reducing poverty through faster growth and pro-poor measures is assumed to be the main objective of recipient governments (which requires some judgment about the quality of governance), the fiduciary and development obligations of development agencies are to both the providers and recipients of aid funds. Correspondingly, ownership becomes a two-sided concept, and partnership becomes the hallmark of effective financial accountability assessments. Still, a tension often exists between assessments of fiduciary risk and long-term development objectives and requirements. In principle, these are two sides of the same coin. Recipient governments should have a strong interest in precise assessments of the risks associated with their public expenditure management systems, as measured in terms of achieving aggre- gate fiscal discipline, efficient resource allocation, and efficient and effec- 12 Assessing and Reforming Public Financial Management tive service delivery. Indeed, such assessments can guide the design and implementation of public expenditure management reforms. But in practice the process through which such information is collected and analyzed is extremely important and can influence government per- ceptions and incentives. If a development agency appears to be setting the agenda and collecting and using information that a country perceives could be used against its interests--for example, in setting difficult condi- tions for a structural adjustment loan or technical assistance operation--it is much harder to internalize the diagnostic process in a country, ensure its ownership by the government, and promote transparent provision and exchange of information. This point has important implications for the structure and management of assessment tools, and these issues are dis- cussed later in this study. ACCOUNTABILITY Though accountability is at the core of good governance, the concept is inherently relative, requiring a specification of accountability to whom and for what. As noted, development agencies must be accountable to aid providers and share with aid recipients responsibility for achieving com- mon objectives. Thus, there must be a way to systematically assess such achievements. There is a panoply of possible results, from the immediate output of a specific activity to the broader outcomes of an overall program. In well- defined projects close to their final users (such as urban transport projects) the link between outputs and outcomes is clear and immediate enough to permit the use of output indicators as a proxy for outcomes. In adjustment lending and budget support this is not the case: to be defined meaningful- ly the results must be defined broadly, and should include both outcomes and process indicators. There is an "accountability tradeoff": accountabili- ty can be tight or broad, but not both (Schiavo-Campo 1999). Individuals can be held strictly accountable for narrow and specific results, but only loosely for broad results--achievement of which partly depends on factors beyond their control. Moreover, accountability has two elements: answerability and conse- quences. A systematic dialogue on the uses and results of expenditure can be valuable even in the absence of direct implications (provided the results are defined appropriately). As a rule, however, such a dialogue should be Fiduciary Risk and Financial Accountability 13 accompanied by the specification of relevant targets or benchmarks, with program managers facing consequences for their success or failure in achieving these targets. Because effective accountability usually requires appropriate bench- marks, it raises (directly or indirectly) the often misunderstood concept of conditionality. In Joseph Gold's (1981) classic definition, conditionality is nothing more than "a means for the appropriate use of resources." Because resources are never provided without reason or purpose, the relevant ques- tion is not whether conditions should exist. Rather, it is whether the condi- tions specified are relevant and sufficient--or unnecessarily burdensome, intrusive, or even counterproductive--and whether they are derived through genuine dialogue (essential for effective implementation of reforms with a significant institutional content)--or imposed from the out- side. Even when an external accountability assessment is intended to be purely advisory, it is generally understood by the aid agency and the recip- ient government that the assessment's conclusions will be taken into account in the future provision of assistance. So, in developing countries the operational issue is not whether public financial accountability is fully adequate. In most cases it is not. The opera- tional issue is whether accountability is being strengthened at a sufficient pace to warrant representing to aid providers and recipients that fiduciary and development objectives are more likely to be achieved, partly through the use of the aid. This in turn calls for measures to improve the legal and organizational framework for public expenditure management, systems and processes for expenditure programming and budget preparation and execu- tion, accounting, reporting, and audit, and administrative and financial man- agement capacity. (As discussed below, these are the five broad headings under which components of public expenditure management are grouped in this study for the technical mapping of the various assessment instruments.) OPERATIONAL IMPLICATIONS The preceding discussion has four main practical implications. First, it can be misleading to assess overall risk by adding up the individual weaknesses in a public expenditure management system. The three levels of public expenditure management are interrelated, as are the various parts of the administrative apparatus. Just as the balance of payments or fiscal accounts must be analyzed as an interrelated whole, the assessment of the risks asso- 14 Assessing and Reforming Public Financial Management ciated with untied budget support should be based on the entire system for public expenditure management. In some cases the overall risk may be smaller than the sum of individual risks; in other cases it may be larger. What is needed is judgment, not a mechanical exercise. Second, because development agencies have differing goals and objec- tives as regards their aid programs and development assistance, their views of risk differ as well--as do the objectives, coverage, and methodologies of their financial accountability instruments. This is a major reason for the duplication between the instruments used by different agencies. Third, if an instrument covers a broad range of issues, it is bound to be more diffuse than an instrument focused on a specific component. It is tau- tological to say, for example, that a procurement review is more narrowly targeted than a public expenditure review. Finally, a given instrument may cover many components of public expenditure management but some only superficially, while another instru- ment may have narrower coverage but greater depth. With these consider- ations in mind, the next section describes the key features of the main assessment instruments. The Main Instruments for Assessments Donors use six main instruments to assess public expenditure, procure- ment, and financial accountability: World Bank Public Expenditure Review (PERs), Country Financial Accountability Assessments (CFAAs), and Country Procurement Assessment Reports (CPARs),1 IMF Reports on the Observance of Standards and Codes of Fiscal Transparency (Fiscal ROSCs), World Bank­IMF Public Expenditure Tracking Assessments and Action Plans for Heavily Indebted Poor Countries (HIPC AAPs), and EC audits. In addition, the U.K. Department for International Development (DFID) is developing a new approach to assessing fiduciary risk based on a methodology similar to that of the HIPC AAPs.2 All these assessment instruments share common approaches and objec- tives, including describing and analyzing systems for public expenditure, procurement, and financial accountability, recommending improvements, increasing financial integrity, and improving budget outcomes. Assessments also foster another crucial outcome: increased interaction between devel- opment agencies and recipient governments (see box 1). Yet the instru- ments' emphasis, coverage, and approach vary considerably, reflecting the fact that they were developed by different agencies or different parts of the same agency. The instruments are summarized below and described in more detail in annex 1. (Because the DFID approach to assessing fiduciary risk is still being developed, it is not part of the mapping exercise--though it is discussed below and in annex 1.) 15 16 Assessing and Reforming Public Financial Management Box 1 The role of assessments in promoting dialogue between donors and recipient governments, and reform For development agencies, recipient governments, and other stakeholders, the process of interaction--the dialogue set in motion by assessments--is just as important as the specific report itself, whether a PER, CFAA, or Fis- cal ROSC. The reason is simple: the goal of any assessment is to facilitate the development of sustainable reforms that will both strengthen a coun- try's public expenditure system and achieve key policy objectives such as poverty reduction. Diagnostic reports provide valuable data, analyses, and recommenda- tions, and form the basis for formal discussions with recipient governments. But if an assessment is conducted well, most such discussions will already have occurred, and change will have been initiated through the assessment process. Moreover, dialogue between an assessment team and a recipient government provides an opportunity for ministries and agencies to com- municate with each other, uncover sources of common difficulties, and begin working together toward solutions that take account of the institu- tional and governance reasons for weak public expenditure management. Indeed, some analysts argue that triggering this internal dialogue is the most important output of an external assessment. Integration, then, should be understood more as integrating assessment processes than as delivering standardized reports on different aspects of public expenditure manage- ment or merely eliminating ex post inconsistencies between these reports. Interestingly, all the instruments reviewed in this study focus on the legal frameworks and organizational structures and processes for public expendi- ture management--describing the laws, regulations, and operational pro- cedures for budget formulation and budget execution. Though such infor- mation is useful and relevant, it is not clearly linked to aggregate fiscal dis- cipline, efficient resource allocation, and efficient and effective delivery of public services--the three basic public expenditure policy objectives defined by Campos and Pradhan (1996) that have become widely accepted in the international development community. Some questionnaires and toolkits follow the Campos-Pradhan approach (such as DFID 2001b), but The Main Instruments for Assessments 17 the main diagnostic instruments reviewed in this study have not been mod- ified to reflect it. Such changes would require significant redesign. WORLD BANK PUBLIC EXPENDITURE REVIEWS The PER--the oldest instrument reviewed in this study--is the World Bank's traditional tool for analyzing public expenditure.3 As noted, when the artificial distinction between development and nondevelopment expen- diture was abandoned and adjustment lending was introduced, investment reviews expanded to assess the overall government spending that budget support helped finance. Until the 1990s traditional PERs consisted mainly of an assessment of fis- cal trends, an analysis of resource allocations between and within sectors, various annexes dealing with expenditure policies and programs in the major sectors, and sometimes a review of the fiscal relationship between govern- ment and public enterprises. The efficiency and operational effectiveness of public expenditure (let alone corruption issues) were rarely if ever addressed.4 Thus, it is not surprising that ostensibly sensible recommenda- tions in early PERs--say, to increase spending in priority social sectors, or to accelerate investment disbursements--were operationally hollow. But in recent years, aided by the spotlight placed on corruption and pub- lic mismanagement by the East Asian crisis, PERs have increasingly added budget systems, implementation, and capacity building to their traditional areas of review.5 The main emphasis is on the upstream phases of expendi- ture management: medium-term expenditure programming, annual budg- et preparation, and legislative approval. Recent years have also seen the emergence in a few countires of govern- ment-led PERs, which are a continuous process rather than a one-off review, often built around the annual budget cycle. The government of Tanzania, for example, owns and leads the country's diagnostic program for public expenditure management, the centerpiece of which is the PER. This program is closely integrated with the budget cycle and involves a broad range of stakeholders and close participation by donors. As a result the PER process has evolved from providing an external evaluation of budget man- agement to supporting the development of a multiyear expenditure frame- work. Tanzania's new approach has also improved donor coordination by ensuring that aid is consistent with budget objectives and priorities and increasingly integrated with the budget. 18 Assessing and Reforming Public Financial Management Tanzania's program for evaluating and reforming public expenditure management is designed to share donors' thematic and sector-specific expe- rience with participating stakeholders, improve resource use by undertaking technical studies of budget issues, raise the profile of budget issues, and sup- port donors in shifting the focus of aid from project to budget support. Sev- eral other countries that depend on aid and have limited capacity--such as Ethiopia and Uganda--have also been developing similar participatory, col- laborative approaches to work on public expenditure management. WORLD BANK COUNTRY FINANCIAL ACCOUNTABILITY ASSESSMENTS The CFAA is the Bank's main diagnostic tool for public financial manage- ment and accountability. It is designed to provide information about the financial management environment in which Bank (and other donor) funds may be disbursed. Thus it helps ensure that donor funds are used properly. The CFAA is not an audit that tracks individual spending, nor does it pro- vide a pass/fail assessment of a country's financial management system or define minimum standards for system capabilities and performance. CFAAs do, however, provide recommendations and action plans. Public financial accountability can be interpreted as covering the entire process of resource mobilization, allocation, use, and controls, as well as the interactions between the executive and legislative branches and civil socie- ty on fiscal and budget issues. Recognizing the need to set reasonable boundaries, CFAAs focus on describing and analyzing downstream finan- cial management and expenditure controls, including expenditure monitor- ing, accounting and financial reporting, internal controls, internal and external auditing, and ex post legislative review. As noted, the upstream phases of public expenditure management are generally the domain of PERs, as are issues of expenditure efficiency and effectiveness. Nevertheless, measures of fiduciary risk include the deviations between actual and budgeted expenditures that arise in many countries and the incomplete budget scrutiny that results from the use of extrabudgetary funds and other off-budget expenditures. Thus a good CFAA cannot entirely neglect the budget preparation process--no more than a good PER can neglect major problems in budget execution, accounting, audit- ing, and financial controls. In practice, this potential overlap tends to be The Main Instruments for Assessments 19 greatest in the area of budget execution. (As argued later in the study, the solution does not lie in better defining the scope, definitions, and bound- aries of the two instruments, but in better integrating them.) CFAAs--and the Bank's other instruments for fiduciary economic and sec- tor work--have become more important in recent years. The Bank requires that recent CFAAs be available for all its major borrowers by the end of fis- cal 2004. While a CFAA is not a formal prerequisite for adjustment lending, such lending does require an ex ante fiduciary assessment (such as a CFAA, CPAR, or PER) of a country's public expenditure, procurement, and finan- cial management systems (World Bank 2003d).6 The assessment is to be done in cooperation with the government and nongovernment stakeholders; part- nerships with other development agencies are also encouraged. But the pos- sible tension between country ownership and the role of CFAAs, CPARs, and PERs in assessing fiduciary risk has yet to be resolved. WORLD BANK COUNTRY PROCUREMENT ASSESSMENT REPORTS The World Bank introduced CPARs in the mid-1980s to determine whether the procurement rules and practices of borrowing governments conform with Bank requirements. CPARs are the most focused of Bank instruments for fiduciary economic and sector work, though their empha- sis has evolved from assessing procurement arrangements for Bank- financed projects to assessing national and subnational procurement laws, regulations, and procedures and encouraging dialogue with recipient gov- ernments (the current CPAR guidelines are described in World Bank 2002c). A CPAR is designed to: · Provide a comprehensive analysis of a country's procurement system-- including legislation, organizational responsibilities, control and over- sight, and procedures and practices. · Assess institutional, organizational, and other risks associated with pro- curement--including identifying procurement practices that are not acceptable in Bank-financed projects. · Develop a prioritized action plan for institutional improvements. · Assess the domestic private sector's participation in public procurement and commercial practices related to public procurement. 20 Assessing and Reforming Public Financial Management IMF REPORTS ON THE OBSERVANCE OF STANDARDS AND CODES OF FISCAL TRANSPARENCY Fiscal ROSCs originated from the IMF's Code of Good Practices on Fiscal Transparency, adopted in 1998 and revised in 2001. The IMF adopted this code in response to the East Asian and other financial crises of the late 1990s in the belief that focusing on fiscal transparency would provide strong market pressure for countries to improve public expenditure man- agement. The code is intended to apply to all IMF member countries, but country participation in a Fiscal ROSC is voluntary. Drawing on the four key requirements for fiscal transparency--clear roles and responsibilities, full provision of information, open budget prepa- ration and execution, and assurances of integrity--a Fiscal ROSC: · Describes country systems and procedures relative to the good practices identified in the fiscal transparency code and manual. · Provides an assessment by IMF staff that prioritizes recommendations for possible improvements. Fiscal ROSCs are formally limited to assessing fiscal transparency, although in practice other public expenditure management issues are also reviewed to a limited extent. They address the efficiency and effectiveness of expenditure processes and outcomes primarily from the viewpoint that lack of transparency contributes to weaknesses or difficulties in assessing these aspects. The exercise is carefully targeted and conducted in accor- dance with detailed instructions. It begins with the government filling out a standard questionnaire and concludes with the publication of the report (if the government agrees), which is based on a standard template. Fiscal ROSCs are prepared as background for staff reports on the Article IV consultations that the IMF engages in with member countries as part of its regular surveillance, and are submitted to the IMF's Executive Board. Fiscal ROSCs inform the Board on the extent to which the fiscal transparency code is being observed and on progress over time. The reports also identify institu- tional constraints and guide IMF country programs and technical assistance. Fiscal ROSCs are more closely linked to international surveillance of fiscal transparency and accountability than are the World Bank's three main instru- ments for fiduciary economic and sector work. But the institutional scope and coverage of Fiscal ROSCs are similar to those of the Bank instruments, particu- larly CFAAs. Recognizing this, the IMF is taking steps to improve coordination The Main Instruments for Assessments 21 with Bank teams preparing CFAAs, and the two institutions are sharing informa- tion on mission schedules, undertaking occasional joint missions, exchanging draft reports for comments, and exploring the development of a joint question- naire and a joint database containing information on fiscal institutions. WORLD BANK­IMF PUBLIC EXPENDITURE TRACKING ASSESSMENTS AND ACTION PLANS FOR HEAVILY INDEBTED POOR COUNTRIES HIPC AAPs were developed in spring 2000, stemming from the Bank and IMF's desire to ensure that HIPCs use resources freed by debt relief for poverty-reducing public expenditures based on full additionality--meaning that, because past aid levels would be maintained, debt relief would translate into additional real resources. Achieving that goal requires that HIPCs track poverty-reducing expenditures, which requires a public expenditure manage- ment system capable of planning, executing, and reporting on spending. HIPC AAPs are innovative because, instead of assessing every aspect of pub- lic expenditure management, they take a systems approach: identifying impor- tant indicators that reflect a system's overall capability and performance. HIPC AAPs started with a questionnaire on public expenditure man- agement that resulted in the identification of 15 benchmarks. In fall 2000, joint Bank-IMF teams conducted desk reviews of the 25 HIPCs that had reached their decision points or were expected to do so soon. A year later 24 AAPs were finalized, and in March 2002 the results were presented to the Bank and IMF boards. Missions for AAPs were often combined with ongoing work by country teams on, for example, Fiscal ROSCs, PERs, and CFAAs (World Bank and IMF 2002). In spring 2003 the Bank and IMF boards reviewed a paper containing updates on progress in implementing HIPC action plans (World Bank 2003b). The updates, based on the 15 benchmarks for public expenditure management identified earlier, covered 21 HIPCs and were prepared by Bank and IMF staff in consultation with country authorities using stan- dardized formats. The updates indicated that many HIPCs had improved their public expenditure systems and that a growing number were report- ing on poverty-reducing spending. For example, more than three-quarters of the measures in HIPC action plans had been or were being implement- ed. Moreover, measures from the action plans had been incorporated in Poverty Reduction Strategy Papers (PRSPs), IMF-supported programs, and Bank adjustment operations. Still, the quality and performance of pub- 22 Assessing and Reforming Public Financial Management lic expenditure management systems, as measured by the 15 benchmarks, did not necessarily improve much. In late 2004 the Bank and IMF will deliver a thorough reassessment of HIPCs to their boards, following anoth- er round of comprehensive assessments starting in fall 2003. EC AUDITS Until recently the European Commission's assessment instrument consist- ed of periodic ex post audits of whether its structural adjustment resources were used in line with the financing and implementation agreements signed with recipient governments. But with the evolution of EC support for structural adjustment, the purpose of and approach to audits have also changed. This methodology has developed gradually, building on the prin- ciples and methodology of the United Nations Development Programme's CONTACT guidelines and on lessons from previous audits, and has not been defined in a formal document.7 Since 1992 EC audits have been carried out in some 32 countries. For Gen- eral and Sector Import Support Operations these audits focused on the legali- ty and regularity of expenditures. For targeted budget support the audits were adjusted to recognize that counterpart funding became subject to national budget rules and procedures. Hence a more comprehensive approach to audits was developed that focused on an assessment of the legal framework and pro- cedures for budget execution--including procurement and financial accounta- bility and control--verified through sample financial audits. In addition, mate- riality checks were introduced to assess the value for money of expenditures. Recognizing that the segregation of counterpart funds undermines effec- tive budgeting in recipient countries (as well as the long disbursement delays engendered by that approach), the European Commission has proposed a new approach based on ex ante assessments and traditional ex post audits (EC 2002). General budget support will be programmed over multiple years and released annually in both fixed tranches (based on the recipient coun- try's conformity with IMF conditions for macroeconomic management) and variable tranches (based on agreed indicators for social and public expendi- ture management). The proposed ex ante assessments would be based on: · Reviews of diagnostic work on public expenditure management, drawn mainly from non-EC sources. · Information from previous EC audits. The Main Instruments for Assessments 23 · EC reviews of government and donor action plans for improving public expenditure management. · Analyses of gaps identified in existing diagnostic work that are to be addressed by follow-up work. Such follow-up work might include, for example, sector-specific sample financial audits used to establish indica- tors to measure progress in improving public financial management. Though the new approach has broad coverage, the assessments will largely rely on recent diagnostic work by the Bank, IMF, and other agen- cies--complemented and validated by EC "compliance tests," which use audit techniques to provide reasonable assurance that public expenditure management systems and procedures are implemented consistently, in line with relevant rules and regulations. (However, the European Commission has not produced an adequate description of compliance tests or guidelines for staff conducting them; see PEFA 2003). The proposed new approach emphasizes strengthening public expenditure and financial accountability and measuring progress using relevant performance indicators. DFID ASSESSMENTS OF FIDUCIARY RISK The DFID approach to measuring and managing fiduciary risk, which is still in an experimental phase, is largely based on reports and analysis carried out by other agencies. Its goal is to ascertain whether it is reasonable to expect that DFID resources will be used for their intended purposes, accounted for properly, and represent value for money. The DFID approach is similar to that of HIPC AAPs in that it is based on a relatively small number of indi- cators focused on different aspects of public expenditure management. The DFID assessments generally rely on World Bank and IMF infor- mation, including PERs, CFAAs, and other documents. In addition, DFID has collaborated with the Bank in carrying out some CFAAs. Whatever the source of information, a fiduciary risk assessment must be completed before DFID provides budget support (DFID 2002b). QUESTIONNAIRES, CHECKLISTS, AND OTHER TOOLS Development agencies have developed a variety of questionnaires, check- lists, and other tools for teams undertaking assessments. Fiscal ROSCs and 24 Assessing and Reforming Public Financial Management CPARs have detailed, well-structured questionnaires. HIPC AAPs also use a questionnaire. Though CFAA guidelines included a detailed question- naire in the past, recently updated guidelines include only a few key ques- tions (World Bank 2002b). The World Bank's Africa Region, however, has developed a detailed CFAA checklist that covers most aspects of budget execution, accounting, audit, and control. The Bank has also developed other tools to analyze public expenditure management and the impact of expenditures, including Public Expenditure Tracking Surveys (PETS), benefits incidence analysis, and anticorruption surveys. Also deserving mention are the questionnaires and checklists linked to the Bank's Public Expenditure Management (PEM) Core Diagnostic, UNDP's CONTACT guidelines, and SIGMA's baseline assessments. (See also the questionnaires that appear as annexes in Allen and Tommasi 2001 and Schiavo-Campo and Tommasi 1999.) The PEM Core Diagnostic was developed to assist in the preparation of PERs, though it was never for- mally tied to them.8 Because it was developed immediately after the Fiscal ROSC, it includes most of the components covered by the Fiscal ROSC in terms of the legal and organizational framework for expenditure program- ming and budget preparation. The PEM Core Diagnostic does not cover budget execution, accounting, or auditing, and directs users to CFAA and CPAR guidelines and questionnaires for information on these activities. (Recently updated CFAA guidelines call the PEM Core Diagnostic a "com- plementary toolkit"--see World Bank 2003a.) UNDP's CONTACT guidelines provide comprehensive checklists cov- ering most aspects of budget preparation and execution, while SIGMA's baseline assessments comprise questionnaires focused on the internal con- trol, procurement, financial reporting, and audit aspects of public expendi- ture management. In addition, the PEFA program is preparing question- naires on the management of government records in areas such as financial management and personnel issues, filling an important gap in the coverage of diagnostic tools. Diagnostic instruments differ in the level of detail they seek through their questionnaires and checklists (table 1). Moreover, some question- naires allow for multiple choice answers or are scaled in some way, while others require only yes or no answers. The Main Instruments for Assessments 25 Table 1: Features of questionnaires and checklists used by various assessment instruments Number of questions Multiple Instrument or indicators choice? World Bank PEM Core Diagnostic Summary Version 60 No Master Version 252 No World Bank CFAA 105 Yes World Bank CPAR 235 No IMF Code of Fiscal Transparency 84 Yes Bank-IMF HIPC Public Expenditure Tracking AAP 15 Yes DFID Fiduciary Risk Assessment 16 Yes UNDP CONTACT (all modules) 605 No Source: SIDA 2002. NOTES 1. Economic and sector work is the Bank's primary country-based analyti- cal and advisory work, and includes these three instruments. For Bank policies on economic and sector work, see the Operational Manual and Policies O.P. B.P.2.11 (Country Assistance Strategy), O.P. 8.40 (Technical Assistance), and O.D. 8.60 (Adjustment Lending Policy) at http://wbln0011.worldbank.org/ Institutional/Manuals. 2. This study does not substantively review similar instruments devel- oped by other international organizations, government self-assessments (which do not follow standardized procedures), or donor analyses per- formed as part of technical assistance operations and aid proposals (such as IMF and World Bank support for treasury systems, debt management, and tax administration). Instruments developed by other organizations (and described in annex 1) include the United Nations Development Pro- gramme's Country Assessment on Accountability and Transparency (CONTACT) and by Support for Improvement in Governance and Man- agement (SIGMA), a joint program of the European Commission and Organisation for Economic Co-operation and Development, used to assess 26 Assessing and Reforming Public Financial Management the public expenditure management and institutional capacity of Central and Eastern European countries applying for EU membership. In addition, the Asian Development Bank has developed an instrument--called Finan- cial Management and Governance Issues in Selected Developing Member Countries--covering accounting, auditing, and other financial management issues, which has been applied in several countries; see http://www.adb.org/ Documents/Books/Financial_Mgt/Selected_ DMCs/default.asp. 3. Much of the information in this section was drawn from World Bank (2001). 4. The first PER with an explicit institutional dimension was World Bank (1989b). But by the late 1990s fewer than a quarter of recent PERs had paid serious attention to budget management or institutions; see World Bank (1999). 5. The new focus has not always been accompanied by a commensurate expansion in the technical skills required for such analysis. In some cases the previous neglect of expenditure management has given way to the far worse outcome of superficial analysis and bad recommendations--usually in a mechanical imitation of complex practices introduced in a handful of industrial countries. 6. A proposal has been made to revise the Bank's Operational Policy, OP/BP 8.60, such that "as a precondition for all development policy sup- port lending, an ex-ante fiduciary assessment of the country's public expen- diture management, procurement, and financial accountability systems is required. Key findings of the fiduciary assessment would be reported in the CAS [Country Assistance Strategy]" (World Bank 2002d, p. 28). 7. The EC approach to audits has been strongly criticized by the Euro- pean Parliament and by the EC internal audit service for failing to ade- quately assess the weaknesses and risks of the public financial management systems for which the Commission provides aid; see EC (2001a, b). 8. The PEM Core Diagnostic has a 60-question Summary Version and a 252-question Master Version. Both can be found at http://www-wbweb. worldbank.org/prem/prmps/expenditure/petoolkit.htm. Methodology and Main Findings METHODOLOGY AND SOURCES The study has "mapped" each instrument's coverage of the various dimensions of public expenditure management, in order to reveal the core issues and iden- tify unnecessary duplications or gaps in coverage. Describing and assessing public expenditure management systems generally calls for reviewing the legal and organizational framework and the stages, processes, and procedures of budgeting. In turn, the budgeting cycle is conventionally understood as com- prising three parts: expenditure programming and budget preparation; budg- et execution and monitoring; and accounting, reporting, auditing, and exter- nal oversight. In addition, the state of administrative and financial manage- ment capacity at all stages of the public expenditure management cycle is fun- damental to any assessment of financial systems and the prospects for their improvement. The mapping exercise breaks down these broad areas into 15 major components and 94 sub-components, as shown in box 2. The major instruments described below have been "mapped" according to the extent of their coverage of the 94 subcomponents of the public expen- diture management cycle. Annex 3 shows the detailed map, with an explana- tory note. A summary map, covering the 15 major components, is in figure 1 below, in which the extent to which each instrument covers each of the public expenditure management components is shown as "high," "medium," or "low or none," illustrated as black, gray and white, respectively. 27 28 Assessing and Reforming Public Financial Management Box 2 Components of public expenditure management Legal and organizational framework Legal framework for expenditure management · Constitutional requirements · Legal framework for budget and fiscal policy · Fiscal management roles of the executive, legislative, and judicial branches Intergovernmental fiscal relations · Allocation of responsibilities between different levels of government · Expenditure transfers to subnational governments · Revenue authority and borrowing rights of subnational governments Relations between government and nongovernment entities · Scope of government fiscal targets and consistency with government finance statistics · Central bank independence · Clarity of distinction between government and public enterprises · Clarity of reporting of government equity holdings · Clarity and openness of formal regulation of private sector Government structure · Type of government (presidential, parliamentary, and so on) · Cabinet arrangements (powers of prime minister, ministry of finance, and line ministries) · Organizational structure of ministries and agencies with financial responsibilities · Arrangements for civil service management · Second-tier organizations (directorates, agencies, autonomous bodies) Expenditure programming and budget preparation Budget coverage · Budget classification and consistency with government finance statistics · Extrabudgetary funds and earmarked revenues · Quasi-fiscal activities · Contingent liabilities Methodology and Main Findings 29 · Tax expenditures · User fees · Donor funding · Transfers to and from public enterprises Expenditure analysis · Fiscal sustainability · Composition of expenditures · Analysis of mandatory and discretionary spending · Review of public investment programs · Intersectoral analysis · Intrasectoral analysis · Efficiency and effectiveness of expenditure programs · Expenditure incidence and poverty impact · Analysis of deviations of actual from budgeted expenditures · Assessment of payment arrears Fiscal framework and expenditure programming · Macroeconomic framework and fiscal outlook · Revenue projections · Donor funding projections · Fiscal scenarios and sensitivity analysis · Fiscal risks and contingent liabilities · Independent scrutiny of macroeconomic models and assumptions · Setting of aggregate revenue, spending, and deficit targets Budget preparation · Cabinet-level setting of policy priorities and expenditure ceilings · Ministry-level setting of priorities and allocation of resources · Level of detail in budget allocations (flexibility of ministerial spending) · Expenditure classification (line item, program, and so on) · Forward costs of ongoing programs · Cost estimates for proposed programs · Medium-term expenditure estimates · Clarity of program objectives · Process for preparing the draft budget · Integration of capital and recurrent budgets · Civil society participation in budget preparation (Box ontinues on the following page.) 30 Assessing and Reforming Public Financial Management Box 2 (continued) · Feedback from program review and evaluation · Approval of draft budget by the legislature Budget execution Treasury systems, cash management, and expenditure monitoring · Laws and regulations on treasury and cash management · Organization of the treasury function · Cash planning and forecasting · Banking and accounting arrangements · Rules and procedures for payments and cash allocations · Flexibility for budget managers (virement rules, carryovers, and so on) · Commitment control · Verification of receipt of goods and services · Monitoring and management of payables and arrears · Payroll monitoring and management · Reconciliation of fiscal and bank information Public procurement and management of physical assets · Procurement laws and regulations · Organization of procurement functions within government · Openness of procurement processes and procedures · Tendering procedures · Documentation and information systems · Control and complaint review procedures · Management of physical assets Internal control and audit · Internal control regulations, organization, and procedures · Internal audit regulations, organization, and procedures Accounting, reporting, and external audit Accounting, reporting, and records management · Accounting policies and standards · Accounting processes and responsibilities · Government financial management information systems · Internal reporting Methodology and Main Findings 31 · Scope and coverage of external financial reports · Timeliness and quality of external financial reports · Records management systems Debt and aid management · Debt and aid management laws and regulations · Management, control, and reporting of government debt · Management, control, and reporting of financial assets · Management, control, and reporting of aid External audit · Legal framework for external audit · Independence of supreme audit institution · Jurisdiction of supreme audit institution · Audit standards · Timeliness and quality of audit reports · Sanctions for irregularity · Legislative review of audit reports · Follow-up on audit recommendations Administrative and financial management capacity Personnel quality, capacity, incentives, and management The methodology gives an overview of the coverage and is thus useful in signaling possible gaps or overlaps in the scope and coverage of the instru- ments. However, as noted earlier, for each of the 94 subcomponents a binary choice must made ­ coverage or no coverage. From such choice little can be inferred regarding the depth or relevance of the analysis ­ let alone its qual- ity. Therefore, the study supplemented the mapping exercise by collecting information from reports, reviews, and other sources from a number of coun- tries where the instruments have been applied.1 Most important for judg- ments about the quality of the assessment, interviews were conducted with staff from the World Bank, IMF and other agencies and, to a more limited degree, government officials. Although still largely anecdotal and partial, this information complements the mapping of coverage and underpins the find- EC AUDIT HIPC AAP Fiscal ROSC AR CP AA management CF e PER expenditur public of components monitoring entities e main assets the nment of expenditur management physical age of and nongover ogramming and coverage management cover management pr e and e ds coverage ecorr elationsr incentives,, substantial nment management coverage and or instruments' expenditur expenditur management, moderate e no fiscal gover and audit or for and cash and capacity, or structur analysis ement ol eporting,r management coverage Complete Partial Little nmental of = = = Assessment between e eparation aid quality 1: coverage pr systems, ocur contr audit e framework nment framework pr nal and nal Level Figur Component Legal Intergover Relations Gover Budget Expenditur Fiscal Budget easuryrT Public Inter Accounting, Debt Exter Personnel Key: 32 Methodology and Main Findings 33 ings and recommendations advanced later in this study. A major qualification, however, should be advanced at the outset. Nowhere in the above list of public expenditure management dimen- sions are the words "corruption" and "governance" mentioned. This is not an oversight. The risk of corruption, i.e., the misuse of public power for private gain, pervades every aspect of the public expenditure management cycle and cannot be handled as an item on a checklist. Recalling the earlier discussion on risk, corruption affects the very first component of fiduciary risk, namely, that the money provided for approved budgetary purposes ends up instead in private pockets. If the funds disappear, analyses of their allocation and efficient use are instantly irrelevant. Although a substantive discussion of corruption is outside the scope of this study, public expendi- 2 ture (and especially public investment), is known to offer some of the best opportunities for corruption, and not only in the procurement phase. As in other areas, while enforcement through internal control and other means is helpful, the most effective ways to combat corruption is to reduce the opportunities and incentives for it ­ normally arising from over complex and opaque regulations ­ and to strengthen transparency and oversight mechanisms. The struggle against corruption must therefore be conducted by closing off corruption opportunities, through concrete improvements in specific aspects and procedures of public expenditure management. Most concrete improvements in one or another public expenditure management component tend to have, among other benefits, a positive impact in terms of reducing corruption. This explains why, although corruption does not appear as a separate item for assessment, the issue of financial integrity underpins almost every one of the components listed earlier. Ideally, diagnostic reviews should go through each phase of the budget cycle to identify areas that are vulnerable to corruption, and recommend ways in which corruption risks can be addressed in the area concerned. For example, during budget formulation, hard budget constraints can be evad- ed by the use of Build-Operate-Transfer (BOT) mechanisms, which in turn can easily become a vehicle for hiding illicit transactions. During budget execution, corruption risks can be reduced by improving the procedures for award of large procurement contracts. And so on. Also conspicuous for their absence from the mapping are the four pillars of governance ­ accountability, transparency, rule of law and participation. As in the case of corruption, however, this absence is only apparent. The four pillars of governance should be seen as guiding principles for the pub- lic expenditure management assessment, providing a prism through which 34 Assessing and Reforming Public Financial Management to scrutinize each part of the system. At each stage of the expenditure man- agement cycle, one can go a long way toward building governance consid- erations into the "technical" assessment by asking the question of the extent to which the systems and procedures are transparent, and whether they provide for genuine accountability, predictability through application of the rule of law, and appropriate participation. It is especially important in this largely qualitative exercise to recognize the institutional reality that infor- mal rather than formal rules may be the determining influence on people's behavior and the effectiveness of the system. This recognition, in turn, leads directly to facing the key issue of incentive, and how to gradually pro- vide stronger incentives for improved behavior, and disincentives for cor- ruption and inefficiency. (Beyond this general principle, the extent to which institutional factors are taken into account in the guidelines and assess- ments of public expenditure management is discussed later). COVERAGE OF THE INSTRUMENTS Figure 1 summarizes the coverage of the main instruments (the detailed map of all 94 components of public expenditure management is in annex 2). Even a cursory look at figure 1 shows that there is both substantial "spe- cialization" in the various instruments, in the sense of a particular focus on certain issues or aspects of public expenditure management as well as sig- nificant overlap in their coverage. Recalling that the high, medium, and low (black, gray, and white) coverage of each of the 15 major components in the map has been measured according to the frequency of coverage of the sub- components, the main observations are as follows: · The scope and range of PERs vary widely but generally focus on tradi- tional areas of public expenditure policy and analysis--both at the macro level and in key sectors such as health and education. Many PERs pro- vide substantial coverage of intergovernmental fiscal relations, govern- ment structure, budget coverage, fiscal framework and expenditure pro- gramming budget preparation, and treasury systems, cash management, and expenditure monitoring. · CFAAs provide substantial coverage of budget coverage, budget prepa- ration, treasury, cash management, and expenditure monitoring, internal control and internal audit, accounting, reporting, and records manage- ment, and external audit. Methodology and Main Findings 35 · Consistent with its focused objectives, CPAR coverage of nonprocure- ment aspects of public expenditure management is limited to occasional mention of relevant issues of budget preparation and execution, particu- larly internal control and audit. · Fiscal ROSCs cover mainly the legal framework for expenditure man- agement, relations between government and nongovernment entities, budget coverage, fiscal framework and expenditure programming, inter- nal control and audit, and external audit (mainly from a fiscal trans- parency perspective). · EC audits focus on downstream issues such as treasury systems, cash management, and expenditure monitoring, procurement, and account- ing and reporting. · None of the instruments provides more than limited coverage of admin- istrative and financial management capacity (personnel quality, capacity, incentives, and management). In countries where the World Bank is preparing or providing an adjust- ment loan, the functional equivalent of a PER is integral to the dialogue under the loan--but a formal PER is normally not done. Because expendi- ture reviews conducted in the context of adjustment lending tend to give greater weight to nuts-and-bolts problems of expenditure management than do formal PERs, their coverage of budget execution and financial con- trol issues is greater, as is the overall scope of the instrument. With the exception of PERs, the assessment instruments provide the most extensive coverage of budget execution, accounting, audit, and con- trol. But within these areas there are substantial variations in coverage and approaches among PERs, CFAAs, Fiscal ROSCs, and EC audits--while seeming overlaps in coverage often mean little, and sometimes reflect dif- ferent objectives and approaches. Moreover, figure 1 does not reflect dif- ferences in the quality and depth of analysis. SIMILARITIES Just as the depth and breadth of coverage vary, so do the procedures for carry- ing out assessments--including the intensity of discussions with recipient countries. Still, all the instruments share common objectives and features. They all examine aspects of public expenditure and financial accountability systems, 36 Assessing and Reforming Public Financial Management diagnose strengths, weaknesses, and risks, and recommend action plans for improving functions and procedures, with monitorable benchmarks or indica- tors identified to gauge and guide such efforts. The recommendations of assessment reports have become more impor- tant in recent years, and in most cases feed (directly or indirectly) into the development agency's country assistance strategy or its equivalent. These recommendations often also influence lending and other donor support, affecting decisions on the type and amount of assistance and sometimes resulting in specific projects or technical assistance to support improve- ments in public financial management. For example, EC audits of the use of EC program aid provide useful information about the quality of underlying public expenditure management systems and procedures for control, procurement, and the like--though, for reasons noted above, the European Commission is moving to a new approach that sharpens its assessment work by combining compliance test- ing (a modified form of audit) and performance measurement. Although the IMF's Fiscal ROSCs are in principle voluntary, acceptance of their analysis and recommendations increases the probability that the governments assessed will receive IMF support and influences the content of the stabi- lization programs the reports are designed to support. Moreover, multilat- eral development banks and other development agencies pay serious atten- tion to the findings of Fiscal ROSCs when designing their aid programs. Similarly, a PER can be a self-standing diagnostic study, but--whether as a formal product or not--is considered essential for a World Bank adjustment loan. As noted, in the absence of a recent formal PER, its functional equiva- lent is carried out during the preparation of an adjustment loan. Rarely does a Bank adjustment loan or credit not include a substantial component for improving public expenditure programming and management. Similarly, CFAAs and CPARs are required in all countries where the Bank is consider- ing adjustment support. It would be disingenuous to pretend that the Bank would not (rightly) insist on implementation of some CFAA or CPAR rec- ommendations as part of the reforms supported by an adjustment loan. Moreover, many bilateral agencies have always paid attention to Bank PERs, and appear just as willing to rely on the findings of CFAAs and CPARs. The assessment instruments also share several operational approaches and processes. First, they all emphasize the importance of ensuring that recipient governments own and participate in the analysis. PERs, CFAAs, and CPARs encourage government participation throughout the process, Methodology and Main Findings 37 Fiscal ROSCs begin with governments filling out a questionnaire, and DFID assessments of fiduciary risk strive to agree with governments on the main policy issues to be addressed and the basis for measuring and moni- toring improvements. Second, to reduce transaction costs for all parties, all the instruments encourage coordination and collaboration among development agencies in conducting the assessments. (See also the discussion below, in the section on issues for further consideration.) Third, all the instruments encourage dissemination of the assessments and their findings--and sometimes participation by a broad cross-section of society in recipient countries--on the premise that better understanding of governments' fiscal and financial decisionmaking may increase pressures for change from civil society. To that end: · PERs have increasingly incorporated consultation with nongovernment entities, and often rely on substantial cooperation from local universities and research organizations. · Many governments have agreed to post their Fiscal ROSCs on the IMF website. · A recent Bank-IMF paper discussing HIPC AAPs emphasizes that mon- itoring by civil society organizations can increase accountability (World Bank and IMF 2002). · DFID plans to combine its support for public expenditure management with measures to encourage input from the private sector and civil society. Finally, development agencies follow (with slight variations) a common sequence of actions in conducting assessments. The process starts with planning and desk studies, then moves to a concept paper, a review meet- ing, identification of peer reviewers and other arrangements for quality control, a field mission (or missions, if needed), multiple drafts of the assessment report and accompanying reviews, a government review, the final report, and public dissemination. Most decisions about whether and how to conduct assessments are driven by the needs of development agen- cies' country assistance programs and by the technical capacity of the coun- tries concerned. Priority tends to be given to countries for which development agencies are considering budget support and where agency rules require assess- 38 Assessing and Reforming Public Financial Management ments before providing such support. Government consent is always required, usually in the form of a formal "request" that the agency conduct the assessment--though, in reality, assessments are almost never genuine- ly requested by the governments concerned. Still, it is hoped that through open dialogue, host governments will come to value the assessments and own the results. In conducting assessments, development agencies have often been crit- icized for sending multiple missions (both from the same agency and across agencies) seeking the same basic information and asking similar questions of public officials. Insufficient coordination across multiple mis- sions and assessments creates several problems. It often results in a lack of coherence between the analysis and recommendations of different assess- ment reports. It can also lead to an excessive number of recommendations for strengthening governance of and tackling technical weaknesses and capacity gaps in public financial management. Such problems, for exam- ple, developed in Burkina Faso, where a CPAR, CFAA, and Fiscal ROSC were conducted in 2000­02 (box 3). In response, agencies are taking steps to ensure that thorough desk reviews of all available information--from both donors and governments-- are completed before defining the scope of assessments, and certainly before sending missions. As interest in assessment instruments has increased in recent years, so has their funding. Thus development agencies are paying more attention to quality control, though approaches vary. Internal evaluations of these instruments have also contributed to their recent evolution, including significant changes in emphasis and increased standardization, rigor, and precision in their application. For example, the World Bank's Quality Assurance Group conducted a major review of CFAAs and CPARs undertaken in 2000­01 (World Bank 2002e), and the Bank recently issued extensively revised guidelines for both. In addition, interim guidelines for PERs were developed in 2001, and the Bank is con- sidering formalizing these guidelines as part of a joint review of Bank-IMF work on public expenditure (see below). Similarly, the DFID has produced a manual for staff on its approach to assessing public expenditure manage- ment (DFID 2001b). The European Commission is replacing its tradition- al audits with assessments that emphasize the development implications of the analysis and the importance of monitoring changes in public financial management. Finally, to improve their capacity to advise governments on public expenditure management, many development agencies are hiring more staff with expertise in these areas. Methodology and Main Findings 39 Box 3 Multiple assessments result in myriad recommendations in Burkina Faso Burkina Faso recently went through a number of assessments in a short period. In 2000 a CPAR was completed, which led to reforms of public pro- curement. Soon after, the government conducted a broad analysis of its sys- tem for public financial management. As part of this analysis, question- naires were distributed to various parts of the administration, with a focus on the Ministry of Finance. The information was then consolidated in a summary report. In parallel, in the first half of 2001 the IMF conducted a Fiscal ROSC (with the final report published in 2002). Then in September 2001 a CFAA was launched to provide external verification of the government's findings in its analysis of public financial management. The CFAA was used as a pilot case for a multidonor assessment and drew on experts from a number of African donor agencies. In addition, the CFAA included an audit of gov- ernment practices to confirm its findings. Also in 2001, several bilateral missions visited Burkina Faso to gather more information on the govern- ment's reform agenda for public financial management. Participating in these various assessments and receiving multiple missions in a short period imposed relatively high costs on the administration, especially given the Ministry of Finance's limited capacity. The assessments may have helped provide a comprehensive view of the various aspects of Burkina Faso's system for public expenditure management. Moreover, the coverage of the assessment work--for example, on the responsibilities of banks and the government for budget execution and reporting in the Fiscal ROSC, and on revenue issues in the CFAA--was broader than in many other countries. But for some elements of the budget cycle, especially budget execution, there was clear duplication in the result- ing recommendations. Moreover, none of the assessment reports refer to the recommendations of the others. As a result more than 280 recommendations were made. And though many have similar objectives, they sometimes pro- pose different approaches to achieve them. These recommendations will need to be made consistent and coherent during the development of the overall reform plan--which will require significant government effort. 40 Assessing and Reforming Public Financial Management DIFFERENCES Even the most original assessment draws on findings from other studies. But even the simplest synthesis contains original elements--and an assess- ment of risk and development needs is always "original" to the agency con- cerned. The instruments surveyed fall into two broad categories: "primary" and "secondary," with "primary" instruments undertaking original data col- lection and analysis and the "secondary" instruments relying mainly on the data, analysis, and findings of the primary sources. The predominantly primary instruments are PERs, CFAAs, CPARs, and Fiscal ROSCs. The secondary instruments are DFID assessments and HIPC AAPs (though AAPs have included short field missions, sometimes combined with missions on PERs, CFAAs, and Fiscal ROSCs).3 The new approach proposed for EC assessments and "compliance tests" falls in between, relying on the findings of other instruments but also carrying out some original analysis--as with the EC's traditional ex post audits. As dis- cussed, the instruments vary considerably in how they gather data and use questionnaires and diagnostic checklists. Coordination and integration issues mainly arise as part of the data collection and analysis performed for PERs, CFAAs, Fiscal ROSCs, and EC audits; they are less important for CPARs given their specialized focus. The first main difference among the instruments is the level of unifor- mity in report approaches, scope, and coverage. Fiscal ROSCs are arguably the most uniform: they are applied using a focused, standardized question- naire and supported by a detailed user manual and very specific format. HIPC AAPs share these characteristics, including the use of questionnaires, a single set of benchmarks and indicators, and a common format. Both are highly organized, carefully structured exercises carried out under tight schedules--another factor that contributes to their consistency. These fea- tures stem from one of the main objectives of these instruments: enabling cross-country comparisons of their results. CPARs, while tailored to country characteristics and problems, also adopt a relatively standardized approach because of their focus on a single technical area. Their uniformity is also aided by the World Bank's 15­20 years of experience in assessing procurement systems.4 CFAAs have evolved rapidly since early 2002, and the revised guidelines issued in May 2003 and recent applications have a more uniform scope and content than in earlier years, with a standard format and basic mandatory coverage. The European Commission and many bilateral agencies now Methodology and Main Findings 41 consider CFAAs a core product, particularly for fiduciary risk analysis. But the considerable overlap between CFAAs and other instruments--especial- ly PERs and Fiscal ROSCs--requires clarification (for further discussion see the next section, on issues for further consideration). Though all PERs contain certain core material (for example, on each country's macroeconomic and fiscal environment), their range and cover- age vary much more than do the other instruments--an outcome that is probably inevitable given the instrument's wide scope. It is perhaps partly for this reason that the draft PER guidelines prepared in 2001 were not issued formally. PERs almost always discuss main fiscal issues and key expenditure issues in a country's major sectors (health, education, trans- portation, energy, and so on), but the choice of sectors, identification of problems, and depth of analysis differ considerably. For example, most PERs refer to the cost of public sector wages and pensions as part of their fiscal analysis, but only a few devote much attention to civil service or pen- sion reform. Because of their wide reach, PERs take longer to complete and are conducted less often than the other instruments.5 Finally, despite common rhetoric, government ownership varies widely by instrument and by country and is very weak in some cases. It bears repeating that strong government ownership makes it far more likely that the results of any analysis will lead to realistic, sustainable reforms. In countries dependent on aid and with limited capacity, such ownership can be fostered by a govern- ment-led PER, which involves governments and development agencies in a continuing dialogue on improving the budget process and increasing local capacity (see the discussion above on Tanzania). In more developed countries with substantial capacity, government leadership ensures genuine ownership-- as in Turkey, where the government set the agenda and defined the scope for an integrated approach to the assessment and reform of public expenditure. INSTITUTIONAL AND GOVERNANCE CONTENT With the possible exception of the broad constitutional and legal descriptions of the state apparatus in PERs and Fiscal ROSCs, the instruments provide lit- tle guidance on defining and accounting for institutional factors in assess- ments. Nor do any of the guidelines recommend that assessment teams include staff with specific expertise in public management and governance. It seems to be assumed that in most cases economists and financial management and procurement staff can undertake any needed institutional analysis. 42 Assessing and Reforming Public Financial Management Not surprisingly, this approach has led to neglect of institutional con- siderations and lack of focus and coherence in preparing action plans and designing capacity building initiatives--and occasionally to major mistakes. A review of the guidelines for assessment instruments--and, to a lesser extent, current practices--points to numerous institutional and governance issues that require increased coverage, deeper analysis, or both. The draft PER guidelines state that, as a first step toward improving budg- et preparation and execution, an institutional analysis should be undertaken of the key dimensions of public expenditure management, guided by three ques- tions: What formal rules and institutions govern budget management? How effective are they? And how could they be improved or applied more effec- tively? Many PER reports also address the broad institutional and legal con- text: constitutional requirements for the budget and its management, the state's legal and institutional structure, and intergovernmental relations. The draft guidelines do not explain how to perform such institutional analysis except to say that capacity issues need to be addressed in a highly participatory way. But PER teams often do not have the skills and experience required to conduct such work. As a result some PERs have confused orga- nizational and institutional issues--and so have made recommendations for changing organizational responsibilities but paid little attention to the insti- tutional necessity of changing the rules of the game and the incentives that lead to organizational forms and behaviors requiring modification. The recently updated CFAA guidelines make little mention of institu- tions, yet the content and nature of these reports imply some degree of institutional and governance coverage (World Bank 2002b). In addition, the guidelines refer to "the legal framework and institutional arrangements for the country's public financial management." Though the guidelines do not explicitly discuss corruption, evidence from corruption surveys, Trans- parency International assessments, World Bank Institutional and Gover- nance Reviews (IGRs), and other reports can be used in CFAAs. The guide- lines emphasize the importance of action plans that are "clearly justified and prioritized, [and] differentiated in terms of their short, medium and longer term impact." Yet no guidance is given on the extent to which action plans should address the root causes--as opposed to the technical symp- toms--of poor expenditure management. The CPAR guidelines state that all aspects of a country's procurement system must be assessed, including its legal and organizational framework, procedures, tools, decisionmaking and control procedures, anticorruption initiatives, and contract administration (World Bank 2002c). Enforcement Methodology and Main Findings 43 of system requirements must also be scrutinized. The guidelines stipulate the need for "detailed institutional analysis of key government ministries and agencies in the actual performance of public procurement responsibil- ities, their strengths and weaknesses, priority capacity building needs, the governance environment in which they operate, and the incentives needed to improve performance." Moreover, action plans should "carefully assess the political feasibility of the main recommendations." In addition, the CPAR checklist and questionnaires have a strong implic- it focus on corruption in that most of the rules, regulations, and procedures are designed to ensure transparency and efficiency. The CPAR guidelines (along with those for EC audits) contain the most explicit, extensive refer- ences to institutional and governance issues of all the assessment instru- ments--though this does not mean that the issues are adequately reviewed in the field or fully reflected in assessment reports. The treatment of institutional and governance issues by CFAAs and CPARs has evolved since the Bank's Quality Assurance Group reviewed fiduciary eco- nomic and sector work undertaken in 2000­01 and recommended changes in their guidelines (World Bank 2002e). The review found that CPARs and CFAAs generally failed to identify how weaknesses in areas such as incentives, technical competence, and the governance environment contributed to poor performance. The reports studied also did not adequately address weak appli- cation and enforcement of formal laws, rules, and procedures for public expen- diture management. More recent CFAAs and CPARs have paid more attention to institutional and organizational issues, including the extent to which rules and procedures are followed. Still, the instruments contain little analysis of the nature and possible causes of cases where they are not.6 Fiscal ROSCs cover broad institutional elements such as the structure of the public sector--including the roles and responsibilities of the executive branch, parliament, and supreme audit institutions--and touch on a num- ber of governance issues. In particular, these reports examine: · Whether commitments and expenditures of public monies are governed by comprehensive laws and openly available administrative rules. · Practices for extrabudgetary expenditures and quasi-fiscal activities. · Regulation of the private sector and whether public-private relations are conducted in an open manner. · The degree of discretion in taxation and regulation. 44 Assessing and Reforming Public Financial Management · Whether procedures are in place to verify the accuracy of fiscal data. · Whether internal controls are sufficient to safeguard public monies. · The degree to which laws are observed in practice. · Weaknesses of watchdog and oversight organizations. · Whether recommendations from internal and external audits are fol- lowed. Fiscal ROSCs also indicate whether corruption or poor governance is impeding the achievement of international standards for transparency, though they do not analyze the causes of such problems. The guidelines proposed for EC audits emphasize the importance of examining the institutional and governance factors that affect public expen- diture management, including the legal framework, arrangements for dem- ocratic oversight and control, anticorruption efforts, incentives of civil ser- vants performing these functions, and organizational constraints (EC 2002). Based on the findings and other information and analysis, the guide- lines urge the European Commission and the government concerned to agree on a corrective action plan. The next section, on issues for further consideration, offers recommen- dations for remedying assessment instruments' neglect of institutional and governance considerations, including by preparing new and comprehensive guidelines on these issues. SUMMARY The mapping of the assessment instruments has led to the following con- clusions: · Although the instruments reviewed in this study are often called assess- ments of public expenditure management, they include issues--such as forecasting government revenue and management of public debt--that are outside the expenditure domain. This broader concept should be used in streamlining and rationalizing the use of these instruments. · There is overlap between the coverage of the instruments, especially between CFAAs and Fiscal ROSCs. The next section offers some pro- posals for rationalizing and simplifying these instruments. Methodology and Main Findings 45 · Though there are differences in the objectives of the instruments that may justify some overlap, they also result in a lack of clarity about how assessment information and analysis should be interpreted and used. For example, PERs have largely developmental objectives, while CFAAs and CPARs have both a fiduciary and developmental orientation--a mix that can reduce the clarity of their information and analysis as well as confuse relationships between the World Bank and client governments. Similar- ly, though Fiscal ROSCs are primarily concerned with assessing how well transparency and accountability arrangements match the IMF's fis- cal code, they can also be given a fiduciary interpretation. And while EC audits have traditionally had a fiduciary focus, the commission is increas- ing its emphasis on developmental objectives. · Though there is overlap between PERs and CFAAs, especially in assess- ing upstream (and sometimes downstream) aspects of the budget process, in most cases it is more potential than real. This is because CFAA coverage of budget preparation is largely confined to issues of comprehensiveness, realism, and classification--and where such issues are covered by a PER, the recently updated CFAA guidelines make clear that the PER information should be used. · HIPC AAPs provide a comprehensive summary of a country's public expenditure management system, including key indicators of perform- ance. So, while this instrument overlaps with the others, and attention needs to be given to integrating it with other diagnostic work, it adds value through its different approach and objectives and its explicit use of performance indicators. · EC audits and compliance tests should be considered complementary to CFAAs, which do not use audit techniques. But as discussed below, greater efforts are needed to substantiate this complementarity, through stronger collaboration between the European Commission and the World Bank. · Technical synergies between the instruments could be more fully exploited. For example, CPARs address issues--such as control and audit of procurement transactions, allocations of funds for capital invest- ment projects, and procedures for cash disbursements--that also receive substantial coverage in some PERs, CFAAs, and EC audits. Addressing such cross-cutting issues requires taking a fresh look at World Bank (and IMF) business practices, including the skill mix of the teams of econo- 46 Assessing and Reforming Public Financial Management mists and specialists in financial management, procurement, and gover- nance that conduct assessments. The mapping also revealed important gaps in the instruments--both in their formal guidelines and in how those guidelines are interpreted when assessment reports are produced: · None of the instruments provides broad, substantial coverage of the forecasting, collection, and administration of taxes and other govern- ment revenue, though many PERs address some of these issues and the recently updated CFAA guidelines include accounting and control aspects of tax administration. This oversight is surprising given that accurate revenue forecasts and efficient tax collection are crucial for sound public expenditure management. · Although asset management is included in the updated CFAA guide- lines, no CFAA has covered this issue--which also requires a wider per- spective than financial management alone, including sound understand- ing of accounting, economic, and procurement issues. · Debt and aid management are covered in PERs and CFAAs but, as with revenue forecasting, not systematically or comprehensively. · Effective management of public records is an important element of pub- lic expenditure management but is hardly covered by the assessment instruments. The United Nations Development Programme's CON- TACT guidelines include a records management module, and the Inter- national Records Management Trust is developing--as a subprogram of PEFA--questionnaires covering management of government records in areas such as financial management and government payroll. Consider- ation should be given to combining the records management compo- nent with the main instruments, particularly CFAAs and CPARs. · The instruments--particularly CFAAs and to a lesser extent PERs and CPARs--tend to focus on central government systems for public expen- diture management. This approach is understandable given the expert- ise of development agency staff and the limited resources for conducting assessments. But while more attention is being paid to fiscal issues involving subnational governments, state enterprises, and agencies not funded directly by national budgets, it is often insufficient given the shift toward fiscal decentralization in many countries and recognizing that these can be areas of high fiscal risk. Methodology and Main Findings 47 These technical gaps in assessment instruments are in addition to the issues identified elsewhere in this study as needing more work--in particu- lar, strengthening the treatment of institutional and governance issues, increasing dialogue with governments and other donors, developing realis- tic action plans, ensuring adequate follow-up work, and establishing robust systems for measuring the performance and results of public expenditure management systems. NOTES 1. These sample reports included the following countries: CFAA-- Bangladesh, Benin, Bosnia-Herzegovina (BiH), Brazil, Mozambique, Philippines, Turkey, Uganda, Ukraine, Vietnam and Zambia; CPAR--BiH, Philippines, Russia, Turkey, Uganda, and Zambia; PER--BiH, Croatia, Ethiopia, Philippines, Uganda, the Philippines, Tanzania (and sector PERs on Agriculture, Health and Education), Turkey, Vietnam and Zambia; Fis- cal ROSC--Bulgaria, Mali, Pakistan, and Turkey; EC Audits--Cameroon, Chad, Cote d'Ivoire, Ghana, Mozambique and Zambia. 2. For a helpful discussion, see Helping Countries Combat Corruption: The Role of the World Bank (Washington, DC: The World Bank, September 1997). See also Anticorruption in Transition: A Contribution to the Policy Debate, (Washington, DC: The World Bank, 2000). See http://lnweb18.worldbank.org/eca/eca.nsf/General/D74DB51B2D46615 D8525695B00678C93?OpenDocument. 3. HIPC AAPs are conducted by Bank and IMF staff with considerable experience in addressing public expenditure management issues in the countries studied. Thus there is no need for long missions by staff new to the countries. 4. The Quality Assurance Group report on fiduciary economic and sec- tor work found that 75 percent of CPARs were satisfactory with respect to internal quality and likely impact, compared with only 50 percent for CFAAs (World Bank 2002e). (PERs and other Bank public expenditure work were not assessed.) 5. In countries for which the Bank is preparing an adjustment loan, a review of public expenditure management is integral to loan preparation and supervision. In such cases formal PERs are unnecessary and not visible in work programs. Thus if adjustment support is being prepared or provid- ed to a country, it is a serious mistake to assume that no attention is being 48 Assessing and Reforming Public Financial Management paid to public expenditure management simply because no PER has been conducted in recent years. 6. The Quality Assurance Group report also recommended a range of other changes in CFAA and CPAR guidelines, including strengthening business processes, revisiting skill mixes for financial management and pro- curement staff, strengthening partnerships, and enhancing knowledge management (World Bank 2002e). Recommendations and Issues for Further Consideration Efforts to strengthen assessment instruments should focus on several press- ing concerns--including increasing integration and improving coordina- tion and cooperation, addressing institutional and governance considera- tions, strengthening follow-up and performance monitoring, and develop- ing a programmatic and modular approach. INCREASING INTEGRATION AND IMPROVING COORDINATION AND COOPERATION Development agencies' growing use of public expenditure and financial accountability assessments has brought to the fore the crucial link between good public management and effective aid. Since the late 1990s there has been broad consensus that the most pressing question for development is not whether aid has been effective, but where it has been effective and under what conditions (World Bank 1998a; Collier and Dollar 1999). It has become clear that aid is most effective when it supports sound policies implemented by legitimate governments. In a major advance, development agencies are now accountable for assessing the risk that aid resources will be stolen, allocated to activities other than those in government budgets, or simply wasted. The problems of duplication and lack of coordination among assess- ment instruments pale in comparison with this advance. Still, as reported 49 50 Assessing and Reforming Public Financial Management by agency staff and confirmed by government officials, duplicated efforts and uncoordinated assessments impose high transaction costs on recipi- ent governments--especially given these governments' limited adminis- trative and financial management capacity. (In some cases agencies do not even try to avoid scheduling missions when budgets are in the final stages of preparation--an extremely busy period for ministries of finance.) Moreover, this state of affairs has sometimes resulted in superficial, over- lapping, or incomplete analyses and prevented the provision of coherent advice and systematic feedback to recipient governments and sponsoring institutions. The Task Force on the Harmonization of Donor Practices--spon- sored by the Development Assistance Committee (DAC) of the Organi- sation for Economic Co-operation and Development (OECD)--recent- ly commissioned a survey of the burdens that donors have placed on eight recipient governments. Although the survey did not focus on the instruments of concern to this study, several of its findings are pertinent. First, the overlapping, donor-driven, and often rushed approach taken by donor missions placed excessive demands on government officials, with different missions asking the same people similar questions again and again. Uganda, for example, hosted six missions at the same time, with four asking similar questions of the same people--and all were World Bank missions. Second, multidonor missions can reduce such burdens, but not always. When all donors insist on direct involvement, it can result in unmanage- able and incoherent missions. Thus there is a need for donors to accept analytical work conducted by other agencies. Third, in Bangladesh and Tanzania the survey found that donors demanded an unrealistic scale and pace of reform, based partly on insufficient understanding of country cir- cumstances. This point underscores the need for better understanding of a country's institutional and governance context when actions are proposed to address weaknesses in public expenditure, procurement, and financial accountability systems (University of Birmingham School of Public Policy 2002, p. 9). Another recent study, sponsored by the Strategic Partnership with Africa (SPA), reviewed the experiences of several African countries (Burkina Faso, Malawi, Mozambique, Tanzania, Uganda) in undertaking participatory CFAAs. The study concluded that there was often poor coordination with- in CFAA teams and between CFAAs and other instruments, and sometimes insufficient government participation and collaboration. And while partic- Recommendations and Issues for Further Consideration 51 ipatory approaches are intended to lower transaction costs, that does not always happen. Although some of the CFAAs were carried out at the same time as PERs and Fiscal ROSCs, the different teams did not coordinate their interactions with government officials. As a result the administrative burden was the same (Reite 2002). Thus, this study's first recommendation is perhaps the most obvious: no consideration should be given to creating or expanding assessment instru- ments until duplication and coordination problems are resolved to the sat- isfaction of all concerned.1 Many of the study's other recommendations-- based on the mapping exercise, reviews of assessment guidelines and reports, and interviews--also involve improving coordination within and between development agencies and partner governments. But before dis- cussing ways to improve coordination, it is worth identifying constraints on harmonization and integration of the various instruments. Constraints on harmonization and integration Because assessment instruments are products of individual development agencies, the issues covered and methods used naturally reflect agency objectives. These instruments will always reflect their origins, and there are many incentives to keep them separate products, including: · Development agencies are overseen by boards, ministries, legislatures, supreme audit institutions, and other entities to which they are ulti- mately accountable. · Boards and senior managers usually demand identifiable products with the stamp of the agency, oversight body, or both. · The staff responsible for assessments often work in organizational "silos" and lack incentives to work collaboratively within their own organization or (to an even lesser extent) with other agencies. There are also significant differences in the specific objectives for improving public expenditure management. For example, the rationale and objectives of EC audits may be partly driven by statutory imperatives, or in the case of both EC audits and DFID assessments by demands from legis- latures, audit institutions, and other oversight bodies for better fiduciary assessments. Similarly, there is growing pressure from the boards of the World Bank and IMF for stronger coordination and collaboration between the two institutions on public expenditure management issues (World Bank 52 Assessing and Reforming Public Financial Management and IMF 2003). Moreover, the demands of managers and oversight agen- cies evolve from changes at the top, and "turf" considerations may also lead to a desire for separate products. At the procedural level, arrangements for peer reviews and other quali- ty control mechanisms vary considerably among development agencies, which reflects their managerial styles and administrative cultures. As a result, there may be genuine professional concerns (as opposed to efforts made for self-protection) about the quality of assessments produced by other agencies. Finally, as in all bureaucracies, groups within agencies fight for their turf and for maintenance of the status quo. Thus, even when boards or top managers announce policies intended to reduce overlap and turf battles within and between agencies, line managers and staff may be able to subvert or ignore such instructions for a considerable period. These constraints have three important implications. First, to the extent that integration occurs, it is somewhat more likely to happen within than between agencies. Second, when a recipient government has enough capac- ity, clarity, and political power, it can usually compel coordination among development agencies. Third, thinking in terms of fewer instruments miss- es the point. Although further proliferation should be halted, some instru- ments can rely largely on the analysis of others without forsaking the agency's independent judgment or desire to have its own separate product. Thus efforts to improve assessments should not focus on eliminating or merging instruments to reduce their number. Rather, they should focus on ensuring the substance and quality of analyses on which all concerned par- ties can rely, and on improving coordination and reducing costs for recipi- ent governments. And as noted, there is scope for streamlining the scope and coverage of instruments to reduce unnecessary overlap and fill gaps-- and possibly going further and developing a programmatic and modular approach, as discussed below. Exchanging information and programming intentions Despite the risks and costs of duplication, there is no systematic exchange of information among agencies on their plans for conducting assessments. The issue here is not the need to create a reliable database on fiscal and budget issues shared by development agencies and recipient governments. That vast task is already being addressed through other donor coordination mechanisms. For example, a lot of information is available from IMF data- bases and statistical publications on government finance. In addition, the Recommendations and Issues for Further Consideration 53 OECD is preparing (with World Bank support) a new public expenditure database containing budget information for a range of countries, including some developing and transition economies. Of concern here is the narrower but still important need of agencies to exchange systematic, timely information on their plans for conducting assessments, and to share the reports. This task should be aided by the Country Analytic Website being developed by the World Bank and other donors.2 In addition, steps are being taken to give development partners online access to IMF and World Bank documents. Both efforts should make it easier to improve the sequencing of assessment instruments. But better programming also requires the cooperation of country project man- agers--and is achieved when a recipient government has sufficient capaci- ty and political strength to take the lead in coordinating assessment work among agencies and instruments. Coordination between agencies The nature of budget support provides a powerful incentive for donors to collaborate. Indeed, it was one of the forces driving the creation of PEFA. There are many examples of successful collaboration between multilateral institutions: the Bank and the IMF on HIPC AAPs, multilateral develop- ment banks (and several bilateral agencies) on CPARs,3 and the Bank and the Inter-American Development Bank on CFAAs. Most donors must jus- tify their provision of budget support using sound assessments of recipient governments' financial management and accountability systems. But while aid agencies are generally unwilling to risk relying entirely on assessments conducted by other agencies, most bilateral agencies do not have the resources and skills required to conduct robust assessments. The obvious response would be to conduct joint assessments, but in most cases these are impractical.4 Such assessments would result in multiagency teams of 20­30 people on three- or four-week missions--a managerial nightmare that would also force the recipient government to put aside its regular activ- ities for a long period. So what is the solution? This predicament creates an important avenue for coordination and raises an equally important corollary. Better coordi- nation and lower transaction costs can be achieved if bilateral develop- ment agencies rely on the assessments of multilateral agencies equipped to perform them--normally the World Bank and, on fiscal transparency issues, the IMF. But these institutions must be far more open to cooper- 54 Assessing and Reforming Public Financial Management ation with each other and to suggestions, requests, and criticisms from their partners. At a minimum, given that the quality of the analysis is paramount for improving public expenditure management and financial accountability, the IMF should strengthen its ex ante consultation with the Bank on the terms of reference for its Fiscal ROSCs, and the Bank should reciprocate for its PERs and CFAAs (once consistency between these two instruments has been assured, as proposed below). The Bank and IMF already share some documents and information and request comments from one another. A broad division of labor has been established--beginning with 1981 guidelines for Bank-IMF collabora- tion--that has largely been respected, though not without friction. Pecu- liarities persist, especially in the separate public expenditure management activities of the Bank's Public Sector Governance Board and the IMF's Fis- cal Affairs Department, sometimes leading to divergent results. There are also strong differences of approach between the areas and regions of the Bank and the IMF. The ongoing review of Bank and IMF collaboration on public expenditure issues should be used to propose changes and streamline work on both public expenditure and revenue management. Bank-IMF collaboration also raises the issue of overlap between CFAAs and Fiscal ROSCs. Both institutions seem to accept that such overlap is substantial, particularly in assessing transparency and fiduciary risk. All the elements described as being central to assessing risk to Bank funds in the recent revision of CFAA guidelines are explicitly defined elements of the IMF's fiscal transparency code. So, in principle, if a ROSC has been com- pleted, a CFAA may only have to deepen or broaden certain aspects of its analysis to achieve its fiduciary risk objective. And if the sequence is reversed, the ROSC should be able to apply CFAA findings directly to its assessment of public expenditure management rules and procedures. Thus it should not require much effort to combine the collection and analysis of information, through joint teams and missions, using a com- bined questionnaire and database, and produce a joint report that satisfies both institutions' fiduciary and surveillance objectives. The two institutions could also use the information for other purposes--for example, the Bank could use it to feed into the dialogue with countries on reforms linked to Country Assistance Strategies and Poverty Reduction Strategy Papers. The Bank and IMF have also suggested other changes to enhance coop- eration: sharing more information on work plans and mission schedules, exchanging information between mission teams working in the same coun- Recommendations and Issues for Further Consideration 55 tries, arranging joint missions and sharing staff, and making more effective use of the Internet to share databases and internal Websites. Though these are steps in the right direction, they may not go far enough. Assessments suffer from several other problems related to cooperation and coordination, in particular: · The somewhat divergent interests and objectives associated with the development of the instruments reviewed in this study means that they are similar but slightly different, and there is not necessarily agreement on the fiduciary protection provided in a particular country. · Ensuring country ownership of the diagnostic process and open provi- sion and exchange of information is likely to be much harder if a devel- opment agency appears to be setting the agenda or collecting informa- tion for fiduciary purposes or to set conditions for aid disbursement. · None of the instruments provides an entirely satisfactory measure of fiduciary risk. · The potential conflict between the need for government ownership and the robustness of an assessment is especially relevant to financial accountability assessments. Against this background, it is recommended that: · A common definition of fiduciary risk be agreed upon between govern- ments and donors. · The relationship between the role of the instruments in evaluating fidu- ciary risk and contributing to long-term development goals be clarified. · Consideration be given to dividing the fiduciary and development aspects of diagnostic work into separate processes and reports and to creating a more independent process--with some element of joint ownership by donors and external quality control and validation--for carrying out assessments. Doing so would allow donors to develop stronger partner- ships with countries, reduce the possibility of conflicts of interest, and strengthen the focus on development issues and capacity building. In addition, important issues arise in the coordination of activities between the European Commission and the World Bank and IMF--par- ticularly between EC audits and CFAAs. The evaluation work related to EC budget support has moved away from traditional ex post audits, focused 56 Assessing and Reforming Public Financial Management on establishing the eligibility of expenditures financed by aid, to an approach based on learning from other institutions' diagnostic work, filling gaps where necessary, and conducting compliance tests and annual updates (using audit techniques) to establish whether public expenditure manage- ment systems are operating efficiently. Linked to this approach is an EC proposal to develop indicators for monitoring improvements in public financial management. In principle there is close complementarity between EC compliance tests and CFAAs, because the CFAA is not an audit and the Bank has eschewed audit techniques. But more needs to be done to increase collaboration. For example, the European Commission and the Bank need more experience working together in the field (some joint work has already been done, espe- cially in Africa), with case histories and lessons recorded from these experi- ences, and should work together with governments. Moreover, as noted, the methodology of EC compliance tests has not been fully defined and docu- mented. The Commission should fill this gap, drawing on the experience and expertise of the consulting companies that conduct most EC audits and compliance tests--and consider how information deriving from this work can be linked with the development of performance indicators (PEFA 2003). In addition, the Bank and IMF could seek input from the two or three largest partner development agencies in each recipient country, allowing them to influence the design of assessment work (for example, through con- sultations on initiating concept memorandums and terms of reference), con- sulting with them during assessments, and inviting them to participate in country dialogue (for example, through government and donor workshops) before assessments are finalized. Doing so would make partner agencies more inclined to rely on Bank and IMF instruments instead of conducting their own analyses, while enabling them to shape and share assessment results. As noted, the agencies involved would not relinquish their rights and responsi- bilities for independent judgments. But such judgments would be exercised through careful scrutiny and synthesis of other agencies' analyses--without necessarily carrying out multiple analyses or duplicating efforts. Often, though, what aid agencies view as better coordination is seen by recipient governments as "ganging up." This difference in viewpoint will persist as long as assessments are top-down exercises carried out by aid agencies with only superficial participation by recipient governments. For coordination to provide the benefits of scale economies, lower transaction costs, and better advice without the threat of heavier interference and "coordinated" leverage, assessments must be at least partly owned by recip- Recommendations and Issues for Further Consideration 57 ient governments. Similarly, dialogue on public expenditure management reform is more likely to achieve its desired outcomes if governments can be reassured that assessment results are generally accepted (if not formally agreed) by the donor community. Two final points are worth mentioning. First, donor coordination issues also occur during efforts to implement assessment recommendations. Again, high transaction costs may arise for recipient countries if the super- vision efforts of multilateral and bilateral donors are poorly coordinated. Two country examples show how this outcome can be avoided. In Mozam- bique, bilateral agencies conduct an annual joint review as part of joint budget support to implement the actions identified in the Poverty Reduc- tion Strategy Paper. One of the review's main goals is to evaluate progress in improving public financial management. Similarly, in Tanzania bilateral donors have combined their budget sup- port under the Poverty Reduction Budget Support program. The donors have developed a matrix to assess progress in implementing the Poverty Reduction Strategy Paper, and public financial management is among the key areas assessed. Moreover, these donors are engaged in discussions with the World Bank to develop a common matrix for Poverty Reduction Bud- get Support and Poverty Reduction Strategy Papers, and to sponsor joint missions for evaluating progress. Second, the OECD Development Assistance Committee's Task Force on the Harmonization of Donor Practices recently defined good practices for measuring performance in public expenditure management (DAC 2003; see also annex 2). These practices include planning and conducting diag- nostic reviews, ensuring quality, sharing reports with stakeholders, engag- ing in follow-up activities, updating reviews, and monitoring performance. Donors should consider this guidance when conducting reviews and coor- dinating activities. Coordination within the World Bank Internal coordination problems affect every large organization. But when it comes to assessment instruments, the people interviewed for this study and a review of existing guidelines indicate that the main ambiguities occur between the World Bank's Financial Management network, Poverty Reduction and Economic Management (PREM) network, and Procure- ment network. The first is responsible for CFAAs; the second for PERs and for leadership on adjustment lending in countries where such lending is 58 Assessing and Reforming Public Financial Management under way or being considered; the third for CPARs. Differences in per- spectives have made it difficult to ensure good coordination of PER, CFAA, and CPAR processes. Planning and budgeting for these instruments are largely based on parallel exercises with separate teams, peer reviews, and the like, and teams usually do not include experts from the other areas--let alone are they integrated. But there is encouraging evidence of a trend toward integration, or at least harmonization, of recommendations as the basis for a single dialogue with recipient governments (PEFA 2002). A recent Bank directive states that country teams can choose to prepare an integrated fiduciary assessment combining the CPAR, CFAA, and PER.5 To qualify as an integrated assess- ment, the final product must comply with the guidelines for each instru- ment, and the sector boards responsible for each must agree on the content and on how the product will be presented to the client. Combined PERs, CFAAs, and CPARS are most common in the Bank's East Asia and Pacific Region, where they have become almost standard--partly in response to pressure from the region's governments. But separation and occasional dis- sonance between PERs, CFAAs, and CPARs remain common. Some argue that a clearer division of labor is needed between PERs and CFAAs. In that view PERs should focus on upstream public expenditure management issues--expenditure programming, policy-budget links, sec- toral resource allocations, and budget preparation--and CFAAs should focus on downstream issues--budget execution, accounting, audit and con- trol, and executive accountability to the legislative branch. The current division of labor is far less clear. CFAA guidelines include budget prepara- tion in the scope of work, but this process has long been covered by PERs due to its direct link to expenditure policy, medium-term programming, and the overall macroeconomic framework. The executive's accountability to the legislature is also part of CFAA guidelines, yet it should be analyzed from an overall governance perspective, not just from a narrow budget and public accounting perspective. To some extent such difficulties arise from the absence of published guidelines for PERs. As noted, the broader the scope of an exercise, the harder it is to define its scope and coverage. This makes it difficult to clear- ly define the terms of reference for an exercise like a PER, which necessari- ly has a wide scope but needs to be tailored to the problems of the recipient country. But while the breadth of the issues to be covered (such as institu- tional and governance concerns) can justify more indicative guidelines for PERs than for, say, CPARs, it cannot justify the continued absence of any Recommendations and Issues for Further Consideration 59 guidelines. The time has come to formulate PER guidelines, reach Bank consensus on them, receive feedback from other development agencies and sample recipient governments, and formally promulgate the guidelines. Thus the recent decision by the Bank's Public Sector Governance Board to prepare such guidelines is encouraging. But the challenges of coordination go beyond the need for separate guidelines, particularly since individual assessments of public expenditure management should be seen as parts of an integrated whole. The World Bank would never advise a recipient government to set rigid organization- al boundaries between budget preparation and budget execution. In every country the ministry of finance is responsible for both preparing a realistic budget that reflects government policy priorities and for facilitating and monitoring its execution. In addition, institutional and governance issues, including corruption, emerge at every stage of public expenditure manage- ment--not just in upstream or downstream phases. Different elements and phases of the budget cycle have multiple inter- dependencies, and integrated approaches can create important synergies that improve assessment work (box 4). Thus the Bank's assessment instru- ments should strive for increased integration and cooperation, not further separation. Yet some respondents believe that the organizational bound- aries between CFAAs and PERs have become more rigid in recent years. Short of creating a single office in the Bank to advise on all phases of the public expenditure management cycle and financial accountability--which is not a realistic possibility at this point--ways should be found to bring the PER, CFAA, and CPAR processes closer together. At the same time, efforts should ensure that the guidelines for these instruments are consistent, facil- itate integrated approaches, and provide stronger guidance for institution- al and governance work. The recent PEFA paper on integrating these instruments concludes that "the potential benefits of integration can out- weigh the costs, if the process of integration is managed effectively, and is adapted to the country circumstances. A review of the Bank's internal rules and procedures may be necessary to support the development of integrative and collaborative work" (PEFA 2002, p. ii). The paper also identifies good practices to guide country teams in adopting integrated approaches and sharpen the Bank's operational procedures. These good practices include: · Recognizing the key role of country directors and country economists, working with sector managers, in designing and implementing diagnos- tic and reform work--which should be closely linked to Country Assis- 60 Assessing and Reforming Public Financial Management Box 4 How an integrated approach can strengthen assessments By focusing on the complete cycle of public expenditure management and bringing together multidisciplinary teams--including economists, account- ants, auditors, procurement experts, and public management specialists-- integrated approaches can generate important synergies: · A comprehensive picture of public expenditure management, without gaps or overlaps. · A consistent analysis of public expenditure management performance. · Recommendations that are consistent, prioritized, and reflect govern- ment capacity. · Greater force to arguments for systemic reform of public expenditure management, by broadening the range of government agencies and other stakeholders involved in the dialogue. Integrated approaches can also identify interdependencies between upstream and downstream phases of the budget cycle and provide better understanding of problems such as: · Unrealistic budgets, which can lead to unpredictable cash flows, diver- gences between budgeted and actual expenditures, and procurement delays caused by funding shortages. · Problems in procurement contracting, which can lead to low comple- tion rates and inadequate returns for investment projects. · Inefficient information systems for public expenditure management, which can fail to link the different stages of the budget cycle in a com- prehensive, unified accounting and reporting framework. · Incomplete audit trails, which can fail to take into account all relevant financial records on cash releases, procurement tenders and contracts, payroll management, and the like. tance Strategies, Poverty Reduction Strategy Papers, and related lending and technical assistance operations. · Being aware that this work may cover several years and a series of inter- related missions and reports. · Establishing a core team with appropriate skills and experience to man- age and conduct the work, with additional teams created as required to carry out specific analytical tasks. Recommendations and Issues for Further Consideration 61 · Developing a single concept memorandum for the work. · Strengthening peer review. · Ensuring the fullest possible participation and leadership of recipient governments. · Promoting the full exchange of information and analysis between proj- ect teams throughout the work, and coordinating the missions and work programs of these teams as much as possible. · Sequencing and prioritizing recommendations and action plans for implementing agreed reforms that reflect the capacity of the recipient country. One approach would be to combine PERs--or at least their public expen- diture management components--and CFAAs into a single product. But while this idea has theoretical attractions, it also raises practical difficulties. Com- bining the two instruments would increase the scope and length of an already lengthy report (unless PERs were split into components based on a sequence of activities spread out over time by use, for example, of the government-led approach described above for Tanzania). Moreover, separating the public expenditure management components of PERs from the expenditure analysis and policy components would effectively revert to a long-discarded approach to PERs, and would lose the essential connections between assessing policy options, preparing budgets, and delivering effective public services. Still, the idea of integrating PER and CFAA processes has merit. It would ensure that economists, public management specialists, and financial management specialists work together more closely and that PERs and CFAAs are seen more as a single process of assessment work, reaching toward a common goal, than as separate products with few connections. Thus the recommended approach is to foster such coordination and col- laboration by preparing consistent (or perhaps consolidated) guidelines and questionnaires, and changing Bank rules and procedures to facilitate the joint planning of PER and CFAA activities, joint missions, and, where appropriate, joint products. A first (minimal) step in the right direction appears to be feasible. The terms of reference for PERs and CFAAs, with sufficient detail on key issues and proposed approaches, should require formal reciprocal clearance by the relevant sector directors in the Financial Management network and Pover- ty Reduction and Economic Management network, respectively. (Subse- 62 Assessing and Reforming Public Financial Management quently, as proposed earlier, these terms of reference should be subject to formal comments by the IMF--which should reciprocate with Fiscal ROSCs--and then be submitted to recipient governments for considera- tion and approval.) The goal should be to make the two exercises start from a common basis, and not attempt ex post reconciliation, amalgamation, or dilution of their findings. Informal consultations on the two instruments, though common, have often been insufficient. In some cases a PER and a CFAA have been carried out in the same country, sometimes even during the same period, with no substantive interaction between the two and only perfunctory recognition of the other's results in the eventual reports. Formal reciprocal clearance, with its concomitant responsibility, could do much to resolve coordination problems upfront without requiring major organizational changes or pre- empting more comprehensive long-term solutions. For CPARs, which have a relatively well-defined and narrow focus, coordination requirements are less demanding. The review, however, should be conducted with full awareness of a country's corruption and gov- ernance problems rather than as an exercise that considers government procurement an isolated function. But this approach is already implicit in CPAR guidelines and can be achieved by adding a governance expert to CPAR teams, by engaging in consultations on terms of reference with the appropriate unit in the Poverty Reduction and Economic Management network (without formal clearance by sector directors), or by doing both. It is also important that PERs reflect the findings of previous CFAAs and CPARs. Thus the PER concept note should always include reference to procurement issues, and the PER process should incorporate CPAR find- ings and recommendations. These practices, already common, should be made explicit in CPAR and (eventual) PER guidelines. A recently launched review of the Bank's public expenditure work should help draw together the issues and concerns discussed above and propose practical solutions. The central concepts underpinning the review--that public expenditure work should be country-focused and country-driven, linked to Poverty Reduction Strategy Papers (where they exist) and donors' aid and technical assistance programs, and cover all relevant components of the public expenditure management cycle--are the same as those advocat- ed in this study (for further details, see the section below on the proposed programmatic approach). A final minor but important point involves Bank terminology for the ini- tial documents for fiduciary economic and sector work. There are initiat- Recommendations and Issues for Further Consideration 63 ing memorandums, concept papers, concept notes, issues papers, issues notes, and initiating concept memorandums referring to the same basic documents. The use of these different terms sometimes causes confusion inside and outside the Bank. The Bank might consider reserving "initiating memorandums" for a structural adjustment loan (as has been done since the late 1980s) and settling on one of the remaining terms for PERs, CFAAs, and CPARs. INSTITUTIONAL AND GOVERNANCE CONSIDERATIONS All the assessment guidelines should include a brief definition of institu- tional and governance issues and explain their relevance to analyses of pub- lic expenditure management. The generally accepted definition of gover- nance is found in World Bank (1992, p. 1): The exercise of authority, control, management, power of govern- ment. A more relevant definition for Bank purposes is "the manner in which power is exercised in the management of a country's economic and social resources for development." The Bank's main concern with sound development management thus extends beyond building the capacity of public sector management to encouraging the formation of the rules and institutions which provide a predictable and transparent framework for the conduct of public and private business and to pro- moting accountability for economic and financial performance. Governance is essentially about decisionmaking and implementation processes and the capacity underpinning these processes. What makes gov- ernance good or bad is the level of transparency, accountability, pre- dictability, and participation. But as noted, it is difficult to use these four characteristics to assess the overall governance of public expenditure man- agement, and much easier to do so for a specific area such as procurement, audit, or internal control. The extent of transparency, accountability, pre- dictability, and participation is determined by the underlying rules of the game--that is, institutional arrangements. Thus one can assess these dimensions by observing the rules in action. When analyzing a specific area, staff could be encouraged to ask whether information and decisionmaking are transparent, how individuals are held accountable for their actions, if regulations and rules are enforced effec- tively and uniformly, and the extent to which appropriate participation is 64 Assessing and Reforming Public Financial Management built into information gathering, decisionmaking, and implementation. Similarly, guidelines should include a brief explanation of the key concept of institutional economics--that behavior is governed by the (formal and informal) norms and rules prevailing in an organization and can be changed only by altering those rules, especially those concerning material and non- material individual incentives. The intention is not to create ersatz institutional specialists, but simply to make the assessment teams aware of the systemic factors affecting effi- cient and effective public expenditure management--and hence to know if and when to ask for help and advice. To the extent possible, assessments should draw on existing information from sources such as anticorruption surveys and the Bank's Institutional and Governance Reviews (IGRs) that have been carried out in some countries. There is no reason for every instrument to have the same approach or extent of coverage in this area. Indeed, it is far better for instruments to be focused on their key objec- tives--as with CPARs and Fiscal ROSCs--than to attempt superficial and probably misleading institutional analyses that would best be carried out by other means. If it is decided that institutional or governance problems are central to the assessment, the relevant expertise should be added to the team. Moreover, though institutional and governance considerations are relevant to every assessment and instrument, they are best handled by PERs due to their broad scope and links to the macroeconomic environment and public administration effectiveness. But putting the onus on PERs is not an entirely satisfactory solution. Stronger efforts must be made to analyze the political and governance underpinnings of the budget process, and doing so requires good under- standing of the political economy of the countries concerned. It may be dif- ficult to obtain such an understanding by inserting governance components into PERs, which are already overloaded. More fundamental, thorough analysis is required. Resources and time permitting, IGRs could be used to conduct such analysis, with the findings then incorporated into other instruments. Politics and governance arrangements usually do not change much over the medium term, so such reviews would only need to be done every five years or so. In response to concerns about overloading missions and overburdening governments, these reviews could be combined with PERs (as with Turkey's recent Public Expenditure and Institutional Review and a similar review in Bolivia). Moreover, successive IGRs would not have to be as extensive as the first. To support this development, consideration should be given to Recommendations and Issues for Further Consideration 65 developing guidelines for IGRs that identify the main concerns of the budget cycle and propose remedies to address them. Doing so would estab- lish a closer link between IGRs and assessments of public expenditure man- agement. If this approach were taken, it would also be critical to form the right team of specialists for IGRs--including political scientists (or politi- cal economists) with knowledge of the countries reviewed. FOLLOW-UP AND PERFORMANCE MONITORING As noted, the recommendations and action plans in assessment reports are often weak. If assessments gave greater emphasis to institutional and gover- nance analysis, in the ways suggested above, action plans would have a stronger basis in reality and be more valuable in promoting reform and build- ing capacity. In addition, many countries often pay insufficient attention to implementing the recommendations made in assessments. Thus experiences with joint donor reviews in countries such as Cambodia, Mozambique, and Tanzania, linked to the budget cycle or the Poverty Reduction Strategy Paper process, could be a useful model for other countries. A related issue involves the need for assessments to develop a frame- work for setting benchmarks and measuring performance and monitoring improvements in public expenditure management over time. This gap has partly been filled by HIPC AAPs, which use 15 indicators to evaluate pub- lic expenditure management. (DFID is piloting a similar framework to evaluate fiduciary risk.) Further work is being done by the Bank under the Governance Operations Progress Indicators (GOPIs) program to develop, among other things, public expenditure management indicators and test them in selected countries. In addition, the HIPC AAP indicators are being updated by the Bank and IMF with PEFA support, and new indica- tors--for example, on procurement--are being introduced for the round of HIPC assessments in fiscal year 2003. The OECD's Development Assistance Committee is interested in following up on its good practice paper on conducting diagnostic assessment and measuring performance (see DAC 2003 and annex 2). Among the questions that need to be addressed in such work are: · What are the primary objectives in measuring the performance of pub- lic expenditure management? Risk assessments, long-term development requirements, or both? 66 Assessing and Reforming Public Financial Management · What conceptual framework should be used to capture the required dimensions of public expenditure management performance? Should polit- ical economy factors and issues such as corruption be explicitly included, or are measures of systems, institutions, and budget outcomes adequate? · Should there be a standard set of indicators that can be applied to many countries, or should the aim be to develop a larger menu of indicators that can be drawn on by governments and their partners in developing national frameworks for measuring performance? · International standards and codes exist for issues such as accounting (IFAC), external and internal audit (IIA and INTOSAI), fiscal classifica- tion (IMF), and fiscal transparency (IMF). Should international stan- dards be developed for other issues, public procurement for example, and if so what bodies could be given responsibility for taking forward such work? · How should a widely accepted set of performance indicators be devel- oped? And how should the information be collected--for example, from World Bank and IMF assessments and EC audits and compliance tests? WHO ASSESSES THE ASSESSORS? In the end the teams that conduct assessments and ensure their quality are more important than the content of any guideline. The research and inter- views conducted for this study suggest that assessment teams do not always have the right skills--or sufficient restraint to focus on the issues most rel- evant to the instrument in question. This problem seems to be most severe for PERs, CFAAs, and Fiscal ROSCs. Some PER teams have engaged recipient governments in discussions about complex accounting or performance budgeting issues without any team member having significant international experience in these areas or understanding of the technical issues involved. In addition, there have been several instances of recommendations that were either wrong or advanced without a sense of proper sequencing and practical requirements. If such recommendations are adopted, the damage can be--and sometimes has been--considerable. Similarly, some recent CFAA teams have ranged far from their core competencies, especially into budget preparation and expenditure pro- Recommendations and Issues for Further Consideration 67 gramming issues, and have sometimes urged purely technical solutions devoid of a sense of the systemic fiscal management and public administra- tion factors involved in their implementation. For example, a recommen- dation in the previous version of CFAA guidelines to foster public access to financial information was misinterpreted by one CFAA team leader as encouraging the introduction of freedom of information legislation--a far broader issue than its financial information aspect, with significant gover- nance and public administration implications, and on which the team did not have all the necessary skills. When assessments are conducted in a purely advisory capacity without direct links to future assistance, unrealistic recommendations or inconsis- tencies can usually be ironed out through subsequent dialogue. But when a Fiscal ROSC (theoretically purely advisory), a PER (theoretically diagnos- tic), or a CFAA (theoretically not an audit) is carried out as part of prepa- rations for IMF support, World Bank assistance, or both, it is important to ensure that recommendations are correct, realistic, and consistent. In addi- tion to the measures for stronger coordination recommended earlier, care- ful attention needs to be paid by the responsible managers to staffing assess- ment teams, identifying competent and independent peer reviewers, and ensuring constructive interaction among teams carrying out fiduciary assessments. In such cases Quality Enhancement Reviews (QERs)--mech- anisms that ensure robust, independent vetting of Bank projects while they are being prepared--should be the rule rather than the exception to improve quality at entry. A related issue is how to provide quality assurance in assessments involv- ing several development agencies. The Bank has mechanisms such as the Quality Assurance Group (QAG), but these are mostly internal. Agencies should consider establishing a quality assurance procedure that applies to all participants in multidonor assessments. DEVELOPING A PROGRAMMATIC AND MODULAR APPROACH In the late 1980s, in response to the growing length, complexity, and repet- itive content of some PERs, it was suggested that the Bank develop a mod- ular PER that offered standard coverage and formats for each module but that was adapted to country circumstances by including only the relevant areas and issues, as represented by modules. Though attempts were made in that direction, however, they did not succeed. 68 Assessing and Reforming Public Financial Management Similar suggestions have recently been advanced in response to the over- lap between assessment instruments. One suggestion, previously noted, is to fully integrate existing fiduciary assessments to ensure consistency and exploit synergies--though this approach poses difficulties. Another option is the "programmatic" approach: reorganizing existing instruments into a set of modules, with each module representing a component of public expenditure management. This approach would eliminate the overlap between instruments, fill gaps, help avoid sending multiple missions to col- lect similar data, and allow countries and development agencies to focus on problems most relevant to each country. A modular approach already exists for the questionnaires, checklists, sur- vey instruments, and other tools that assist the fact finding on which assess- ments are based. The UNDP's CONTACT guidelines, for example, are essentially a set of linked questionnaires. As noted, there are also useful checklists and questionnaires for the Bank's PERs and other fiduciary eco- nomic and sector work. Similarly, the IMF's Fiscal ROSC can be consid- ered a module assessing the transparency and accountability of budget institutions and processes. To support teams conducting assessments, it may be worth consolidat- ing the various questionnaires and other tools into the modules created for each component of public expenditure management. These modules could be based on the classifications in box 2, supplemented by additional mod- ules covering the revenue side of the budget and possibly other issues out- side the scope of existing instruments and questionnaires. A more radical approach would be to establish a framework for public expenditure assessment and reform that is both programmatic and modular. Here "programmatic" means a sequenced program of diagnostic and capac- ity building activities, rooted in a strategy that the recipient country agrees to with donors and other stakeholders. Such an approach is illustrated in fig- ure 2, where the columns--representing the proposed modules--are equiv- alent to the main categories in box 2, complemented by a few other areas of concern (such as institutional analysis).6 The programmatic, modular approach is not intended as a blueprint for reform, but rather as a concep- tual framework that can be used to gradually move away from existing instruments without having to fully integrate fiduciary assessments. Relevant to such efforts is the working group created in July 2003 by the Bank and IMF, with PEFA support, to examine options for improving the robustness, relevance, and cost-effectiveness of public expenditure work. This working group was established partly in response to the February Recommendations and Issues for Further Consideration 69 Figure 2: A new public expenditure assessment framework Monitoring and Performance Evaluation and Results Government Strategy and Action Plan for Capacity Building Uses and Outputs: New Assessment Standardized Assessment (Overview) Framework Module Module Module Module Module Module Module Module X: X: X: X: X: X: X: X: Policy Budget Budget Monitoring Audit Revenue Compliance Institutional and and Formulation Execution Modules Expenditure Oversight Administration and Test Analysis Reporting Analysis PER Existing CFAA HIPC AAP Assessment ROSC Instruments CPAR EC Audit IGR Guidelines, Questionnaires, and Checklists Methodology 2003 Bank-IMF paper on collaborative public expenditure work (World Bank and IMF 2003), partly in response to the strong interest of other donors and stakeholders, and partly in response to PEFA's work on this study. The following paragraphs offer some preliminary thoughts on the main issues involved and describe the conceptual and practical questions that need to be answered. Under a programmatic and modular approach there would be strong emphasis on examining the public expenditure management system as a whole and identifying links and relationships between its parts. Each coun- try would be encouraged to develop a strategic plan for reforming public expenditure management, linked to its Poverty Reduction Strategy Paper 70 Assessing and Reforming Public Financial Management and agreements with donors for financial and technical support. Table 2 summarizes along various dimensions the main differences between the currently dominant and proposed new approach to public expenditure work. It would be important to ensure that the use of modules does not fragment assessment work and thus run directly counter to the basic idea of integration. Table 2: Alternative approaches to public expenditure diagnostics and reform Current approach New approach Entry Point Donor-required diagnostic. Country-led process. Vehicle Donor-financed report. Donor-financed expertise supporting country process. Champion Donor. Senior policy official initially, and technical staff as well. Timing Commonly provided off-cycle or Advice as requested, feeding into after opportunity for maximum country process and schedule. impact has passed. Output Single, large report; numerous Separate reports or papers to meet recommendations, frequently client needs. neither sequenced nor prioritized. Engagement Discontinuous, frequently with Continuous, frequently with same wide variety of donor counter- advisor or donor counterpart. parts at the technical level. Priorities Donor driven, frequently Country-determined, though donors following fashion of the day may provide advice. (environment, gender, decen- tralization, empowerment, private sector, social safety nets, debt sustainability, etc.). Credit for Donor. Client. success Capacity Limited to involvement in data Direct capacity building through building gathering and report production. processes, on-the-job training, and participation. Ownership At best, country co-production Direct through country formulating of report; frequently, client plays recommendations and making passive, secondary role in process, choices, or joint development of if at all. Prescriptive solutions conclusions and reforms. supplied to client. Recommendations and Issues for Further Consideration 71 Table 2: (continued) Current approach New approach Reform strategy Donor determined; feasibility Country specified; feasibility externally assessed, frequently inherent in government priority ignoring limits of political and strategy setting. economy. Client may object to some recommendations on feasibility grounds, but donors may counter with pressure for "tough measures." Follow-up Technical advice through loans, Country determined; donor support grants, trust funds, or staff provided where country requests; time; frequently stand-alone donors can evaluate success or operations. Loan conditionality, progress through annual process. including triggers for release May feed into investment loan of aid funds. conditionality or trust-fund financed support, but as requested by government. Participation Disconnected participation of Government-wide through budget power ministries, line ministries, process (e.g., core ministries and as report scope dictates; may sector ministries); encompassing involve civil society. Frequently policy officials (via cabinet or equiv- focuses on technical staff, and alent as a collective entity), and line Ministry of Finance as primary staff. May involve civil society counterpart; frequently leaves dialogue. out policy officials, or cabinet as collective decision-making body. Donor Through participation in report Through participation in process; coordination production, or commenting on ascribing to outcome of report; or separate donor government-led process. coordination group. Note: The authors are grateful to Bill Dorotinsky, World Bank, for suggesting and preparing this table. The goal of integrated assessments could be achieved by using the mod- ules to prepare a standardized overview of public expenditure management issues--one that brings together relevant technical analysis and is seen by donors as a comprehensive assessment of the environment in which they are providing budget support. Standardized assessment overviews would be equally valuable to recipient countries, providing authoritative information on strengths and weaknesses in public expenditure management and a foundation on which to develop and implement action plans for reforming 72 Assessing and Reforming Public Financial Management legal and institutional structures and building capacity. Standardized assess- ments could be updated periodically and could also include performance indicators, like those in HIPC AAPs, that could be monitored. Under such an approach existing fiduciary assessments (through PERs, CFAAs, CPAR, and Fiscal ROSCs) would be replaced by streams of work-- part diagnostic, part capacity building--tailored to country requirements, sequenced over time, and satisfying donors' fiduciary requirements. To illustrate: a country's five-year strategy for public expenditure management might include, in addition to preparing and regularly updating a standard- ized assessment, assistance from the Bank, IMF, or other donors in con- ducting a detailed assessment of accounting and audit in the first year of the strategy and of procurement practices in the second, advice on establishing a modern treasury and financial management information system and a review of institutional and governance issues in the first or second year, assistance in strengthening the management of government financial and personnel records in the first to third years, and assistance in training finance officials in the second to fifth years. In developing this new programmatic approach, many important issues will need to be addressed, including: · The scope and nature of the approach and of the standardized assess- ment overview, and their relationship with existing instruments and approaches. · The extent to which the scope and content of the data collected and ana- lyzed would be standardized or tailored to country circumstances and requirements, and the frequency of data collection. · The agreements under which donors would collaborate in providing information and analysis--for example, the relationship between the Bank's work on budget execution (through CFAAs and CPARs) and EC compliance tests. · How the transition to the programmatic and modular approach would be designed and managed. · The operational, budget, and staffing implications of the approach for the Bank and other development agencies, and how these should be managed. · The strategy and modalities for disseminating information on the new instruments, operational procedures, and assessment reports to govern- ments, donors, and other stakeholders. Recommendations and Issues for Further Consideration 73 · The quality control mechanisms for standardized assessment overviews, involving both donors and recipient governments, perhaps using QAG as a basis for conducting quality reviews. NOTES 1. Despite the questionable need for them, three new instruments have recently been proposed. The first is an IMF Fiscal Management Assess- ment that would focus on emerging economies and that has already been piloted in Turkey. The second is a country-led peer review mechanism under the New Partnership for Africa's Development (NEPAD), which is to cover all aspects of governance. In addition, the World Bank is piloting a Fiduciary Review that focuses on corruption in Bank-financed projects but also overlaps with CFAAs, CPARs, and Institutional and Governance Reviews (IGRs). 2. See http://wbln1023.worldbank.org/Caw/CawCover.nsf/homepage? OpenForm. 3. An agreement between multilateral development banks to use CPAR as the basic tool for conducting assessments is contained in World Bank 2002c. 4. There are a few exceptions. For example, the IMF­World Bank Financial Sector Assessment Program (FSAP) was created after the East Asian financial crisis in the late 1990s. The program conducts financial sec- tor assessments using joint teams and producing joint reports. HIPC expenditure tracking is another example of successful IMF-Bank collabora- tion, though on a more modest scale. Another is Bank-IMF collaboration on Poverty Reduction Strategy Papers. 5. In the World Bank, "Core Economic and Sector Work" is the primary country-based analytic and advisory work, and includes the three instruments under discussion (PER, CFAA, and CPAR). See the World Bank's Opera- tional Manual online at: http://wbln0018.worldbank.org/institutional/ manuals/opmanual.nsf/textonly. The integration of fiduciary assessments is also enunciated in the "Country Financial Accountability Assessment: Guidelines to Staff." Operations Policy Core Services Network, Financial Management Sector Board (Washington, D.C.: The World Bank, May 27, 2003). 6. Figure 2 shows the programmatic/modular approach as a linear struc- ture, running neatly from bottom to top. In practice the process of assess- 74 Assessing and Reforming Public Financial Management ment and reform is repetitive and interactive, with multiple feedback loops and iterations between the government, donors, and other stakeholders. Assessment work and related technical assistance and capacity building are generally not carried out as single blocks of activity, but rather in stages-- sometimes within a coherent overall reform strategy and work program that build on and reinforce each other through dialogue between govern- ment and donors. The circle around figure 2 illustrates this point. A Concluding Word This study and its findings and recommendations are intended to set in motion a debate between the partners in the PEFA program, other devel- opment agencies and international organizations, client governments, and other stakeholders. Because a well-managed budget process is essential for the efficient use of public resources, effective delivery of public services, and reduction of poverty, the views of all stakeholders should be taken into account at an early stage. If the study's findings and recommendations are accepted, there will be a need to translate them into changes in the operational rules, practices, and procedures of the agencies concerned. Organizational structures, manage- ment processes, staffing requirements, training mechanisms, budget alloca- tions and internal incentives will need to be reviewed, as will arrangements for increasing cooperation and coordination between development agen- cies, governments, and other stakeholders. As noted, some potentially important reforms in these areas are already being considered by the World Bank, IMF, and other agencies--including proposals for enhanced collabo- ration between the World Bank and IMF on public expenditure work, reform of the Bank's procurement assessment function, and stronger col- laboration between the Bank and the European Commission on CFAAs and compliance tests. It is hoped that this study will make a useful contri- bution to these and other emerging reforms. 75 Annex 1 Scope and Application of the Main Instruments This annex expands on the summaries in the main text--providing a more complete description of the main instruments that development agencies use to assess public expenditure, procurement, and financial accountability systems in client countries. WORLD BANK PUBLIC EXPENDITURE REVIEWS Public Expenditure Reviews (PERs) are the World Bank's traditional and best-known public expenditure instrument, and they have a long history. In the 1970s, when the Bank provided only investment lending, Public Invest- ment Reviews were its main analytical instrument. In the early 1980s PERs were introduced, and their use accelerated in the mid-1980s after the Bank began providing nonproject lending. Because this new type of lending was effectively budget support, the Bank considered it mandatory to examine the complete budget in recipient countries. The first PERs emphasized analyzing the quantity of expenditure and its sectoral and economic allocations. In addition to examining macroeco- nomic issues and the budget, many of these PERs focused on specific sec- tors (such as education or health) or significant expenditures (such as the civil service wage bill). Recommendations were made about issues such as 77 78 Assessing and Reforming Public Financial Management the composition of investment spending, the share of education expendi- ture, and the size of the wage bill. In the 1990s the emphasis of PERs began to shift to the efficiency of expenditure and to the quality of budgeting and financial management, reflecting the Bank's increasing focus on governance and corruption. Campos and Pradhan (1996) and the Bank's Public Expenditure Management Handbook (World Bank 1998b) contributed to the incorporation of institutional and governance issues in PERs. Thus PERs began studying whether formal rules and procedures for financial and economic management are actually applied. They also began examining institutional constraints to rigorous, transparent budgeting and financial management. The growing influence of institution- al economics and of the so-called new public management played a signifi- cant role in this evolution of the Bank's thinking (see World Bank 1997a, b). PERs may address a range of issues and may serve many objectives. The World Bank's Website describes them as "comprehensive macro reports with a mandate to focus on the efficiency and effectiveness of resource allocation." While they typically review a country's public expenditure policy and man- agement, they also tend to address macroeconomic issues such as fiscal sus- tainability and may include significant coverage of government revenue. PERs have two main objectives. The first is to strengthen budget analy- sis and processes to achieve a better focus on growth and poverty reduction. The second is to assess public expenditure policies and programs to meet donor requirements and provide recipient governments with an external review of their policies. The balance in emphasis on these objectives depends on the country. In addition, PERs may address the efficiency of expenditure in major sectors such as health or education, or issues such as civil service reform, fiscal decentralization, and service delivery. The Public Sector Governance Board is responsible for the overall development of PERs. In management terms, country directors and PREM sector managers in the Bank's regions are responsible for the quality of indi- vidual PERs. As with all of the Bank's economic and sector work, task man- agers must make provisions for quality assurance--including peer reviews--when planning PERs. The peer review process has an important influence on the design and coverage of PERs. PER preparation and approval begins with a concept note describing the review's substantive focus and content--including how the proposed activ- ity fits into the overall strategy for addressing the quality of the budget and related public resource management issues. This description typically includes a detailed statement of the issues to be reviewed, an outline of the Annex 1: Scope and Application of the Main Instruments 79 report, the target audience and dissemination strategy, the participatory approach to be followed and the involvement of stakeholders, the role of other external partners, data issues, criteria for evaluating the success of the activity, and proposed quality assurance measures. The reporting and dis- semination strategy is determined by the type of PER and the nature of the involvement by the government and other stakeholders. The multipurpose function and adaptability of PERs are reflected by the fact that there are no formal guidelines for them, though draft guidelines are proposed in World Bank (2001). The draft guidelines reflect current thinking about PERs and their uses at a time when the Bank is seeking increased participation by clients and other stakeholders--in both public expenditure work and sector work more generally. As part of the develop- ment of a proposed new approach to public expenditure work, the Bank's Public Sector Governance Board recently decided that these guidelines should be revised and issued formally. The draft guidelines propose three prototype PERs involving different approaches to client participation: · In-house PERs, for which the Bank and consultants do the data gather- ing and analysis. Interaction with clients is limited and focuses on fol- low-up to the final product. · Bank-led participatory PERs, for which the Bank manages the process with substantial government participation in the analysis. Other stake- holders and donors may also be involved. · Joint or government-led PERs, with active Bank participation. The first and second prototypes would look much like traditional PERs. The third would be less traditional--consisting, for example, of a summa- ry assessment of the budget management process. Most PERs today fall into the second or third category, and the draft guidelines stress that while client-led efforts may be of lower quality, they are likely to have a greater impact if they respond to client concerns. The guidelines also state that tra- ditional in-house PERs are justifiable only in special circumstances, such as when a client with sound public expenditure management seeks indepen- dent advice or when the Bank is familiarizing itself with a new client or reengaging in a country where it has been inactive. Perhaps the best example of the third type is the annual PER that has been conducted by the government of Tanzania since fiscal 1998 (other African countries that have adopted a similar approach, or are considering doing so, 80 Assessing and Reforming Public Financial Management include Ethiopia, Kenya, and Uganda). In this case the two main objectives of the PER are to support the budget process through macroeconomic and sectoral studies and to provide the government with a peer review of fiscal issues. The review is led by the Bank and covers macroeconomic and fiscal developments, strategic allocation issues, and budget management questions. Many other donors and stakeholders are involved in the process, receive the findings, and participate in review meetings and seminars. In addition, Tan- zania's annual consultative group meeting is part of the budget preparation and PER process. This type of government-led PER is radically different from the traditional approach in that it becomes an integral part of the annu- al budget cycle and is driven by government priorities and concerns. WORLD BANK COUNTRY FINANCIAL ACCOUNTABILITY ASSESSMENTS Country Financial Accountability Assessments (CFAAs) are the Bank's main diagnostic tool for public financial management and accountability. Their objective is to enhance the Bank's knowledge of financial accountability arrangements in client countries. The assessments support both the Bank's: · Fiduciary responsibilities, by helping to identify risks in the use of Bank funds. · Development objectives, by designing and facilitating--in close cooper- ation with the national authorities--programs to improve financial accountability. Given the growing share of adjustment and programmatic lending in the Bank's overall lending, the Bank must have a sound understanding of the financial accountability environment in which its funds are spent. Such knowledge enables the Bank to better assess risks and advise countries on how to strengthen financial management. CFAAs also provide guidance on whether a country's expenditure management system can be used to man- age Bank-supported projects. Formal guidelines for CFAAs were issued to Bank staff in July 2000 and updated in May 2003. Like all economic and sector work, CFAAs are sub- ject to requirements for formal review and clearance--in this case by the anchor of the Financial Management Network, network advisers in the Bank's regions, and country directors. The updated guidelines include a Annex 1: Scope and Application of the Main Instruments 81 section on quality and also refer staff to the internal quality section of the Quality Assurance Group's questionnaire for economic and sector work. As with all such work, task managers must make provisions for quality assur- ance, including peer reviews, when planning CFAAs. The updated guidelines define fiduciary risk in greater detail than do the existing ones. They indicate that a CFAA does not try to assess value for money in public spending, nor does it assess the level of financial or sover- eign risk--that is, the risk that Bank funds will not be repaid on time or at all. Rather, a CFAA's concern is whether Bank funds are spent on authorized or intended purposes, as expressed in the country's budget. The key compo- nents of this type of risk are that the budget is not implemented as passed and that the budget does not cover significant areas of government activity. Like related sector work, such as PERs, the context and processes for preparing CFAAs have evolved and now give more attention to consultation and collaboration with recipient countries. CFAAs are closely linked to Coun- try Assistance Strategies (CASs), and Bank policy requires a through discus- sion of fiduciary issues. Information from CFAAs significantly influences deci- sions reflected in CASs about the amount and nature of lending to individual countries. This link can create tension between the need to promote country ownership of CFAAs and their use as a fiduciary instrument by the Bank.1 The Bank has decided that current (less than five years old) diagnostic assessments should be completed for all major borrowers by the end of fis- cal 2004. Moreover, Poverty Reduction Support Credits require ex ante fiduciary assessments--and this is interpreted as a prima facie requirement for CFAAs. It is expected that the forthcoming revised guidelines for adjustment lending will make ex ante fiduciary assessments, including CFAAs, a prerequisite for such lending. Between fiscal 1996 and 2001 about 20 CFAAs were completed. In fis- cal 2002 another 21 were completed. The anchor of the Financial Man- agement Network is preparing a program that will ensure that CFAAs are carried out in all client countries on a regular basis. Some will be done every two years, some every three years. CFAAs are carried out by Bank staff in cooperation with country officials, and the revised guidelines encourage the involvement of civil society, the pri- vate sector, and other nongovernmental partners. The guidelines describe in detail the process that staff should follow in planning CFAAs, including finding the resources, establishing the role of the client and other donors, performing initial desk studies, conducting the planning mission, preparing an initiating memorandum or concept paper, and conducting the review meeting. 82 Assessing and Reforming Public Financial Management CFAAs are increasingly being carried out as combined or parallel products with related Bank diagnostic instruments--for example, with PERs (as in Mozambique, the Philippines, and Turkey) and Institutional and Governance Reviews (as in Bolivia and Peru). There is also increasing collaboration and coordination with Country Procurement Assessment Reviews (CPARs). In addition, a growing number of CFAAs are carried out in collaboration with other development partners, which reflects the international harmo- nization agenda being promoted by the Task Force on the Harmonization of Donor Practices (sponsored by the OECD's Development Assistance Committee) and the Harmonization Group (sponsored by multilateral development banks). This collaboration should reduce the transaction costs for client countries of multiple donor assessments. For example, CFAAs in the Bank's Latin America and Caribbean Region are generally carried out jointly with the Inter-American Development Bank. Other development partners that collaborate on CFAAs include the United Nations Development Programme, the African Development Bank, the Asian Development Bank, the European Community, and the Strategic Partnership with Africa, as well as bilateral donors such as the U.K. Department for Inter- national Development, the Norwegian Agency for International Develop- ment, and the Swedish International Development Authority. It is expected that the Bank's development partners will increasingly use CFAAs as a basis for their decisions on budget support. CFAAs can also increase the depth of work on public expenditure management--for example, by using EC audits and compliance tests as complementary sources of information (see below). The guidelines review other Bank and International Monetary Fund (IMF) instruments relevant to CFAAs and emphasize that the level of analysis in CFAAs will vary, reflecting country circumstances and the timing, scope, and coverage of other planned or recently completed analytical work. They also stress that the Bank is moving toward integrated fiduciary assessments and that this will increasingly affect the sequencing and timing of products. At present CFAAs are not public documents and are not released except with the agreement of the government concerned. However, governments are encouraged to disseminate CFAA reports. The guidelines stress that staff preparing a CFAA should strive to obtain empirical evidence of what is happening on the ground--as opposed to what is contained in formal rules and regulations--and that conducting this reality check is difficult. Discussions with the national audit agency and the use of questionnaires are recommended. The anchor of the Financial Man- agement Network is developing a standard questionnaire for CFAAs. Annex 1: Scope and Application of the Main Instruments 83 A CFAA typically reviews the following issues in the public sector: · Budget preparation procedures--particularly comprehensiveness, real- ism, and classification. · Budget execution and monitoring (including cash management and internal control). · External fiscal reporting and transparency. · External auditing and legislative follow-up. · Financial management staffing capacity. Past CFAAs have also reviewed corporate governance and financial reporting in the private sector, including the accounting profession. In the future it is expected that these issues will instead be addressed through IMF Reviews of Standards and Codes (ROSCs) for private sector accounting and auditing. But issues related to the accounting profession may be covered in CFAAs if they are relevant to the public sector's need for accounting profes- sionals and to the reporting and accountability of state enterprises. The Bank considers follow-up an integral part of the CFAA process. The updated guidelines stipulate that from the outset the Bank and the govern- ment should agree that the CFAA is targeted at remedial action. The docu- ment should include an action plan of sequenced activities, and should define the resources and technical assistance required. Specific benchmarks or indi- cators should be developed to assess performance. The action plan should be agreed by the government and its key elements reflected in the CAS. CFAA guidelines suggest that other Bank instruments, especially PERs and Institutional and Governance Reviews (IGRs), should be the primary source of information for institutional and governance aspects of public expenditure management. But the guidelines also recommend that CFAAs include a section on the legal and institutional framework for public expen- diture management (annex A, p. 17) and identify aspects of the public expenditure management system that might facilitate corruption or in which improvements might reduce it (annex C, p. 30). WORLD BANK COUNTRY PROCUREMENT ASSESSMENT REPORTS Country Procurement Assessment Reports (CPARs) were introduced in the late 1980s. In 1998 their focus and objectives were redefined when the assess- 84 Assessing and Reforming Public Financial Management ment process was redesigned to reduce the focus on procurement in devel- opment projects and increase the emphasis on diagnosing national procure- ment systems and generating dialogue with governments about needed reforms. The main purpose of CPARs is to determine the need for and devel- op action plans to improve procurement systems. Specific elements include: · Providing a comprehensive analysis of public sector procurement sys- tems--including the legal framework, organizational responsibilities, and control and oversight capabilities. · Conducting a general assessment of the institutional, organizational, and other risks associated with procurement--including identifying procure- ment practices that are unacceptable in Bank-financed projects. · Developing a prioritized action plan to bring about improvements. · Assessing local private industry's participation in public procurement and the adequacy of commercial practices related to it. To ensure country ownership, CPARs are launched with the agreement of the government concerned and often involve its active participation. The guidelines for CPARs suggest techniques for demonstrating the benefits that can arise from the procurement review and thereby foster government support. In addition, assessment teams should strive to involve the business community and other nongovernmental partners. The timing and scope of other diagnostic instruments help determine the content of individual CPARs. The World Bank plans to increasingly integrate its economic and sector work on accountability, risk assessment, and public expenditure and to better coordinate these analyses with those of other donors to avoid duplication, overlap, and high transaction costs. Joint efforts have already been undertaken, for example, in Bosnia and Herzegovina, the Philippines, and Turkey. Because sound procurement sys- tems are a priority interest for all donors, the Bank encourages its staff to collaborate with other agencies, and the CPAR guidelines spell out the var- ious forms that such partnerships may take. A country's CPAR, like its PER and CFAA, is closely linked to its CAS. Bank policies indicate that CASs should include thorough discussions of fiduciary issues.2 The CPAR is the Bank's primary product for procurement issues. The Procurement Board is responsible for the overall development of CPARs, and the anchor of the Procurement Network reviews initiating concept memorandums and may provide other assistance. Country directors Annex 1: Scope and Application of the Main Instruments 85 and regional procurement advisers work together to schedule CPARs based on country needs and regional priorities. These staff are also responsible for the quality of individual CPARs. CPAR guidelines include a section on qual- ity and emphasize the need to base the analysis on a strong analytical frame- work and rigorous empirical evidence; they also refer staff to the internal quality section of the Quality Assurance Group's questionnaire on econom- ic and sector work. As with all such work, task managers must make provi- sions for quality assurance, including peer reviews, when planning CPARs. CPAR guidelines describe in detail the main steps for developing initiat- ing concept memorandums and for conducting review meetings and peer reviews. They also explain how to ensure adequate staffing and resources and define the dual purposes of the exercise: analyzing a country's procure- ment system with a view to proposing concrete measures for improvement and identifying risks to the Bank's loan portfolio. The Bank encourages governments to involve a wide range of stake- holders in the preparation of CPARs, to disseminate CPAR findings to these stakeholders, and to involve them in the design of follow-up action plans. Governments are also encouraged to circulate draft CPARs widely before giving their reactions. World Bank policy is that CPARs are publicly disclosed unless governments object. CPAR guidelines consider the follow-up phase critical to determining the report's impact. They stipulate that the Bank and the government should agree before launching a CPAR that its findings will form the basis for remedial actions in the case of significant deficiencies. The review should lead to agree- ment on the actions to be taken, a timetable for implementation, and responsi- bilities for executing the program, and these three elements should be incor- porated in the CAS. Procurement reform typically involves activities such as adjusting the legal framework, strengthening institutions, providing training, and improving records management. The action plan should be monitored by the government and donors using performance indicators for the public pro- curement system; CPAR guidelines give examples of suitable indicators. IMF REPORTS ON THE OBSERVANCE OF STANDARDS AND CODES OF FISCAL TRANSPARENCY The IMF adopted a Code of Good Practices on Fiscal Transparency in 1998 and approved an updated version in March 2001 (see IMF 2001). The code is based on several key requirements for transparency: 86 Assessing and Reforming Public Financial Management · Government roles and responsibilities should be clear, including the legal and administrative framework for fiscal management. · The government should provide the public with complete information on its past, present, and projected fiscal activities. · Budget preparation, execution, and reporting should be conducted in an open manner. · Fiscal information should meet international standards for data quality and be subjected to independent assurances of integrity. The code describes the principles and practices that governments should follow to meet these requirements. Guidance on implementation of the code is provided by a manual on fiscal transparency and by a questionnaire that helps gauge country practices relative to the standards of the code. Reports on the Observance of Standards and Codes (ROSCs) have been developed as part of a World Bank­IMF initiative to promote standards and codes. ROSCs provide information on implementation of standards to assist the IMF's regular surveillance (Article IV) consultations with member countries. The report on fiscal transparency--the Fiscal ROSC--has two main components. The first is a description of country practices relative to the good practices defined for each of the 37 elements of the code. The second is a staff commentary, giving the IMF's overall assessment and recom- mending possible improvements. Completion of a Fiscal ROSC is volun- tary. Once a government decides to participate, it fills out the question- naire. Based on the answers and on discussions with the authorities, IMF staff prepare a draft ROSC, which is discussed with the authorities before being finalized. This approach strives to stimulate government ownership of and interest in the process and its results. Participation in a Fiscal ROSC helps national authorities identify weak- nesses. Government approval and publication demonstrate commitment to increasing fiscal transparency. Published results are useful for preparing and updating other fiduciary instruments used by donor agencies. They may also be used by the IMF and the Bank in the development of their lending and technical assistance programs. A Fiscal ROSC is a carefully targeted instrument with a very specific focus, as reflected in the detailed instructions in the manual on fiscal trans- parency. The manual emphasizes that a Fiscal ROSC should be limited to fiscal transparency issues and should not deal with the efficiency or effec- Annex 1: Scope and Application of the Main Instruments 87 tiveness of fiscal management (though this stipulation is not always strictly observed in practice). Fiscal ROSCs should, however, refer to relevant laws and indicate the extent to which these laws are observed. To the extent possible, the IMF coordinates the preparation of Fiscal ROSCs with that of CFAAs and other World Bank products. Most coordi- nation has involved the exchange of information on mission schedules and the exchange of draft reports for comment. In addition, some experimental missions have been conducted involving coordinated field work (as in Bangladesh and Mauritania). If country authorities agree, the final assessments of Fiscal ROSCs are made publicly available on the IMF Website. By July 2003, 59 Fiscal ROSCs had been completed--meaning that they had been finalized with the authorities and sent to the IMF Board--and 52 had been published on the Website. The section of Fiscal ROSCs containing the IMF staff commentary identifies priorities and offers recommendations for improving fiscal trans- parency. Fiscal ROSCs usually do not develop action plans, though they have occasionally done so with participation from country authorities. Where transparency weaknesses appear to have significance for IMF pro- grams or IMF country policy analysis, improvements in these areas may be included in program conditionality. Improving transparency in the agreed areas often requires addressing underlying institutional weaknesses over the medium to long term. The IMF or other agencies may provide technical assistance to address transparency and underlying institutional weaknesses. Once a Fiscal ROSC is completed, IMF policy is that brief updates should be undertaken as part of regular surveillance consultations. These updates are also published on the IMF Website. Thus Fiscal ROSCs allow for continual monitoring of countries' implementation of the fiscal trans- parency code. In addition, it is envisaged that Fiscal ROSCs will periodi- cally be completely revised and new assessments published. WORLD BANK­IMF PUBLIC EXPENDITURE TRACKING ASSESSMENTS AND ACTION PLANS FOR HEAVILY INDEBTED POOR COUNTRIES In 1996 the Bank, IMF, and numerous bilateral donors introduced the Heavily Indebted Poor Country (HIPC) initiative to reduce the external debt of the world's poorest, most indebted countries.3 Following a 1999 review, the initiative was enhanced to provide faster, deeper debt relief to 88 Assessing and Reforming Public Financial Management more countries. Of the 42 countries that qualify for the initiative--as deter- mined by their debt sustainability--26 have received debt relief.4 The enhanced HIPC initiative emphasizes that recipient countries should have sustained implementation of integrated programs for poverty reduction and economic reform (see http://www.worldbank.org/hipc/about/hipcbr/ hipcbr.htm). In spring 2000 the boards of the Bank and the IMF asked the institutions' staff to address concerns about how resources freed under the initiative are used. (Such funds are intended to be used for poverty reduction, augment- ing--not substituting for--a country's spending in this area.) In response a joint initial desk assessment was conducted in fall 2000 on 25 HIPCs that had reached the decision point or were expected to do so soon. Joint Bank- IMF teams developed a questionnaire to assess the countries' ability to track poverty-reducing public spending, including funds freed by debt relief. The questions were designed to identify weaknesses in public expenditure man- agement to guide the development of performance indicators. HIPC track- ing is innovative because, rather than assessing every aspect of public expen- diture management, it takes a systems approach--identifying important indicators that reflect a system's overall capability and performance. If a country scored poorly, its system for public expenditure manage- ment may be unable to plan, execute, or accurately report poverty-reduc- ing public spending. By defining minimal standards for public expenditure management, the assessment helped prioritize needs in upgrading capacity. The findings were summarized through 15 benchmarks covering budget formulation, execution, and reporting. The focus was not on the use of tar- geted debt relief (HIPC initiative) funds but on the quality of public expen- diture systems for all budget resources. The March 2002 Bank-IMF board paper concluded that 15 of the 24 HIPCs assessed required substantial upgrading in their public expenditure management systems, 9 required some upgrading, and none would be able to carry out satisfactory tracking with a small amount of upgrading (World Bank and IMF 2002). The initial desk assessments were finalized in fall 2001. The boards asked Bank and IMF staff to continue close coordination, to help country authorities develop action plans for the short and medium terms, to identi- fy gaps in donor assistance supporting those plans, to ensure that all gaps were filled by the Bank, IMF, or other donors, and to provide periodic progress reports to the boards. The March 2003 board paper (World Bank and IMF 2003b) reiterated that the HIPC process would be conducted based on four main principles: Annex 1: Scope and Application of the Main Instruments 89 · Ultimate responsibility for tracking poverty-reducing spending lies with the countries. · All poverty-reducing spending should be tracked. · Poverty Reduction Strategy Papers provide the basis for identifying all relevant poverty-reducing programs. · Where systems are weak, countries should establish short-term "bridg- ing" mechanisms to facilitate medium-term tracking. HIPC AAPs are becoming accepted practice and provide a useful com- plement to the Bank's other economic and sector work (PERs, Public Expenditure and Institutional Reviews, CFAAs, CPARs). All the HIPCs have endorsed AAPs, and Bank and IMF staff are encouraging country authorities to incorporate tracking of poverty spending into their Poverty Reduction Support Credits and thereby mainstream the approach. The formal requirements for HIPC AAPs involve process requirements, which are evolving (see below), and instrument requirements, which are made up of the guidelines, the questionnaire, and a set of action plans. While the revised guidelines are still being finalized by the Bank and IMF, the 2001 and 2002 board papers reviewed progress and led to gener- al agreement on the approach. The boards requested that new tracking assessments be carried out for all new or planned HIPCs and that an updat- ed table be included in all adjustment loan documents going to the boards of the Bank and IMF showing the status of the action plans. HIPC AAPs are not designed to be self-assessments; as with Fiscal ROSCs, the ques- tionnaire and short-form report could be sent in advance of the related mis- sion. HIPC assessments are not audits but rather strive to help countries manage their resources well and overcome a key constraint: lack of capaci- ty that undermines effective public expenditure management, including the ability to track poverty-related expenditures. Bank and IMF staff agreed that the organization and conduct of the agreed assessments be evenly divided and led by joint Bank-IMF missions following the guidelines issued in April 2001. The Bank-led HIPC AAP missions were carried out by the Bank's Regions and combined with other missions. For the IMF-led missions, draft procedures were prepared for combined ROSC-HIPC missions, and missions were either by IMF's Fiscal Affairs Department or area depart- ments. Given the tight schedule imposed to finish the final assessments in 2001 and report to the boards by March 2002, systematic and disciplined 90 Assessing and Reforming Public Financial Management planning and implementation were undertaken, with progress reviewed every two weeks. This time pressure meant that while Bank-IMF coopera- tion was very close, it was usually not possible to involve bilateral agencies in these assessments. Follow-up is an integral part of the exercise, and the Bank, IMF, and country authorities should seek agreement on: · A draft schedule over the next three years for upgrading tracking capacity in light of planned technical assistance and the country's absorptive capacity. · An agreed action plan for improving public expenditure management, including identification of existing or additional technical assistance. Another comprehensive round of assessments is planned before the end of 2004 and will be used to prepare a joint Bank-IMF board paper. New draft guidelines have been prepared and circulated for comment. Updates will be required on an ongoing basis, while assessments of new countries should be completed by mid-2004. Updates and new assessments will need to be agreed with the governments concerned and between the Bank and IMF. In most cases the updates and assessments can be conducted as part of missions already planned. EC AUDITS Structural adjustment support provided by the European Commission has traditionally been subject to ex post control through an audit procedure. With the recent shift to providing aid through direct budget support, greater emphasis has been given to ex ante assessment of public expendi- ture management systems in beneficiary countries and to monitoring and evaluating improvements over time. EC audits of the use of its program aid must be viewed in the context of the evolution of its structural adjustment support. During the 1990s the European Commission provided structural adjustment support in two forms: in 1991­96 through general or sector import programs under the 7th European Development Fund, and in 1996­2000 through structural adjust- ment support introduced under the 8th European Development Fund. Although general and sector import programs remained an important source of EC program aid, structural adjustment support quickly became a key source of EC assistance. Annex 1: Scope and Application of the Main Instruments 91 While general and sector import programs were designed to strengthen weak balance-of-payments positions, structural adjustment support provid- ed budget aid to support sound macroeconomic management and imple- mentation of economic and social reforms. The funds were targeted and made available in two steps. First, a tranche in the relevant foreign exchange was deposited into a government account at the central bank. Second, funds were converted into local currency and deposited in a trea- sury double-signature account. Use of the funds required the authorization of the head of the EC delegation and the government official nominated as the national authorizing officer. Counterpart funds generated by structural adjustment support are considered targeted budget support aimed primari- ly at protecting social expenditures in the face of stabilization programs developed by the World Bank and IMF. Although counterpart funds are the property of the beneficiary country and responsibility for their management and use lies with the national authorizing officer, the European Commission has a control responsibility through its co-signature on the account containing the funds. These roles and responsibilities are set out in broad terms in financing agreements and implementation agreements, with further details provided in a memoran- dum of understanding, implementation agreement, or framework of mutu- al obligations signed by the European Commission and the national authorizing officer. Financing agreements, signed by the EC commission- er and the national authorizing officer, constitute the legally binding bilat- eral agreement, while implementation agreements define responsibilities for managing, monitoring, and controlling counterpart funds. The use of EC structural adjustment support has traditionally been audited annually by independent auditing firms. The purpose of these audits is to determine whether the use of funds is consistent with their objectives, as set out in the financing agreements and implementation agreements. With the evolution of EC structural adjust- ment support, the purpose of and approach to auditing and controlling the use of EC resources have also evolved. Since 1992 EC audits have been car- ried out in 32 countries, including audits of General Import Programs, tar- geted budget aid (including food support), and internal debt and institu- tional assessment of financial control procedures and agencies in some African, Caribbean, and Pacific countries. In the case of general and sector import programs the audits focused on the use of foreign exchange and generated counterpart funds, particularly the legality and regularity of expenditures. Many audit results were used to impose corrective measures 92 Assessing and Reforming Public Financial Management and sanctions on the countries concerned, including suspensions of struc- tural support, based on identified irregularities in managing the foreign exchange or counterpart funds. With the shift toward targeted budget support, EC audit procedures were adjusted to reflect the fact that use of generated counterpart funds became subject to national budget rules and procedures. Hence a more com- prehensive approach to audits was developed to provide a broad evaluation of beneficiary governments' public financial management systems, with a focus on the legal framework, budget execution, procurement, and financial accountability and control. In addition, materiality checks were introduced in EC audits to assess the value for money of the expenditures concerned. The methodology of the EC audit approach developed gradually, building on the principles and methodology of the UNDP CONTACT instrument (see below) and on lessons from previous audits. However, the evolved EC audit methodology has not been recorded in a formal document. In response to proposals by the European Parliament on the use of EC development resources, the European Commission committed to carrying out a series of special multiyear audits of budget support and food aid in various African, Caribbean, and Pacific countries. These audits were man- aged by the Commission's Internal Audit Service and conducted under con- tract by the French firm 2AC between 1998 and 2002. The audits were based on methods developed since 1996 in auditing the use of generated counterpart funds in line with national budget rules and procedures. 2AC examined whether the counterpart funds held in double-signature accounts were used as agreed in the financing agreements and implementation agreements in terms of "eligibility" in line with mutually agreed purposes and "conformity" with government budget rules. Follow-up audits were carried out using a sample of transactions to test the initial assessments of countries' public expenditure management systems. In conducting these audits, the Commission started by examining the quality of the systems in place and the legal framework (systems audit), ana- lyzed transactions based on a representative sample (financial audit), and looked at the efficiency and effectiveness of the systems (performance audit). The audits identified major weaknesses in public expenditure man- agement in the beneficiary countries that had not been apparent before, including a lack of conformity with prevailing rules, excessive payments for goods and services procured, weak administrative arrangements for records management, and inadequate institutional arrangements for internal con- trol and external audit. If ineligible expenditures were found and agreed by Annex 1: Scope and Application of the Main Instruments 93 the Commissions and the country, the amounts involved were redeposited into the double-signature account and kept for use during the next budget cycle--subject to confirmation that progress was being made in imple- menting an action plan of corrective measures. The emphasis was on fidu- ciary aspects: the eligibility and conformity of expenditures, and materiali- ty (for example, whether planned investments were made and goods and services were procured as intended). This approach had several weaknesses. Because memorandums of understanding included preconditions for disbursements, the release of funds was sometimes severely delayed. Although the annual audits pro- duced action plans to correct identified weaknesses, the Commission was not systematic in its follow-up--so progress was not carefully tracked by subsequent audits. And there was little positive development impact. The Commission recently adopted a new approach for providing budg- et aid. Within the context of the Poverty Reduction Strategy Paper, untar- geted budget support is being made available on a multiyear basis, subject to annual assessments of progress against performance indicators identified in the paper's priority areas--including the quality of public expenditure management. Annual budget support is divided into two tranches: a fixed tranche that reflects conformity with IMF conditions for macroeconomic management and a variable tranche based on agreed performance indica- tors for public expenditure management and social sectors. To support this new approach, the Commission has developed guide- lines for a new instrument, called the Public Financial Management Assess- ment, that consists of: · A desk review of existing diagnostic work on public expenditure man- agement--usually Bank and IMF reports (PERs, CFAAs, CPARs, HIPC AAPs, Fiscal ROSCs) and previous EC audits. This review includes identifying gaps in the existing work that should be addressed in follow- up work, including an EC compliance test (see below), and defining per- formance indicators to measure progress over time. · A compliance test that provides empirical evidence on the performance of the public expenditure management system, to supplement the findings and recommendations of the ex ante diagnostic work. For example, weak- nesses in public expenditure, procurement, and financial accountability identified in CFAAs and CPARs could be verified based on a sample audit in priority areas (such as health and education). The results of a compli- ance test can also provide a baseline for measuring progress over time. 94 Assessing and Reforming Public Financial Management · A review of government and donor action plans to strengthen public expenditure management and monitor improvements. · Establishment of performance indicators--in the context of national action plans or country assistance programs--that are monitored and evaluated with beneficiary governments and other donors. Annual updates of the compliance tests will contribute to this process (EC 2002). DFID ASSESSMENTS OF FIDUCIARY RISK The U.K. Department for International Development (DFID) recently decid- ed to channel more of its resources through direct budget support. But while it is highly committed to this initiative, DFID realizes that in many cases the con- ditions are not yet appropriate. As a result only about a quarter of DFID fund- ing takes the form of direct budget support or sectorwide approaches. To improve its ability to conduct fiduciary risk assessments, DFID has prepared a series of internal papers, including "Managing Fiduciary Risk When Providing Direct Budget Support" (March 2002). The purpose of this paper and related documents is to provide guidance to DFID staff-- many of whom are not specialists in financial management--on how the public expenditure elements of fiduciary risks should be assessed and man- aged when DFID provides funds directly to recipient governments to be spent as part of their budgets. The paper also sets out DFID's approach to using additional safeguards to ensure that such budget support focuses on poverty reduction, and proposes directions for future collaboration with other development agencies to harmonize approaches. A fiduciary risk assessment must be completed before DFID can provide budget support. Thus such assessments are key inputs into DFID decision- making about many of its client countries. The timing of an assessment is determined by the needs of the relevant country desk. DFID assessments of fiduciary risk are mainly derived from data and analy- sis gleaned from other instruments such as CFAAs, CPARs, HIPC AAPs, and Fiscal ROSCs. The 2002 DFID paper defines fiduciary risks and states that assessments should focus on whether it is reasonable to expect that transferred resources will be used for the intended purposes and properly accounted for and that the expenditures will represent value for money. The paper recognizes that in many of the poorest countries that DFID supports, financial manage- ment and accountability systems fall below internationally accepted standards Annex 1: Scope and Application of the Main Instruments 95 but that direct budget support may be the most effective way to deliver assis- tance--provided that a thorough evaluation of fiduciary risk has been under- taken, the government has a credible program to improve public expenditure management, the potential development benefits outweigh the risks, and the risks are explicitly recorded as part of DFID's decisionmaking process. As noted, DFID's assessments rely heavily on the results of CFAAs, CPARs, and Fiscal ROSCs, and in some countries (such as Malawi) it has collaborated with the Bank in carrying out CFAAs. (DFID has also pro- duced an interim guide for staff on the conduct and uses of CFAAs; see DFID 2001a.) DFID staff are given a list of good practice principles and related benchmarks for assessments. Monitoring should be closely tied to the government's budget cycle, linked to the government's poverty reduc- tion strategy, and shared with other donors. In Uganda, for example, the government and DFID have reached an understanding on how fiduciary risk will be monitored over time based on four information sets: one annual expenditure tracking survey per sector, an annual review of the government's audited accounts, the outcomes of PER updates, and technical assistance and dialogue built around the govern- ment's public expenditure reform program. DFID expects that in most cases its fiduciary risk assessments will show that governments are working to address their weaknesses in financial man- agement. In such cases DFID is prepared to complement its budget sup- port with long-term capacity enhancement programs--including, in close collaboration with other donors, provision of technical assistance and train- ing. DFID will also combine such support with efforts to address some of the root causes of poor accountability and transparency, such as by empha- sizing the importance of "voice" mechanisms in the private sector and civil society, enhancing civil service incentives, and fighting corruption. DFID is undertaking political economy research to explore these issues further. In addition, DFID is increasing its resources for public expenditure manage- ment by training staff and hiring more advisers with relevant professional skills and experience. EC AND OECD SUPPORT FOR IMPROVEMENT IN GOVERNANCE AND MANAGEMENT IN CENTRAL AND EASTERN EUROPE Support for Improvement in Governance and Management (SIGMA) in Central and Eastern Europe--a joint initiative of the European Community 96 Assessing and Reforming Public Financial Management and OECD financed mainly by the Community--involves two separate instruments: baseline assessments and peer reviews. Baseline assessments use defined criteria to measure changes in public expenditure management sys- tems, with the results used by the European Commission in its monitoring of Central and Eastern European countries' progress toward achieving the nec- essary conditions for EU membership. Peer reviews help supreme audit insti- tutions and ministries of finance meet requirements for EU membership by developing their institutional capacity. Baseline assessments Baseline assessments describe and analyze changes in public management systems as a contribution to EC progress reports on countries that are can- didates for joining the EU. The assessments also offer recommendations on areas where systems require further strengthening. Baseline assessments have been carried out every year since 1999 and cover: · Civil service and administrative frameworks. · Public rxpenditure management. · Public internal financial control, including internal audit. · External audit. · Policymaking and policy coordination. · Public procurement. · Efforts to combat fraud (in 2001 only). The baseline assessments were designed for the 10 EU candidate coun- tries in Central and Eastern Europe (Bulgaria, Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, Slovak Republic, and Slove- nia, but in 2000 it was also applied in two additional Balkan countries (Alba- nia and the Former Yugoslav Republic of Macedonia) and may be extend- ed to others in that region. The methodology used in preparing the annual baseline assessments has three parts. First, an agreement is reached with the European Com- mission on the conditions that a country must meet to satisfy accession cri- teria for the subject areas listed above. These conditions--including legal, operational, and institutional capacities--are based on the acquis commu- nautaire and common European practices and take into account any Annex 1: Scope and Application of the Main Instruments 97 changes in the acquis since the previous year. The conditions are support- ed by detailed guides and questionnaires, specific to each subject area, used to measure administrative capacity in ministries, government agencies, and other public sector institutions. The questionnaires are completed with officials in the relevant government institutions and form the basis for the second part of the assessments: inten- sive information gathering missions carried out by SIGMA staff in preparing the assessment reports. The third part occurs after the reports have been drafted, when they are submitted to the countries concerned for factual veri- fication. The reports are then reviewed and calibrated by SIGMA staff to ensure consistency across countries and subject areas. The final reports are sent to the European Commission for use in preparing their annual reports and often inform decisions on pre-accession financial support and technical assistance provided by the Commission to support capacity building. Peer reviews of supreme audit institutions Peer reviews of supreme audit institutions began in 1998 when SIGMA was becoming increasingly involved in efforts by Central and Eastern European supreme audit institutions to develop their activities in line with good European audit practices. The first reviews--for Slovenia (1998), Latvia (1998­99), and Estonia (1999)--were conducted by SIGMA staff and expert practitioners from EU member states (usually in a team of three peers) and covered all aspects of each audit institution's functions. Since 1999 seven more peer reviews have been conducted. A peer review provides an independent assessment of the issues that a supreme audit institution needs to address to become more effective, prepare for EU accession, and adapt to EU auditing standards and good practices. The emphasis is on providing practical recommendations for change and on helping the institution and other interested parties understand the need for institutional development in the context of negotiating EU accession. A supreme audit institution must take the initiative to request a peer review, which has seven phases: · Determining the review's organization and stages. · Forming the review team. · Preparing the review. · Arranging for two one-week fact-finding missions. 98 Assessing and Reforming Public Financial Management · Writing the report and engaging in consultations. · Finalizing the report. · Identifying next steps. Annexes to the peer review guidelines explain how information should be collected (mainly through interviews and file examinations) and the technical analysis to be carried out. Questionnaires have been developed-- foe example, on parliamentary audit committees--to help obtain all neces- sary information. The SIGMA project manager is responsible for ensuring high-quality work, editing the final report, and preparing a summary that accompanies the report. The intended results include endorsement of peer review recommenda- tions by the supreme audit institution and agreement on further steps such as revising the audit law, establishing a strategic development and action plan, identifying requirements for further technical assistance, and intro- ducing specific benchmarks and standards. Peer reviews of public internal financial control systems All EU candidate countries must develop comprehensive, reliable systems for public internal financial control. Legislative frameworks have to be implemented, together with secondary legislation providing for the imple- mentation of new internal control capabilities that ensure the functional independence of internal auditors. These efforts are essential for strength- ening internal financial capacity prior to receiving pre-accession financial support and for receiving EC structural support after accession. Against this background, SIGMA conducts peer reviews of public expenditure and financial accountability systems in candidate countries. These reviews provide practical suggestions to help candidate countries establish adequate systems for public expenditure management and control as well as for public internal audit. The reviews conduct independent assessments of existing systems, report the findings, and make focused, practical recommendations for change. This type of peer review is requested by the candidate country's ministry of finance, often at the suggestion of the European Commission and SIGMA. The review team is composed of expert practitioners from EU member states. The process is similar to that used for peer reviews of supreme audit institutions and includes two one-week information gather- Annex 1: Scope and Application of the Main Instruments 99 ing missions and preparation of a detailed report submitted to the ministry of finance for consideration. The report is also reviewed within SIGMA and benchmarked against successful SIGMA reviews for other candidate countries. The final stage is the presentation of the report's key findings and recommendations to the relevant authorities in the country concerned and to the European Commission. The intended results are endorsement of the peer review's recommen- dations by the ministry of finance and the Commission and, after appro- priate consultations, agreement on next steps for the ministry--such as pur- suing changes to relevant laws and regulations, issuing a paper on the strategic development of relevant agencies and organizational structures, and identifying further technical assistance needs. UNDP CONTACT GUIDELINES The United Nations Development Programme (UNDP) developed the Country Assessment in Accountability and Transparency (CONTACT) to provide the international development community with a comprehensive tool for assessing national systems for financial management, accountabili- ty, and financial integrity. Guidelines setting out the approach and method- ology were issued in 2001 (UNDP 2001). CONTACT is a comprehensive set of generic guidelines that help: · Governments conduct self-assessments of the strengths, weaknesses, needs, and priorities of their financial accountability and integrity sys- tems. · Consultants hired by bilateral and multilateral development agencies to assess these systems. Consultant work is intended to support govern- ments (at their request) in their self-assessments and to review the feasi- bility of cooperation with governments (for example, through a brief mission to identify strengths and weaknesses of public expenditure man- agement prior to a full-scale programming mission by the UNDP, World Bank, or another agency). CONTACT provides stakeholders with a tool to support quality con- trol, measure performance, and offer recommendations at desired stages of the accountability cycle. It is a dynamic, continuously evolving instrument that requires regular updates based on best practices and pilot tests. Pilot 100 Assessing and Reforming Public Financial Management tests of CONTACT have been carried out in several countries, and train- ing in its use conducted in almost 30 countries in Asia and Africa. The CONTACT methodology includes guidelines and checklists cov- ering 12 areas: accounting infrastructure, information management, expen- diture planning and budgeting, internal control and audit, financial report- ing, external audit, revenue administration, debt management, project and foreign aid management, procurement and asset management, corruption prevention and control, and public cash management. CONTACT guidelines have much broader coverage than the other instruments reviewed in this report--particularly their treatment of the management of government records, revenue administration, debt man- agement, project and foreign aid management, and corruption issues. And unlike in World Bank assessment instruments, asset management is linked with procurement rather than treated as an expenditure management issue. NOTES 1. See BP (Bank Procedure) 2.1.1, annex A (January 1995), as amended, at http://wbln0018.worldbank.org/Institutional/Manuals/OpManual.nsf/ whatnewvirt/F773411614919ECF8525672C007D080A?OpenDocument. 2. Ibid. 3. This section draws heavily on World Bank and IMF (2002). 4. For the status of countries covered by the HIPC initiative, see http://www.worldbank.org/hipc/progress-to-date/status_table_Sep03.pdf. Of the 28 countries that have received debt relief, 20 countries have reached the "decision point" (reached after a country has established a three-year track record of appropriate policies and reforms), and eight have reached the "completion point" (reached after a further period of strong policy performance, where the Bank, IMF, and other donors commit to provide sufficient relief to reduce a country's debt to a sustainable level). Annex 2 Measuring Performance in Public Financial Management--Guidance from the Development Assistance Committee KEY ISSUES Effective financial management of public resources is essential to achieve the objectives of development programmes. It also promotes accountabili- ty within developing countries and provides donors with assurance on the use of their funds. Good financial management systems in partner countries are required for all forms of aid, but are particularly important for budget support, where donor funds are not allocated to finance specific expendi- tures. Diagnostic reviews in public financial management are a growing source of information to both governments and donors. They reflect the state of public financial management systems, the risks and constraints that these pose to the implementation of development programmes and the use of donor resources (Box A2.1).1 Diagnostic reviews and performance meas- Note: This annex contains the text of chapter 3 from Harmonizing Donor Practices for Effective Aid Delivery, a volume published in 2003 by the OECD's Development Assistance Committee. 101 102 Assessing and Reforming Public Financial Management ures assist governments in creating strategies that improve management of public finances. Diagnostic reviews provide information on the strengths and weak- nesses of the public financial management systems of partner countries. They support country efforts to improve the performance of their public financial management systems, and so contribute to improved development outcomes. They also provide information to allow donors to consider the risks that their funds may not be used for agreed or appropriate purposes. Thus, partner country governments and donors have a shared interest in high quality diagnostic reviews. In recent years, the proportion of development funds made available through government financial systems has increased significantly. Efforts by donors to improve their understanding of the functioning of these sys- tems have brought about the risk of an uncoordinated development of dif- ferent diagnostic reviews. For developing countries, this means that a dis- proportionate amount of scarce administrative resources may be spent ful- filling multiple donor requirements. For development agencies, it may lead to unnecessary duplication of work. Thus there is considerable value, for donors and partner countries alike, in harmonising different diagnostic reviews and placing them in the context of a coherent, sequenced pro- gramme of work to strengthen public financial management. This, in turn, should be an integral part of participating donors' strategies for country assistance and the country's own poverty reduction strategy. Box A2.1 What is a diagnostic review? A diagnostic review examines a partner country's public financial manage- ment system and practices. Diagnostic reviews are generally not audits and do not track individual items of expenditure. Nor do they provide a pass or fail assessment of a country's public financial management system in terms of its adequacy for managing external funds. Rather, they provide donors and governments with information on: · The strengths and weaknesses of public financial management systems. · The risks to which funds channelled through governments' systems may be exposed. · The government programmes aimed at improving these systems. Annex 2: Measuring Performance in Public Financial Management 103 Diagnostic work records the state of public financial management at a point in time. Partner countries and donors have a shared interest in being able to monitor progress over time in improving public financial manage- ment systems. A performance measurement framework--and an associated set of reform measures--assist them in this by building on the information set out in diagnostic reviews. Much of the formal diagnostic work in the area of public financial man- agement is still evolving. A number of diagnostic tools are used. The main instruments with a brief description of their purpose are listed in Box A2.2. In addition to these formal reviews, important diagnostic work is often car- ried out through specific technical assistance projects or as part of the preparation and implementation of donor financed activities. PURPOSE The purpose of this paper is to put forward a set of good practices on how donors and governments can collaborate to make diagnostic reviews and measurement of performance in public financial management more effec- tive. In pursuing this goal it seeks to achieve four specific objectives: · Maximise developmental benefits--Diagnostic reviews are a valuable input to government efforts to improve public financial management systems. Their quality and impact can be improved by encouraging stronger country ownership and in-country consultation, and by integrating the reviews into both government and donor decision-making cycles, pro- grammes of country assistance and poverty reduction strategies. · Make information more readily available--Sharing the information and knowledge they provide with both donors and partner governments enhances the impact of diagnostic reviews. · Rationalise diagnostic reviews--Diagnostic reviews should provide full coverage of public financial management issues without unnecessary duplication. Donors should reduce the burden on developing countries through fewer and better co-ordinated diagnostic reviews. · Provide a framework for performance measurement in public financial man- agement-- Partner countries and donors have a shared interest in being able to monitor progress over time in improving public financial man- agement systems, as reflected in diagnostic reviews. This paper provides 104 Assessing and Reforming Public Financial Management Box A2.2 Current diagnostic tools Country Financial Accountability Assessment (World Bank) CFAAs are a diagnostic tool designed to enhance knowledge of public financial management and accountability arrangements in client countries. Public Expenditure Review (World Bank) PERs analyse the recipient country's fiscal position, its expenditure poli- cies--in particular the extent to which they are pro-poor--and its public expenditure management systems. Country Procurement Assessment Review (World Bank) CPARs examine public procurement institutions and practices in borrower countries. HIPC Expenditure Tracking Assessment (World Bank & IMF) These assess the ability of the public financial management systems in high- ly indebted poor countries (HIPCs) to track poverty-reducing expendi- tures, using fifteen public financial management benchmarks. Fiscal Transparency Review (IMF) This is a module of the Reports on Observance of Standards and Codes (ROSC) which uses the Code of Good Practices on Fiscal Transparency adopted by the IMF in 1998. Diagnostic Study of Accounting and Auditing (Asian Development Bank) These examine financial management and governance practices in the pub- lic and private sectors of borrower countries. Ex ante assessment of country financial management (European Commission) Traditionally, the EC has carried out audits of its "targeted" budgetary sup- port with a view to determining expenditures "eligible" or "ineligible". For future budget support, however, it is developing a new approach using ex ante PFM assessments based on a mix of diagnostic work completed by other donors/governments and a "compliance test" to provide an empirical evidence of performance of the PFM systems. Country Assessment in Accountability and Transparency (UNDP) CONTACT is a toolkit to assist governments and consultants in conduct- ing missions to assess public financial accountability systems. Annex 2: Measuring Performance in Public Financial Management 105 broad guidance on how donors might address this emerging issue in individual partner countries. The practices set out in this paper are consistent with those described in the Good Practice Paper entitled Country Analytic Work and Preparation of Projects and Programmes. GUIDING PRINCIPLES Diagnostic work The following principles guide the elaboration of good practices in diag- nostic work in this paper: · Partner country governments should be fully involved in and have ownership of diagnostic reviews--There is scope for partner governments to be increas- ingly involved in the conduct of diagnostic reviews and this paper sug- gests a number of good practices to that end. This should improve the quality of these reviews and provide greater incentives for donors and partner governments to strengthen public financial management systems. · Harmonised diagnostic reviews alleviate the burden on partner countries-- Donors should rationalise the scope, timing and conduct of diagnostic reviews so as to avoid unnecessary duplication of work, and should rely as far as possible on other donors' diagnostic reviews to satisfy their requirements. · Harmonisation does not mean standardisation--In many countries receiving development assistance, public financial management systems may fall well below international good practices. This does not mean that finan- cial assistance should not be provided to these countries: each donor must decide what level of risk it accepts in providing funds through the partner country's budget, in relation to developmental benefits. Thus, different donors may make different decisions on the basis of the same diagnostic review. · Diagnostic reviews should be responsive to country development context-- Improvement in public financial management systems is part of the overall development agenda, driven by the partner country and support- ed by donors. Undertaking of diagnostic work will be influenced by and 106 Assessing and Reforming Public Financial Management feed into the overall country support strategies. Diagnostic studies should be regarded as part of a process, not as a single product, and sequenced over time. · Diagnostic reviews should be conducted according to open and transparent process- es--There should be full consultation with all stakeholders before com- pletion. There should be "no surprises" when the final report is published. · Understanding the institutional and governance context--Reviews of public financial management systems are not simply "technical" exercises. They require a full understanding of the underlying governance arrangements in a country, the informal rules and incentive structures, and the factors that can undermine efficient and effective delivery of public services through waste, mismanagement and corruption. Performance measurement The following principles guide the elaboration of good practice in per- formance measurement: · The measurement framework needs to encompass the critical aspects of public finan- cial management--covering budget formulation, execution, reporting, and review--and be fully integrated with diagnostic reviews and processes. · The measurement framework should encompass internationally agreed codes and standards--which represent the target for public financial management systems--where appropriate. This leads to two specific requirements: ­ Codes and standards need to be developed to fill existing gaps in cov- erage. ­ Developing countries should fully participate in the development and formulation of codes and standards. · Performance measures should be comprehensive with respect to the critical areas to be covered without being excessively numerous. They should be readily understandable, cost effective, and capable of being calibrated to cover different stages of development and monitored on a regular basis. They should be acceptable to both partner countries and donor agencies and, where appropriate, consistent with current and proposed relevant inter- national codes and standards. · While there can be no single overall measure of public financial management per- formance, it is necessary to avoid having too many indicators. In addition to indi- Annex 2: Measuring Performance in Public Financial Management 107 cators of the overall effectiveness of the system, it can be useful to have more detailed indicators of specific aspects or areas of public financial man- agement, e.g., public procurement, internal control and external audit. GOOD PRACTICES IN DIAGNOSTIC WORK Planning the review As far as possible donors should seek to satisfy their requirements by rely- ing on the diagnostic reviews that are already available, or that will be made available within a reasonable period of time. Where individual donors' requirements are not satisfied by available diagnostic reviews, they should seek to limit unnecessary duplication of work by collaborating with other donors and stakeholders and sharing the results of their reviews. All donors should promptly share information on plans for carrying out diagnostic work in a particular country. The reasons for conducting the review and its relationship with the overall country development strategy and the strategies of participating donors should be clear to, and accepted by, all participants at the beginning of the review. Collaboration between partner country and donors is as important as collaboration between donors. Collaboration may take a number of forms: · Joint undertaking of a diagnostic review--With work being undertaken by staff or consultants of any number of development partners, including the partner country. · One or some partners undertaking the study--This may include the partner country undertaking a self-assessment, with appropriate review by donors, or one or more donors carrying out work on behalf of a wider group, with others providing funding or specific technical resources or indicating their intention to use the results of the study. Conducting the review The nature and extent of collaboration should be agreed and documented at the start of the planning process. This should include: · The role of the partner country in the work. 108 Assessing and Reforming Public Financial Management · Whether there is to be a joint or separate reports for different donors. Whether there is to be a joint report, and the preparation of a joint ini- tiating memorandum. · Which donors may be lead-participants, and which may take a more sec- ondary role. Where there are many donors interested in a diagnostic study it may be necessary to limit the direct participation of some so as to keep the process cost-effective. This would be achieved through mutual agreement. · The staffing contribution and division of labour and costs between the participants. · Processes for communicating conclusions with other stakeholders. Obtaining objective information on the quality of the public financial man- agement system requires consultation with all stakeholders in the partner coun- try. Apart from central ministries such as Ministry of Finance, these include spending ministries, parliaments, the business community and civil society. Progress and emerging conclusions should be shared with all partici- pants during the course of the review through regular meetings. The part- ner country should be encouraged to share draft conclusions with other country stakeholders before finalisation. Quality assurance All participants have a shared interest in achieving a quality diagnostic review. Where joint reviews are carried out, quality assurance arrangements should be agreed at the planning stage and recorded in a jointly owned paper. These arrangements will normally provide for a regular review of work done by each participating organisation, and for clearance by others before the report is issued. Sharing the review report All review reports will be shared to the fullest extent, observing individual participating donors' disclosure policy and the prior agreement reached with the partner country.2 Follow-up activity Donors will work to develop an understanding with the partner country that the review is intended to bring about improvements in the public Annex 2: Measuring Performance in Public Financial Management 109 financial management system. This should lead to the elaboration of an agreed action plan and follow-up activity. Collaboration on follow-up is important and should also involve identi- fication of capacity development needs and provision of technical assistance where necessary. The partner country should co-ordinate this process and avoid donor competition. Updating of the review Consistent with the review's role as a knowledge tool, the information needs to be kept up to date. While a full diagnostic review may be under- taken only every three to four years, updating of the overall assessment should be undertaken every year. This includes using appropriate public financial management performance indicators. Desirably, this should take place through a joint mechanism agreed between the partner country and donors and built into the country's budget cycle. Box A2.3 Improving public oversight of public expenditures Bangladesh--During 1999-2000, a Country Financial Accountability Assessment was conducted by the World Bank and UNDP with participa- tion from the Ministry of Finance, line ministries, parliamentary oversight bodies and private firms of accountants. The assessment focused on the quality of financial accountability and transparency in Bangladesh, and made recommendations for improvements. The financial management standards and practices of agencies using public funds were compared to international good practices as well as those of external oversight agen- cies--nine Audit Directorates of the Comptroller and Auditor General's Office, parliamentary committees concerned with public expenditure, donor agencies and the media. It included an assessment of the steps that would be needed to support a shift by donors from projects to budget sup- port and sector programmes. Among the responses of the government is the creation of an independent Public Expenditure Review Commission, headed by a retired Comptroller and Auditor General. The Commission will review all aspects of public expenditure on an annual basis and report to the government and to Parliament. 110 Assessing and Reforming Public Financial Management Box A2.4 Partners support government-led diagnostic process in Tanzania The Government of Tanzania owns and leads the diagnostic work program in public financial management--and that is what accounts for its success. The programme, of which the centrepiece is the annual Public Expenditure Review (PER)--is closely integrated into the government's budget cycle and involves a broad cross-section of stakeholders, and close partnership with donors. The PER process has evolved from its traditional role of external evaluation of budget management to support the government's programme to improve budget management within a medium term expen- diture framework. It supports better donor co-ordination by ensuring that aid is consistent with budgetary objectives and priorities and increasingly integrated in the budget. More recently, the PER has been supplemented by the Country Financial Accountability Assessment (jointly undertaken by the World Bank and DFID, published as a government document, and shared with all donors) and the Country Procurement Assessment Review, which look at critical aspects of budget execution. The programme has the following benefits: (a) donors share their sector specific or thematic experi- ence, to the benefit of all who participate, (b) more effective use of resources in undertaking technical studies on budget issues, (c) a higher profile of budgetary issues in the work of all parties, and (d) a platform to support the shift from project to budget support by several donors. GOOD PRACTICES IN PERFORMANCE MEASUREMENT Given the emerging nature of work in this area, good practices have yet to fully emerge. However, based on work to date, the following appear important: · Partner country governments and donors should collaborate in the set- ting and monitoring of country-specific performance indicators in pub- lic financial management. · Donor collaboration should take place within a government-led strategy for improving the overall management of public finances that sees per- formance measurement fully integrated with diagnostic work and capaci- ty building efforts. Common performance indicators can avoid govern- ments being presented with an excessive number of--potentially conflict- ing--targets, and can serve to co-ordinate donor capacity building efforts. Annex 2: Measuring Performance in Public Financial Management 111 · Given the absence of a comprehensive, internationally accepted perfor- mance measurement framework in public financial management, donors should support work on the development of such a framework. They should also ensure that developing country conditions and voice are reflect- ed. Where available, donors should be guided by existing international standards, codes and approaches in formulating country-specific perform- ance targets, as appropriate to country conditions (see Box A2.5 below). · Indicators of public financial management performance assist donors in assessing risks to their programmes. While donors should collaborate in the formulation of performance indicators in individual countries, donors decide individually on the nature and level of resource transfers in a given set of circumstances. They will take into account previously agreed measures. Box A2.5 Elements of a performance measurement framework Among the elements of an emerging international framework for performance measurement in public financial management are: International standards, such as the International Public Sector Accounting Standards (IPSAS) of the International Federation of Accountants and the auditing standards of the International Organisation of Supreme Audit Institutions (INTOSAI). Work is also commencing on IPSAS for develop- ment assistance and budget reporting. Codes of good practice, such as the IMF's Code of Good Practices on Fiscal Transparency and budget classification procedures promulgated by the IMF (Government Finance Statistics) and United Nations (COFOG). Current benchmarks, such as the set of 15 performance benchmarks in pub- lic financial management used by the IMF and World Bank in tracking poverty reducing expenditures in Heavily Indebted Poor Countries (HIPCs). An updated set of these benchmarks will be available in 2003. In addition, UK DFID has developed a set of public financial management performance benchmarks in the context of managing fiduciary risk when providing direct budget support. 112 Assessing and Reforming Public Financial Management Table A2.1: Indicators of good practice in measuring performance in public financial management Good Practices Indicators Sources of Information Diagnostic coverage Reduction in number of diagnostic Country Analytic Work without duplication reviews that address the same Website area Reduction in number of donor missions Higher impact of Proportion of reviews carried Information on country diagnostic work out in the context programmes and Country of country strategy Analytic Work Website Proportion of reviews carried out through agreed collaborative mechanisms Enhanced partner Government/donor Announced government/ country capacity in community consensus donors programmes, and public financial on public financial management progress reports thereon management performance benchmarks Agreed approach to building public financial management capacity NOTES 1. Public financial management includes all phases of the budget cycle, including the preparation of the budget, internal control and audit, pro- curement, monitoring and reporting arrangements, and external audit. The broad objectives of public financial management are to achieve overall fis- cal discipline, allocation of resources to priority needs, and efficient and effective allocation of public services. 2. See the Country Analytic Work Website at www.countryanalytic- work.net, also referred to in the Good Practice Paper Country Analytic Work and Preparation of Projects and Programmes. Annex 3 Technical Map of the Assessment Instruments Table A3.1 shows the coverage of public expenditure management issues for the six main assessment instruments--PERs, CFAAs, CPARs, Fiscal ROSCs, HIPC AAPs, and EC audits--summarized in the main text of the study and described in more detail in annex 1. Each instrument's coverage is mapped in five broad areas: the legal and organizational framework, expen- diture programming and budget preparation, budget execution, accounting, reporting and external audit, and administrative and financial management capacity. The 15 main components of these five areas are shown in bold text, and each component is divided into subcomponents--94 in all. The findings in table A3.1 are based on the guidelines for the instru- ments, reviews of a sample of assessment reports, and discussions with experts on each instrument.1 (For PERs, which do not have formal guide- lines, the findings are based on the coverage of actual reports.) Three lev- els of coverage are shown for the 15 main components: complete or sub- stantial, partial or moderate, and little or none. For the 94 subcomponents a decision was made about whether each instrument covers or does not cover each subcomponent. A double check-mark () indicates that the subcomponent is covered by the instrument's guidelines and regularly fea- tured in assessment reports. A single check mark () means that the sub- component is not covered by the guidelines but is often featured in reports. No judgments were made about the depth of the coverage or the technical quality of the instruments' analysis. 113 114 Assessing and Reforming Public Financial Management As explained in the main text, transparency, accountability, participation, and rule of law are critical cross-cutting issues that apply to all areas of pub- lic expenditure management. So, while they are not shown in table A3.1, they are implicit in every component. For example, transparency is a key criterion for assessing budget coverage and external audit. Similarly, some instruments, such as CFAAs, have a strong focus on accountability--while others, such as Fiscal ROSCs, focus on fiscal transparency. Although cor- ruption is not explicitly covered in the table, financial integrity issues are fundamental to the assessment of most components of public expenditure management. NOTE 1. These sample reports included the following countries: CFAA-- Bangladesh, Benin, Bosnia-Herzegovina (BiH), Brazil, Mozambique, Philippines, Turkey, Uganda, Ukraine, Vietnam and Zambia; CPAR--BiH, Philippines, Russia, Turkey, Uganda, and Zambia; PER--BiH, Croatia, Ethiopia, Philippines, Uganda, the Philippines, Tanzania (and sector PERs on Agriculture, Health and Education), Turkey, Vietnam and Zambia; Fis- cal ROSC--Bulgaria, Mali, Pakistan, and Turkey; EC Audits--Cameroon, Chad, Cote d'Ivoire, Ghana, Mozambique and Zambia. s EC AUDIT Plans Fiscal a Bank) of HIPC AAPs Action Bank) orld (W and orld Codes b (W and eportsr Fiscal ROSCs ds eportsr Bank) Reports Assessments Assessments ARs CP orld Standar assessment (W of rackingT assessment in Assessment e audits in ed AAs ed CF Reviews e Accountability featur ement Observance featur (IMF) amework fr ocur the Expenditur often Commission PERs Financial Pr on ency is Expenditur Bank­IMF) Public egularlyr but orld opean Public Country Country Reports ransparT HIPC (W Eur and = = = = = = ganizational branches guidelines s or nment guidelines 3 s judicial and gover ROSCs and of AAPs AAs ARs audits instrument' instruments Legal Instruments PERs CF CP Fiscal HIP EC instrument' the I. levels the by legislative, ent by ed fer policy dif assessment ed cover the management fiscal executive, of cover not e substantial is is and the between or of moderate elationsr none mapping or or expenditur ements budget olesr fiscal for for Complete Partial Little equirr esponsibilitiesr echnicalT coverage = = = Subcomponent Subcomponent of of = = amework A3.1: fr framework management governmental bleaT Level Key Component/subcomponent Legal Constitutional Legal Fiscal Inter Allocation 115 s EC AUDIT HIPC AAPs Fiscal ROSCs ARs CP ation AAs epar CF pr PERs budget and bodies) amming statistics nments finance, of ogr gover on) pr finance enterprises e entities sector so with public and, ministry, autonomous nment private nments subnational and holdings of agencies gover of consistency minister agencies, Expenditur gover and II. with nongovernment and nment rights equity parliamentary prime and egulationr of gover management owing targets nment ectorates, ministries statistics (dir subnational formal of consistency borr to fiscal gover esidential, (powers e service between of and and government of e (pr finance civil nment structur for independence (continued) transfers structur nment organizations age e authority between nment gover distinction eportingr openness bank esponsibilitiesr of of and gover arrangements ministries) tier classification A3.1: of cover gover of line bleaT Component/subcomponent Expenditur Revenue Relations Scope with Central Clarity Clarity Clarity Government ypeT Cabinet and Organizational financial Arrangements Second Budget Budget 116 es expenditur ograms pr spending e evenuesr amming outlook budgeted impact ogr etionary ograms expenditur omfr pr e fiscal enterprises pr and earmarked es discr of poverty ears actual arr and public and and of expenditur funds omfr and expenditur investment fectiveness framework es of ef payment activities liabilities and analysis analysis analysis e mandatory incidence of to public e deviations ojections and of pr funding of of amework fees sustainability fr oeconomic expenditur Extrabudgetary Quasi-fiscal Contingent axT User Donor ransfersT ficiency Expenditur Fiscal Composition Analysis Review Intersectoral Intrasectoral Ef Expenditur Analysis Assessment Fiscal Macr Revenue 117 s EC AUDIT HIPC AAPs Fiscal ROSCs ARs CP AAs CF PERs spending) ceilings ces e assumptions on) targets esourr so and of ministerial deficit of and expenditur models and e and allocation ogram, eparation (flexibility pr budgets pr evaluation and ent and legislatur analysis spending, oeconomic priorities ograms budget ograms pr budget the liabilities (line-item, pr estimates draft ecurrr macr in by policy priorities allocations e eviewr of evenue,r sensitivity of of oposed the and pr objectives ojections budget ongoing ogram budget pr and contingent egate for pr scrutiny ation setting setting in of expenditur classification eparing capital (continued) and ogram participation draft aggr epar e pr of pr omfr detail costs of funding of pr d of for A3.1: scenarios risks of estimates society oval bleaT ocess Component/subcomponent Donor Fiscal Fiscal Independent Setting Budget Cabinet-level Ministry-level Level Expenditur Forwar Cost Medium-term Clarity Pr Integration Civil Feedback Appr 118 execution on) so Budget and III. monitoring e nment ears es assets carryovers, arr gover allocations management ocedur expenditur rules, and cash pr physical within es cash and of and services and and ement payables information systems (vir and ocedur of function functions pr ocesses easurytr bank pr arrangements payments goods assets management management, on easurytr ecasting management eviewr for managers of and egulationsr ement for es ol and and ement information cash, and ocur es the fiscal physical and eceiptr management pr and of of ocur accounting ocedur budget contr laws of pr of egulationsr of and ement ocedur complaint pr systems and for of pr ocur monitoring and and planning and pr ement ol oll easuryrT Laws Organization Cash Banking Rules Flexibility Commitment erificationV ocur Monitoring Payr Reconciliation Public Pr Organization Openness enderingT Documentation Contr Management 119 s c EC c AUDIT HIPC AAPs Fiscal ROSCs ARs CP AAs audit CF PERs external nda, eportingr, es es debt ocedur systems assets pr ocedur pr Accounting. nment and IV eportsr eportsr and gover financial aid information egulationsr of of of management financial ds ds financial and nal organization, organization, ecorr nal esponsibilitiesr laws eportingr eportingr eportingr standar exter exter audit and, and management of systems and and and and of ol, ol, ol, and egulations,r ol ol egulations,r quality ocesses contr contr contr (continued) eportingr, financial management management policies pr coverage and contr contr audit eportingr aid aid management A3.1: nment and nal nal nal ds and and bleaT Component/subcomponent Internal Inter Inter Accounting Accounting Accounting Gover Inter Scope imelinessT Recor Debt Debt Management, Management, Management, 120 . capacity ement debt. manag nal e exter capacity expenditur cover not public do owing. they overall borr of but management local ears on arr indicative nal e ar financial estraintsr inter that on to and elater audits indicators ative they special as of sentinal management management Administr.V number and, and assignments a identifying institution eportsr oach, evenuer audit institution performed nal audit audit eportsr ncentivesi, incentives,, appr and has audit exter eme of audit eme ecommendationsr systems for supr of capacity, capacity, a transfers of supr quality egularity ds irr audit cover Commission of and for eviewr follow on quality quality audit opean standar AAPs ROSCs framework Eur External Legal Independence Jurisdiction Audit imelinessT Sanctions Legislative Follow-up sonnelreP HIPC Fiscal The Personnel a. b. c. 121 References Allen, Richard, and Daniel Tommasi, eds. 2001. Managing Public Expenditure: A Ref- erence Book for Transition Countries. Paris: Organisation for Economic Co-opera- tion and Development. Asian Development Bank. 1995. Governance and Development Policy Paper. Manila. Campos, J. Edgar, and Sanjay Pradhan. 1996. The Impact of Budgetary Institutions on Expenditure Outcomes: Binding Governments to Fiscal Performance. Washington, D.C.: World Bank. Collier, Paul, and David Dollar. 1999. "Aid, Risk and the Special Concern of Small States." World Bank, Poverty Reduction and Economic Management Network, Washington, D.C. DAC (Development Assistance Committee). 2003. "Measuring Performance in Public Financial Management." In Harmonizing Donor Practices for Effective Aid Delivery. DAC Guidelines and Reference Series. Paris: Organisation for Eco- nomic Co-operation and Development. Devarajan, Shanta, and Vinaya Swaroop. 2000. "The Implications of Foreign Aid Fungibility for Development Assistance." In Christopher Gilbert and David Vines, eds., The World Bank: Structure and Policies. Cambridge: Cambridge Uni- versity Press. De Vries, Barend A. 1967. "External Aid for Plans or Projects: An Alternative Approach." Economic Journal 77 (307). DFID (Department for International Development). 2001a. "Country Financial Accountability Assessments: Interim Guidance for Staff." London. 123 124 Assessing and Reforming Public Financial Management ------. 2001b. Understanding and Reforming Public Expenditure Management: Guide- lines for DFID. Version I. London. ------. 2002. Managing Fiduciary Risk When Providing Direct Budget Support. Lon- don. EC (European Commission). 2001a. Counterpart Funds from Structural Adjustment Support Earmarked for Budget Aid (7th and 8th EDFs). Special Report 5/2001. Brussels. ------. 2001b. Report on the Management, Monitoring and Control Arrangements Gov- erning the Use of Counterpart Funds and Budgetary Support in Development Aid. Internal Audit Service, Brussels. ------. 2002. "Guide méthodologique pour la programmation et la mise en ouvre des appuis budgétaires dans les pays tiers." Brussels. Gold, Joseph. 1981. Conditionality. Washington, D.C: International Monetary Fund. IMF (International Monetary Fund). 2001. "Code of Good Practices on Fiscal Transparency." Washington, D.C. [http://www.imf.org/external/np/fad/trans/ code.htm#code]. Merriam-Webster. 1997. Merriam-Webster's Collegiate Dictionary, 10th ed. Spring- field, Mass. PEFA (Public Expenditure and Financial Accountability). 2002. "Integrating PER, CFAA and CPAR--Recent Country Experience." Discussion paper. PEFA Sec- retariat, Washington, D.C. ------. 2003. "The European Commission's Compliance Test: An Overview of the Current Approach." PEFA Secretariat, Washington, D.C. Reite, Torun. 2002. Lessons Learned from Donor Participation in Country Financial Accountability Assessments (CFAAs)--Synthesis Report. Washington, D.C.: World Bank. Schiavo-Campo, Salvatore. 1999. "Performance in the Public Sector." Asian Jour- nal of Political Science (December). Schiavo-Campo, Salvatore, and Hans W. Singer. 1970. Perspectives of Economic Development. Boston: Houghton Mifflin. Schiavo-Campo, Salvatore, and Daniel Tommasi. 1999. Managing Government Expenditure. Manila: Asian Development Bank. SIDA (Swedish International Development Agency). 2002. Financial Management Issues for Programme Support Methodologies--Diagnostic Practices. Stockholm. UNDP (United Nations Development Programme). 2001. Country Assessment on Accountability and Transparency. New York. University of Birmingham School of Public Policy. 2002. Needs Assessment Report: Survey of Partners' Priorities and Perspectives on Harmonizing Donor Practices. Paper References 125 prepared for the OECD DAC Task Force on the Harmonization of Donor Practices. Walker, David M. 1980. The Oxford Companion to Law. Oxford: Clarendon Press. World Bank. 1987. "The Bank's First Loan: $250 Million to France." The Bank's World (May). Washington, D.C. ------. 1989a. From Crisis to Sustainable Growth--Sub-Saharan Africa: A Long Term Perspective Study. Washington, D.C. ------. 1989b. Madagascar: Public Expenditure, Adjustment and Growth. Report 7803-MAG. Washington, D.C. ------. 1992. Governance and Development. Washington, D.C. ------. 1994. Governance: The World Bank Experience. Washington, D.C. ------. 1997a. Helping Countries Combat Corruption: The Role of the World Bank. Washington, D.C. ------. 1997b. World Development Report 1997: The State in a Changing World. New York: Oxford University Press. ------. 1998a. Assessing Aid: What Works, What Doesn't, and Why. A World Bank Policy Research Report. New York: Oxford University Press. ------. 1998b. Public Expenditure Management Handbook. Washington, D.C. ------. 1999. "Public Expenditure Reviews: Progress and Potential." PREMnote 20. Poverty Reduction and Economic Management Network, Washington, D.C. ------. 2000a. Anticorruption in Transition: A Contribution to the Policy Debate. Wash- ington, D.C. [http://lnweb18.worldbank.org/eca/eca.nsf/General/D74DB 51B2D46615D8525695B00678C93?OpenDocument]. ------. 2000b. Helping Countries Combat Corruption: Progress at the World Bank since 1997. Washington, D.C. ------. 2000c. Reforming Public Institutions and Strengthening Governance: A World Bank Strategy. Public Sector Board, Washington, D.C. [http://www1.world- bank.org/publicsector/strategy.htm]. ------. 2001. "Interim Guidelines for Public Expenditure Work and Public Expen- diture Analysis (including PERs)." Poverty Reduction and Economic Manage- ment Network, Public Sector Group, Washington, D.C. [http://www1.world bank.org/publicsector/pe/guidelinespework.htm]. ------. 2002c. Country Procurement Assessment Report: Information for Borrowers. Procurement Policy and Services Group. Washington, D.C. [http://www.world bank.org/html/opr/procure/cpar.htm]. ------. 2002d. From Adjustment Lending to Development Support Lending: Key Issues in the Update of World Bank Policy. Operational Policy and Country Services. Washington, D.C. 126 Assessing and Reforming Public Financial Management ------. 2002e. Quality of Fiduciary Economic and Sector Work (January 2000­June 2001): A QAG Assessment. Quality Assurance Group. Washington, D.C. ------. 2003a. Reforming Public Institutions and Strengthening Governance: A World Bank Strategy Implementation Update. Public Sector Board, Washington, D.C. ------. 2003b. "Country Financial Accountability Assessment: Guidelines to Staff." World Bank Financial Management Sector Board, Washington, D.C. World Bank and IMF (International Monetary Fund). 2002. "Actions to Strength- en the Tracking of Poverty-Reducing Public Spending in Heavily Indebted Poor Countries (HIPCs)." Washington, D.C. [http://www1.worldbank.org/public- sector/pe/newhipc.pdf]. ------. 2003a. "Bank/Fund Collaboration on Public Expenditure Issues." World Bank Poverty Reduction and Economic Management Network and IMF Fiscal Affairs Department, Washington, D.C. ------. 2003b. "Update on Implementation of Action Plans to Strengthen Capac- ity of HIPCs to Track Poverty-Reducing Public Spending." World Bank Pover- ty Reduction and Economic Management Network and the IMF Fiscal Affairs Department, Washington, D.C. About the Authors Richard Allen is a senior staff member of the World Bank and Head of the Public Expenditure and Financial Accountability (PEFA) Program. He was previously a senior official at HM Treasury (the British Ministry of Finance), the OECD and the Asian Development Bank, a UK government diplomat in Washington, D.C., and a board member of the European Investment Bank. He holds degrees in economics from the Universities of Edinburgh and York. Salvatore Schiavo-Campo, a former senior staff member at the World Bank, Asian Development Bank, IMF, and USAID, and previously Profes- sor and Chairman of Economics at the University of Massachusetts at Boston, is currently an international consultant, including as Senior Advi- sor for the PEFA Program. With a Ph.D. in economics and a Master's degree in international affairs from Columbia University, and an LL.D. from the Universita' di Palermo, he is the author of a dozen books and many articles in public sector management and economic development. Thomas Columkill Garrity is a consultant to the World Bank and the PEFA program on governance-related issues. He holds a Master's degree in social anthropology from the University of St. Andrews, Scotland; a Mas- ter of Business Administration degree from Hult International Business School; and a Master of Arts in Law and Diplomacy degree from the Fletcher School of Law and Diplomacy, Tufts University. 127 Index Use of italic b, f, n, and t indicates EC audits and, 90­94 box, figure, note, and table. fiduciary assessment required for, 19 accountability, 12­13, 33­34, 37 PER not formally done for, 35 See also Country Financial shift to policy basis for, 4 Accountability Assessments administrative and financial man- (CFAAs); governance principles agement capacity accounting, reporting, and external as components of public expen- audit diture management, 27, 31b2, as CFAA focus, 58, 83 32f1 as components of public expen- coverage by assessment instru- diture management, 27, 30b2, ments, 121tA3.1 32f1 limited coverage of, 35 coverage by assessment instru- African Development Bank, 82 ments, 120tA3.1 Albania, 96 as focus of most assessment Anti-Bribery Convention (1999), instruments, 35 5 international standards for, Asian Development Bank, 5, 82 111bA2.5 Diagnostic Study of Accounting adjustment lending, xi and Auditing, 104bA2.2 assessments required for, 36, Financial Management and 47n5 Governance Issues in Select- 129 130 Assessing and Reforming Public Financial Management ed Development Member public expenditure management, Countries, 26n2 defined, 2 assessing of assessors, xvi­xvii, purpose of assessments, 15, 27 66­67 purpose of study, 1, 2­3 SIGMA peer reviews, 97­99 quality control of. See quality assessment instruments, 15­26, control of assessments 104bA2.2 asset management, lack of coverage See also specific instrument by of, 46 name audits. See accounting, reporting, checklists. See questionnaires and external audit; EC and checklists, use of audits diagnostic reviews as, 101­3 definition of diagnostic review, Bangladesh, 50, 109bA2.3 102bA2.1 Bolivia, 64, 82 differences among, 40­41, Bosnia, 84 51­52 budget execution gaps in, 46­47 as CFAA focus, 58 internal evaluations of, 38 as component of public expendi- objectives and role of, 9­10, ture management, 27, 30b2, 16b1 32f1 primary vs. secondary instru- corruption and, 33 ments, 40 coverage by assessment instru- process followed by, 37­38 ments, 119tA3.1 questionnaires. See question- as focus of most assessment naires and checklists, use of instruments, 35 scope and application of, xiii­xiv, as PER focus, 35 34­35, 44, 77­100, 113­21 budget preparation similarities among. See duplica- as CFAA focus, 18, 24, 34 tion and similarities of instru- as component of public expendi- ments ture management, 27, 28b2, study coverage of, 34­35 32f1 assessments of public expenditure corruption and, 33 management, 1­7, 101­12 coverage by assessment instru- background, xi, 3­5 ments, 116tA3.1 costs of. See costs of assessments as EC audit focus, 22 guidance from Development as PER focus, 17, 24, 34, 58 Assistance Committee. See Build-Operate-Transfer (BOT) OECD mechanisms, 33 new framework for, 69f2, 73n6 Bulgaria, 96 Index 131 Burkina Faso, 38, 39b3 coordination within World Bank, participatory CFAA, 51 57­63 corruption, effects of, xi, 5, 33 Cambodia as example of coordinat- CFAAs looking at, 42, 83 ed assessments, 65 Fiduciary Review (World Bank) Campos-Pradhan approach, 16, 78 developed to address, 73n1 Central Europe. See EC and recommendations concerning, OECD Support for Improve- 64 ment in Governance and costs of assessments, 50, 53, 57, Management in Central and 103 Eastern Europe (SIGMA) Country Analytic Website, 53, CFAAs. See Country Financial 112n2 Accountability Assessments Country Assistance Strategies and checklists. See questionnaires and Poverty Reduction Strategy checklists, use of Papers, 54 collaboration and cooperation. See Country Financial Accountability integration and coordination Assessments (CFAAs), 1, 15, components and subcomponents of 104bA2.2 public expenditure manage- accounting issues, coverage of, ment, 27, 28b2 83 map of major assessment tools, asset management inadequately 113­21 covered by, 46 summary map of assessment for Burkina Faso, 39b3 tools, 32f1 combining with PERs, 61, 82 conditionality, 13 coordination with CPARs, 82 consent of government to assess- coordination within World ment, 38 Bank, need to improve, CONTACT. See UNDP CON- 57­63 TACT Guidelines description of, 18­19, 80­83 coordination between agencies, dissemination of, 82 53­57 Financial Management Net- assessment instruments encour- work's role, 80­81 aging, 37 government participation in CFAAs and, 82 process, 36 lack of coordination as problem, institutional considerations and, 38 42, 43 recipient government com- overlap with other instruments, pelling, 52 xiii, 41, 44, 45, 51, 58 coordination within agencies, 52 as primary instruments, 40 132 Assessing and Reforming Public Financial Management questionnaires, use of, 24, 25t1 as successful collaboration relationship to EC audits, 55­56 between agencies, 53 revised guidelines for, 38, 40, Czech Republic, 96 42, 83 risk definition used for, 10 debt management, 46 scope and range of, 34, 45, 81, See also World Bank-IMF Public 83, 115tA3.1 Expenditure Tracking Assess- significance of, 36 ments and Action Plans for skills of teams conducting, Heavily Indebted Poor 66­67 Countries (HIPC AAPs) standardization of, 40­41 DFID assessments of fiduciary risk, Strategic Partnership with Africa 15 (SPA) study of, 50­51, 82 benchmarks used by, 95, as successful collaboration 111bA2.5 between agencies, 53 CFAAs, collaboration with, 82, taxes inadequately covered by, 95 46 description of, 23, 94­95 Country Procurement Assessment government participation in Reports (CPARs), 1, 15, 19 process, 37 for Burkina Faso, 39b3 input from private sector, 37 coordination and integration "Managing Fiduciary Risk less important for, 40 When Providing Direct Bud- coordination with CFAAs, 82 get Support," 94 coordination within World questionnaires, use of, 25t1 Bank, need to improve, 58, risk definition used for, 10­11 59, 62­63 as secondary instruments, 40 description of, 19, 83­85 similarity to other instruments, government participation in 23, 65 process, 36 staff manual on, 38 institutional considerations and, statutory requirements behind, 42­43 51 as primary instruments, 40 diagnostic reviews of public finan- questionnaires and checklists, cial management, 101­3 use of, 24, 25t1, 43 See also assessment instruments revised guidelines for, 38, 62 definition of diagnostic review, scope and range of, 35, 45, 84, 102bA2.1 115tA3.1 dialogue between donors and significance of, 36 recipient governments, 16b1, standardized structure of, 40 57 Index 133 differences among assessment institutional considerations and, instruments, 40­41, 51­52 44 dissemination of assessments. See recommendations concerning, information sharing among 55­56 agencies relationship to CFAAs, 55­56 duplication and similarities of revised guidelines, xiii, 22­23, instruments, xiii, 20­21, 36, 38, 40, 44 35­39, 41, 44­45, 103 scope and range of, 35, 91­93, See also integration and coordi- 115tA3.1 nation significance of, 36 costs of, 50, 103 statutory requirements behind, 51 East Asian financial crisis of 1997- Estonia 1999, 5 baseline assessments for, 96 Eastern Europe. See EC and peer reviews of audit institu- OECD Support for Improve- tions, 97 ment in Governance and Ethiopia Management in Central and government-led PER, 18, 80 Eastern Europe (SIGMA) European Community EC and OECD Support for See also headings starting with Improvement in Governance "EC" and Management in Central CFAAs, collaboration on, 82 and Eastern Europe Public Financial Management (SIGMA) Assessment, 93­94 baseline assessments, 24, 25n2, expenditure programming 96­97 as component of public expendi- description of, 95­99 ture management, 27, 28b2, peer reviews 32f1 of public internal financial coverage by assessment instru- control systems, 98­99 ments, 116tA3.1 of supreme audit institutions, as PER focus, 58 97­98 external audits. See accounting, EC audits, 1, 15, 104bA2.2 reporting, and external audit building on UNDP CON- TACT Guidelines, 22 Fiduciary Review (World Bank), counterpart funds and, 91 73n1 criticism of, 26n7 fiduciary risk, 9­14 description of, 22­23, 90­94 See also DFID assessments of follow-up problems, 93 fiduciary risk 134 Assessing and Reforming Public Financial Management accountability, 12­13 EC Public Financial Manage- common definition, need to ment Assessment to address, establish, 55 93 corruption and, 33 Gold, Joseph, 13 definition of fiduciary, 10 good practices for public expendi- distinguished from development ture management, 57, 65, risk, 10 103­12 evaluation and development of diagnostic work, 105­6, goals, as recommendations, xvi 107­9 operational implications, 13­14 follow-up activity, 108­9 recommendations concerning, IMF's Code of Good Practices 55 on Fiscal Transparency, 20, risk, 10­12 111 financial integrity, 33 information sharing, 103, 108 Financial Sector Assessment Pro- performance measurement, gram (FSAP), 73n4 106­7, 110­12, 111bA2.5 Fiscal ROSCs. See IMF Reports on quality assurance, 108 the Observance of Standards Governance Operations Progress and Codes of Fiscal Trans- Indicators (GOPIs), use of, parency 65 follow-up and performance moni- governance principles, 33­34, toring, 65­66 41­44 to EC audits, 93 definition of governance, 63 EC Public Financial Manage- recommendations concerning, ment Assessment to address, 63­65 94 to Fiscal ROSCs, 87 Harmonization Group, 82 good practices requiring, 103, Harrod-Domar thinking, 5 108­9 Heavily Indebted Poor Country France initiative. See World Bank- receiving first loan made from IMF Public Expenditure World Bank, 6n4 Tracking Assessments and FSAP (IMF-World Bank Financial Action Plans for Heavily Sector Assessment Program), Indebted Poor Countries 73n4 (HIPC AAPs) fungibility of resources, 4 Herzegovina, 84 HIPC AAPs. See World Bank-IMF gaps in assessment instruments, Public Expenditure Tracking 46­47 Assessments and Action Plans Index 135 for Heavily Indebted Poor coordination with other World Countries Bank products, 87 Hungary, 96 description of, 20­21, 85­87 dissemination by governments, IGRs. See Institutional and Gover- 37 nance Reviews (IGRs), use of follow-up to Fiscal ROSCs, 87 IMF government participation in See also headings below starting process, 37 with "IMF"; World Bank- institutional considerations and, IMF Public Expenditure 43­44 Tracking Assessments and modular approach of, 68 Action Plans for Heavily overlap with other instruments, Indebted Poor Countries xiii, 20­21, 41, 44, 51 (HIPC AAPs) as primary instruments, 40 coordination with other large questionnaires, use of, 25t1, 37 development agencies, need scope and range of, 35, 45, 86, to improve, 56 115tA3.1 coordination with World Bank, significance of, 36 need to improve, 52, 54­55, skills of teams conducting, 62 66­67 information already available uniformity and structure of, 40 from, 52 IMF-World Bank Financial Sector online access to information of, Assessment Program (FSAP), 53 73n4 transparency analysis of, reliance IMF's Code of Good Practices on of other agencies on, 53 Fiscal Transparency, 20, 111 working group (with World information sharing among agen- Bank) on collaborative public cies, 37, 52­53, 55, 103, 108 expenditure work, 68­69 Institutional and Governance IMF Fiscal Management Assess- Reviews (IGRs), use of, 42, ment, 73n1 64­65 IMF Reports on the Observance of coordinated with CFAAs, 82 Standards and Codes of Fis- institutional considerations, xvi, cal Transparency (Fiscal 41­44, 63­65 ROSCs), 1, 15, 104bA2.2 draft PER guidelines on, 42 accounting issues, coverage of, problems created by lack of 83 understanding of, 50 background of, 5 integration and coordination for Burkina Faso, 39b3 benefits of, 60b4, 103 136 Assessing and Reforming Public Financial Management constraints on, 51­52 diture management, 27, 28b2, coordination between agencies, 32f1 37, 52, 56­57 coverage by assessment instru- coordination within World ments, 115tA3.1 Bank, 57­63 legislative approval as focus of efforts to strengthen, viii, 20­21 assessment instrument, 17 exchange of information and Lithuania, 96 programming intentions, 52­53, 55, 103, 108 Macedonia, 96 good practices recommended by Malawi and participatory CFAA, 51 PEFA, 59­61 DIFD participation in, 95 multidonor missions, problems "Managing Fiduciary Risk When of, 50 Providing Direct Budget problems of differences among Support" (DIFD paper), 94 instruments, 40­41, 51­52 methodology of study, 27­34 recommendation to increase, modular approach. See program- xiv­xvi, 45­46, 49­63 matic and modular approach Inter-American Development Mozambique Bank, 53, 82 as example of coordinated International Public Sector assessments, 57, 65, 82 Accounting Standards participatory CFAA, 51 (IPSAS), 111bA2.5 International Records Management New Partnership for Africa's Trust, 46 Development (NEPAD), international standards, 111bA2.5 73n1 See also good practices for public Norwegian Agency for Interna- expenditure management tional Development, 82 IPSAS (International Public Sector Accounting Standards), OECD 111bA2.5 See also EC and OECD Support for Improvement in Gover- Kenya, 80 nance and Management in Central and Eastern Europe Latvia (SIGMA) baseline assessments for, 96 Development Assistance Com- peer reviews of supreme audit mittee, viii, 50, 57, 65, 82, institutions, 97 101­12 legal and organizational framework public expenditure database, as components of public expen- creation by, 53 Index 137 operational implications of risk, recommendations for, xvii­xviii, 13­14 67­73 replacement of present instru- participation, 33­34 ments with streams of work, consent of government to 72 assessment, 38 PRSPs. See Poverty Reduction CPARs and, 84 Strategy Papers insufficient, effect of, 50­51 Public Expenditure and Financial PERs and, 79 Accountability (PEFA), viii recommendations concerning, mandate of, ix, 6n1, 53 55, 56­57, 63 on need to integrate World significance of, 41 Bank assessment instruments, peer reviews 59 of public internal financial con- study sponsored by, ix, 1 trol systems (SIGMA), 98­99 public expenditure management of supreme audit institutions defined, 2 (SIGMA), 97­98 distinguished from public finan- PEFA. See Public Expenditure and cial management, 2 Financial Accountability fiduciary objectives related to, 11 performance monitoring. See fol- good practices for (defined by low-up and performance Task Force on the Harmo- monitoring nization of Donor Practices), PERs. See World Bank Public 57, 65, 103­12 Expenditure Reviews link to effective development Peru, 82 outcomes, 5 PETS (Public Expenditure Track- major components and subcom- ing Surveys), 24 ponents of, 27, 28b2 Philippines, 82, 84 performance monitoring of, Poland, 96 65­66 Poverty Reduction Strategy Papers Public Expenditure Management (PRSPs), 21, 57, 60, 65, 69, Handbook (World Bank), 78 73n4, 93 Public Expenditure Management procurement. See Country Pro- (PEM) Core Diagnostic, 24, curement Assessment Reports 25t1 (CPARs) Public Expenditure Tracking Sur- programmatic and modular veys (PETS), 24 approach public financial management alternative approaches, 70t2 distinguished from public new framework for, 69f2, 73n6 expenditure management, 2 138 Assessing and Reforming Public Financial Management scope of, 112n1 programmatic and modular public investment programs approach to be developed, shortcomings of, 6n2 xvii­xviii, 67­73 World Bank Public Investment records management Reviews, 4 See also accounting, reporting, public records management, lack of and external audit coverage of, 46 lack of coverage of, 46 purpose of study, 1, 2­3 reporting. See accounting, report- ing, and external audit quality control of assessments, 38, risk, 10­12 52, 67, 73 Romania, 96 good practices for, 108 Rome Declaration on Harmoniza- Quality Assurance Group tion (2003), viii review of CFAAs and ROSCs. See IMF Reports on the CPARs, 38, 43, 47n4, 48n6, Observance of Standards and 67, 81, 85 Codes of Fiscal Transparency Quality Enhancement Reviews (Fiscal ROSCs) (QERs), 67, 73 rule of law, application of, 33­34 questionnaires and checklists, use of, 23­24, 25t1, 37, 43, 68 SIGMA. See EC and OECD Sup- good practices and, 112tA2.1 port for Improvement in for public records management, Governance and Manage- 46 ment in Central and Eastern for SIGMA assessment of audit Europe institutions, 98 similarities. See duplication and similarities of instruments recommendations, xiv­xviii, Slovak Republic, 96 49­74 Slovenia assessing of assessors, xvi­xvii, baseline assessments for, 96 66­67 peer reviews of supreme audit follow-up and performance institutions, 97 monitoring, 65­66 state and local fiscal issues, lack of institutional and governance coverage of, 46 considerations, xvi, 63­65 Strategic Partnership with Africa integration, coordination, and (SPA), 50­51, 82 cooperation to be improved, study xiv­xvi, 49­63 coverage of, 34­35 See also integration and coordi- methodology of, 27­34 nation purpose of, 1, 2­3 Index 139 sponsored by PEFA, ix, 1 Uganda Support for Improvement in Gov- DIFD and monitoring of fiduci- ernance and Management. ary risk in, 95 See EC and OECD Support duplication of World Bank mis- for Improvement in Gover- sions in, 50 nance and Management in government-led PER, 18, 80 Central and Eastern Europe participatory CFAA, 51 (SIGMA) U.K. Department for International Swedish International Develop- Development. See DFID ment Authority, 82 assessments of fiduciary risk UNDP CONTACT Guidelines, Tanzania 25n2, 104bA2.2 coordination among agencies in, description of, 99­100 57 EC Audits development from, government-led assessment, 22 17­18, 51, 61, 79, modular approach of, 68 110bA2.4 questionnaires and checklists, implementation of assessment use of, 24, 25t1, 68 recommendations in, 65 records management under, unrealistic demands placed on, 46 50 United Nations Development Pro- Task Force on the Harmonization gramme of Donor Practices CFAAs, collaboration on, 82 (OECD Development CONTACT. See UNDP CON- Assistance Committee), 50, TACT Guidelines 57, 82 taxation, lack of coverage of, 46 World Bank transparency, 33­34 See also Country Financial See also governance principles; Accountability Assessments IMF Reports on the Obser- (CFAAs); Country Procure- vance of Standards and Codes ment Assessment Reports of Fiscal Transparency (Fiscal (CPARs); headings below start- ROSCs) ing with "World Bank" CFAAs looking at, 42 Africa Region, use of checklist, IMF's Code of Good Practices 24 on Fiscal Transparency, 20, coordination with IMF, need to 111 improve, 52, 54­55, 62 Turkey, 41, 64, 73n1, 82, 84 coordination within, need to 2AC and EC audits, 92 improve, 57­63 140 Assessing and Reforming Public Financial Management coordination with other large questionnaires, use of, 24, 25t1 partner development agen- revision of guidelines, xiii, 22, cies, need to improve, 56 89 Country Analytic Website, 53 scope and range of, 115tA3.1 duplication of missions in Ugan- as secondary instruments, 40 da, 50 standardized structure of, 40 Fiduciary Review, 73n1 as successful collaboration first loan from, 6n4 between agencies, 53 Governance Operations updating of, 65 Progress Indicators (GOPIs), World Bank Public Expenditure use of, 65 Reviews (PERs), 1, 15, Institutional and Governance 104bA2.2 Reviews (IGRs), use of, 42, background of, 4 64­65, 82 combining with CFAAs into sin- need for other agencies to rely gle assessment instrument, 61 on analysis of, 53 consultation with nongovern- online access to information of, ment entities as part of, 37 53 coordination within World Quality Assurance Group. See Bank, need to improve, quality control of assessments 57­63 terminology of, causing confu- description of, 17­18, 77­80 sion, 62­63 draft guidelines for, 38, 41, 59, working group (with IMF) on 62, 79 collaborative public expendi- government-led PERs, 17, 41, ture work, 68­69 79­80, 110bA2.4 World Bank-IMF Public Expendi- government participation in ture Tracking Assessments process, 36 and Action Plans for Heavily institutional considerations and, Indebted Poor Countries 42 (HIPC AAPs), 1, 15, less frequent use of, 41 104bA2.2 modular approach recommend- accountability increased by, 37, ed for, 67 65 overlap with other instruments, benchmarks used by, 21­22, xiii, 41, 45, 51, 58 111bA2.5 as primary instruments, 40 description of, 21­22, 87­90 Public Sector Governance experienced staff used for, 47n3 Board as responsible for, 78 interagency consultation, need scope and range of, 34, 41, 45, to improve, 54 78, 115tA3.1 Index 141 significance of, 36 skills of teams conducting, 66­67 taxes inadequately covered by, 46 World Bank Public Investment Reviews, 4 T his book was written as part of the work of the Public Expenditure and Financial Accountability (PEFA) Program, a partnership of the World Bank; the European Commission; the International Monetary Fund; the Strategic Partnership with Africa; and several bilateral donor agencies, including those of France, Norway, Switzerland, and the United Kingdom. This study compares and contrasts the various instruments and approaches used by these organizations to assess and reform public expenditure management systems in developing and transitional countries. It finds weaknesses in these instruments, including overlap and duplication in their technical scope and coverage, as well as insufficient or inconsistent coverage in some areas. In addition, countries often are subjected to multiple assessments and multiple missions by the donors, which can impose heavy transactions costs on government agencies. Furthermore, the instruments have a variety of objectives--fiduciary, surveillance, and capacity building--which are divergent and potentially conflicting. This study recommends a new approach that is country led, multidonor, medium term in orientation, focused on better management of the budget, and supplemented by donor aid funds, as a key mechanism to reduce poverty and attain other policy goals. It provides concrete and practical recommendations for achieving four important objectives: · Streamlining the coverage of instruments to avoid unnecessary duplication · Enhancing collaboration between donors, governments, and other stakeholders · Providing a more complete, accurate, and timely assessment of fiduciary risk · Improving the ultimate development impact of assessment and reform work This book will be of interest to development practitioners in the area of public finance, finance ministers, policy analysts, and students and scholars of international development. 0-8213-5599-6