Budget Discipline Responsibility Sustainability Information External Scrutiny 78673 Accountability Anti-Corruption Transparency Equity Transport Governance Indicators for Sub-Saharan Africa Angela Christie Don Smith Kate Conroy Working Paper No. 95 Transport Governance Indicators in Sub-Saharan Africa Transport Governance Indicators in Sub-Saharan Africa Angela Christie Don Smith Kate Conroy January 2013 The SSATP is an international partnership to facilitate policy development and related capacity building in the transport sector in Sub-Saharan Africa. Sound policies lead to safe, reliable, and cost-effective transport, freeing people to lift them- selves out of poverty and helping countries to compete internationally. ******* The SSATP is a partnership of  38 SSA countries  8 Regional Economic Communities  2 African institutions: UNECA and AUC  Financing partners for the Second Development Plan: European Commission (main do- nor), Austria, France, Norway, Sweden, United Kingdom, Islamic Development Bank, Af- rican Development Bank, and World Bank (host)  Many public and private national and regional organizations ******* The SSATP gratefully acknowledges the contributions and support of its member countries and partners. ******* More publications on the SSATP website: www.ssatp.org The findings, interpretations, and conclusions expressed here are those of the author and do not necessarily reflect the views of the SSATP or its partners. © 2013 The International Bank for Reconstruction and Development / The World Bank All rights reserved. Table of Contents Acronyms and abbreviations ___________________________________________ vii Foreword ___________________________________________________________ ix Acknowledgements __________________________________________________ xi Executive summary _________________________________________________ xiii Introduction _________________________________________________________ 1 The evolution of a methodology: technical approach and stakeholder feedback __ 5 First phase _________________________________________________________ 5 Second phase ______________________________________________________ 11 Selecting and testing transport governance indicators ______________________ 15 Critical governance indicators: Make or break decision making ______________ 15 Quality assessment of the selected indicators using revised criteria ____________ 27 Subsector relevance and indicator potential ______________________________ 30 Composite indicators: Complexity made simple? _________________________ 31 The importance of perception ________________________________________ 32 A workable approach to measuring and managing transport sector governance _ 35 A monitoring and evaluation system: Preliminary design and sample scoring ___ 35 Availability of data __________________________________________________ 48 Institutional arrangements: Balancing internal and external engagement in data gathering and advocacy ______________________________________________ 48 Methodology for data collection _______________________________________ 56 Interventions to support building M&E capacity __________________________ 58 Intervention options in the governance cycle _____________________________ 58 Conclusions and recommendations _____________________________________ 61 Appendix A. Long list of indicators ______________________________________ 65 Appendix B. Short list of indicators _____________________________________ 71 Appendix C. SSATP transport governance indicators _______________________ 75 Bibliography ________________________________________________________ 91 v Acronyms and abbreviations CSO civil society organization CSPR Civil Society for Poverty Reduction (NGO) DfID Department for International Development (UK) FIDIC International Federation of Consulting Engineers GDP gross domestic product HDM-4 Highway Design and Maintenance Standards Model IFMIS Integrated Financial Management Information System IMF International Monetary Fund IRTG Improved Road Transport Governance Initiative JISR Joint Infrastructure Sector Review KPI key performance indicator M&E monitoring and evaluation MDA ministries, departments, or agencies MoF ministry of finance MoR ministry of roads MoT ministry of transport MoW ministry of works MTEF medium-term expenditure framework NGO nongovernmental organization OAG Office of the Auditor General OECD Organisation for Economic Co-operation and Development OECD-DAC OECD—Development Assistance Committee PEFA Public Expenditure and Financial Accountability PEMFA Public Expenditure Management and Financial Accountability PPRA Public Procurement Regulatory Authority PRSP Poverty Reduction Strategy Paper PRTSR Poverty Reduction and Transport Strategy Review RAG red-amber-green SSATP Sub-Saharan Africa Transport Policy Program TSFG transport sector governance framework TSIP Transport Sector Investment Programme UEMOA West African Economic and Monetary Union UNECA United Nations Economic Commission for Africa USAID U.S. Agency for International Development VFM value for money vii Foreword Good governance in transport is critical to sustain the development of the sector and the economic growth in Africa. With an expected growth of six percent per year be- tween 2010 and 2040, responding to the rapidly increasing demand for transport in- frastructure and mobilizing funds estimated at $18 billion per year will be a huge challenge. This challenge can be met by optimizing the sector efficiency and as a result minimizing funding requirements in a constrained budgetary environment. Reducing cost and time overruns on civil works contracts to stay within contract values, or re- ducing overengineering in road design would on average generate savings of 30 per- cent of road construction costs. At the same time, addressing policy and institutional issues will be required to achieve the objectives and sustain the results of development programs in the sector. Indeed, a review of road transport costs by the World Bank has concluded that the benefits from road improvements were not transferred to the populations when the right policy environment was not in place, such as when road cartels exist as a result of a restrictive regulatory framework. Similarly, a recent review of the European Development Fund has concluded that the European Commission (EC) is partially effective in its support for a sustainable road network in sub-Saharan Africa for reasons such as underfunding of road maintenance and overloading of trucks, which have resulted in accelerated deterioration of the road network and cost- ly rehabilitation earlier than anticipated. The creation of road funds promoted by SSATP was one step towards a more efficient framework for road management. The underlying reasons, other than limited funding, that currently hamper the transport sector efficiency reflect a poor governance environment which is not conducive to improving this efficiency. The EC has put governance at the top of its agenda. In recent years, the Commission has made major efforts to implement this commitment through the development of a comprehensive policy framework on governance and the promotion of the concept of governance in sector programs. The first dimension of governance is about rules, interests, resources and power and how power is used and how institutions function. The second dimension is about key principles such as participation, inclusion, trans- parency and accountability. The third dimension is that governance encompasses several themes that impact on the state's ability to serve its citizens: democratization, human rights, rule of law and administration of justice, sole of civil society, public administration reform and decentralization. In the transport sector, the 2000 EC Communication considered that governance is a prerequisite to sustainable develop- ix Transport governance indicators in Sub-Saharan Africa ment of the sector and analyzed governance in the transport sector through the lens of the three dimensions above. In 2011 the EC launched a study on “Assessing Gov- ernance in the Transport Sector in Sub-Saharan Africa� in order to assess how to in- crease the impact and sustainability of infrastructure development on poverty reduc- tion and socioeconomic growth. This effort is now being pursued with the prepara- tion of a document to provide guidance and support to stakeholders of the transport sector in beneficiary countries to address governance issues when designing or im- plementing transport projects and programs. The present SSATP publication on governance indicators in the transport sector is an important contribution to moving the governance agenda forward in that sector. The paper identifies a critical subset of indicators that can be used to demonstrate in a clear and measurable way, the level and quality of governance in a particular country / sector / sub-sector. These indicators provide a tool for countries not only to measure governance in the transport sector, but also to monitor results on actions that are taken in this area. I trust that countries will progressively adopt these indicators and that they will be reflected in the programs supported by development partners to con- tribute to render aid more effective. Kristian Schmidt Director for Sustainable Growth and Development, DEVCO C Directorate General for Development and Cooperation-EuropeAid European Commission x Acknowledgements The study initiated by Stephen Vincent, former SSATP Program Manager, was car- ried out between September 2011 and January 2012 in four countries, Mali, Tanzania, Kenya and Zambia. The work was undertaken under the leadership and supervision of Anca C. Dumitrescu, World Bank task team leader and in charge of governance issues within SSATP. The authors, Don Smith and Kate Conroy from IMC World- wide and Angela Christie from ITAD Ltd would like to thank the peer reviewers Gael Raballand and Andreas Kopp from the World Bank. The authors are also grateful to Jean-Noel Guillossou for his valuable comments as SSATP Program Manager and to the publication officer, Monique Desthuis-Francis, who finalized the manuscript for publication. xi Executive summary Good governance—or the absence of it—has concerned policy makers and other stakeholders in the transport sector for decades. Most stakeholders recognize that effective governance is crucial if improvements in transport infrastructure are to en- dure and contribute to sustainable economic growth. In Africa, billions of dollars have been spent on improving and rehabilitating transport infrastructure, but it has been long recognized that the poor performance of the transport sector is due to far more than merely inadequate finance or technical capacity constraints. Governance has been indeed the subject of increasing attention and surveys are regularly carried out by highly respected organizations, measuring people’s perceptions of whether “things are getting better�. The underlying reasons for poor governance are complex, and finding a set of easily collectible indicators that can be used not only to track progress but also to advocate for policy change has so far proved challenging. To support further efforts, the SSATP commissioned this study to identify suitable transport governance indicators. The assignment was to carry out a survey and collect data that would identify the main transport sector governance issues in four African countries—Mali, Tanzania, Kenya, and Zambia—and to recommend a methodology for data collection and monitoring that could define a set of easily collectible advocacy performance indicators to meas- ure transport governance at the national level. The indicators would have to be repli- cable in a consistent fashion in several or all countries, thereby producing a compara- ble measure of results achieved in the implementation of good governance policies. Poor governance occurs at many levels of the policy cycle—from the ways in which legislation is drafted and regulations, systems, and procedures are worded and applied in practice, to how services are eventually delivered to the users of transport and whether their expectations are met. The last 10–15 years have seen a vast expansion in the number of indicators being used to measure performance across many sectors (not least transport) and across all phases of the policy cycle. This process is helping to introduce transparency and accountability into the routine operations of many government ministries, departments, and agencies. The practice has complemented a long-term transition from command-type public sector management structures to a more commercialized approach to the transport sector (e.g., the introduction of se- cond-generation road agencies). This transition has included separation of the func- xiii Transport governance indicators in Sub-Saharan Africa tions of policy formulation, regulation, planning, programming and budgeting, im- plementation, and operational activities. These measures all serve to improve the quality of public financial management and budget discipline (and, for example, help enable general budget support). Much investment has been made in transport sector performance. And yet, even though many highly respected surveys do show improvements in perceptions over recent years, significant negative perceptions persist. If improved institutional struc- tures, systems, and procedures are really in place, and to some extent are being fol- lowed, then what is it that is not happening? And how does this relate to governance? And can we shine a light on where things can be done better? Or can we at least high- light success stories where they occur, which might then be replicated more widely? This paper sets out to identify a critical subset of governance indicators in the transport sector that can be used to demonstrate in a clear, measurable way the quali- ty of governance in a particular country, sector, or subsector. By means of consulta- tion with key transport sector stakeholders, it examines transport sector governance issues in four pilot countries in order to determine whether there is a consensus on what transport sector governance means in practice; why it matters; how it can be measured; and in what priority ways improvements in governance might make a real difference in the sector and its contribution to national development. At its core, the study attempts to reduce the indicator set to what is at the heart of the governance matter. For example, a shrinking budget is sometimes cited as the main cause of inef- fective delivery, and yet in reality this rarely explains the difference when comparing planned and actual spending. Rather, the primary problem is inefficient management and the political economy of rent seeking—aspects that can be measured through indicators such as cost or time overruns or contract variations. The technical approach to the study has drawn on the considerable literature that explores governance in general and the transport sector in particular. A framework of “second-generation� indicators was used to guide and shape the investigation. Com- plementing earlier work linking governance and development and following the re- port by Knack et al. (2002), we selected second-generation indicators as those most likely to lead to practical reform because they are generated through a transparent process, are available across many countries, and are accurate and specific. Through- out the work, the stakeholder perspective was of primary concern. Consequently, this perspective shaped both the process and the outputs. This led to a paper that has sought to avoid abstract governance definitions, complex composite measures, and theoretical solutions, which are often perceived by stakeholders to be significant ob- xiv Executive summary stacles to understanding, ownership, and action. More practically, it has meant that the study adopted a broad and shared understanding that governance is best under- stood as the “exercise of authority with discretion and integrity,� while recognizing that both “supply� and “demand� side governance should be considered to ensure an appropriate balance between transport sector delivery and external scrutiny. This report describes the process followed to select potential indicators from an ex- tensive list. The “candidate� indicators were matched against a set of dimensions that reflected different aspects of governance 1 that many involved in the sector would rec- ognize. The indicators were also assessed against a set of criteria that would validate them:  Are the indicators actionable—that is, can the primary institution involved “do things better� in response to the findings?  Are they credible, nationally ownable?  Are they relevant—do they capture a critical dimension of the quality of gov- ernance?  Are they sensitive to changes in the underlying phenomena?  Are they understandable?  Are they available—are the data required to measure the indicator available?  Are they reliable—can the data be trusted?  Can external agencies use the results to promote progress? The indicators were then tested against other quality assessment criteria to determine whether they could be selected as a mutually reinforcing “set.� Even though a rigorous methodological process was used to develop the indicators, it was essential that they be tested in the field against the criteria just listed. The value of visiting the four pilot countries lay in being able to match possible indicators against the daily, practical, real experiences of those dealing (and living) with aspects of—or the absence of—governance. The time allotted to the country visits was limited, but to the greatest extent possible we met with people from across the public and private sectors and from civil society, from the various transport subsectors, and from differ- ent governance levels—from the highest officials to relatively junior employees. Good governance is recognizable no matter the perspective. 1 These were later refined into a more intuitive transport sector governance cycle. xv Transport governance indicators in Sub-Saharan Africa Table 1 Proposed Indicators Transport Governance for Sub-Saharan Africa Governance area Indicator 1. Institutional mandates Clarity of and distinction between mandates and responsibilities of and responsibilities key ministries, departments, and agencies in the transport sector 2. Strategic priorities Coherence of transport sector policy and extent to which its prioriti- zation process is based on objective criteria 3. Budget discipline Budget allocations based on reliable financial forecasts and aligned to priorities based on objective criteria 4. Program design Quality and use of key performance and value for money indicators 5. Procurement Comprehensive and timely public disclosure of transport sector pro- curement plans (including bidding opportunities, contract awards, and data on resolution of procurement complaints) 6. Delivery Comprehensive time and cost reports on progress of work for major (top 10) transport sector projects, disclosed to the public in a timely and accessible manner 7. Sustainability Transport sector agency has established benchmarks for routine and periodic maintenance of assets and allocates the budget accordingly. 8. Information dissemination Transparency and timeliness of annual budget and expenditure dis- closures 9. External scrutiny Rules applied to the membership and appointment process for key transport sector governance boards 10. Accountability Percentage of recommendations from independent technical and financial auditor reports implemented within one year In summary, this study sought to identify indicators (listed in table 1) that offer the potential to credibly measure governance in the transport sector, where this potential is understood to be linked not only to the relevance of the indicator to key issues af- fecting the sector but also to the capacity of local institutions to embrace the indica- tors by monitoring and recording results and acting on findings. The study recognizes that the majority of indicators rely on data generated by government bodies (i.e., ministries, departments, and agencies) and less so on perception. But this does not necessarily limit the scope for external scrutiny and advocacy. That said, we believe that for the indicator set to be sustainable and a sound basis for the initiation (and improvement) of engagement between internal and external stakeholders, senior offi- cials in the respective ministries responsible for transport across the region should take clear ownership of, and commit to, the further development and use of the indi- cator sets. They possess much of the data and can exert pressure on the other organi- zations responsible for generating other elements. Where an outside or ad hoc body is xvi Executive summary responsible, there is sometimes a sustainability issue, although many of these non- transport sector institutions (e.g., auditors, the media, and think tanks) have a critical role to play. As for perception data, within the context of this study it was hard to identify sector, or even specific subsector, indicators for which data sets would be robust enough to pass the stringent selection criteria applied. It was thus concluded that others may be better placed to continue to conduct the large-scale surveys need- ed to provide the necessary statistical integrity. The process adopted and the tentative findings produced suggest some interesting first conclusions and some options for the consolidation of both the approach and the findings as the basis for a wider rollout. Key findings and conclusions The following key findings and conclusions emerged from this study.  It was initially suggested that a transport governance indicator be proposed for each subsector, but because of the many cross-sectoral issues identified in the master indicator list developed in the early stages of the study, it was deemed more appropriate and practical to use governance dimensions to organize the indicators.  Key transport governance issues identified in the study across the four pilot countries included the following: political interference in projects and key gov- erning board appointments; limited or non-existent integrated transport sector policy; identification of new and strengthened institutional and regulatory ar- rangements, together with continued evidence of on-going blurred boundaries, unrealistic budgets, and no objective criteria for prioritization; long-term un- derinvestment and prioritization in maintenance across subsectors; and on- going dominance of roads across the transport sector.  Easily understood, precise, and actionable indicators are needed. Such a list would exclude some of the internationally recognized robust indicator sets based on composite scores (or indexes). Such sets are thought to obscure reali- ties through artificial “smoothing� and to reduce engagement because of per- ceptions of external ownership.  There is an expressed and credible desire among stakeholders to own indica- tors, but realization of this desire requires taking institutional considerations seriously. Technical competence needs to be balanced against political credibil- ity when advocating particular policy actions. xvii Transport governance indicators in Sub-Saharan Africa  The development of the indicator set is only the first step. Monitoring will gen- erate findings indicating that the indicator set needs to be acted on in ways that lead to measurable results, which may require (external) advocacy or support.  SSATP should receive support to conduct a pilot test in one or more countries in the study (perhaps in all four) of the method proposed (red-amber-green) to rate the indicators and provide a baseline for a needs assessment and priori- tization of actions in these countries for improving governance in the transport sector. This report offers a starting point.  Finally, SSATP should receive support to identify and develop more accessible ways to present the proposed indicator set to key stakeholders in each pilot country. For many stakeholders, their interest in and potential ownership of the indicators may be better achieved through short presentations, short sum- mary guides, and specific forums or meetings. xviii Introduction This paper examines transport sector governance issues in four pilot countries by means of a process of consultation with key transport sector stakeholders. The con- sultation was intended to determine whether there is a consensus on what transport sector governance means in practice; why it matters; how it can be measured; and in what priority ways improvements in governance might make a real difference to the sector and to its contribution to national development. The technical approach draws on the considerable literature that explores governance in general and the transport sector in particular. A framework of second-generation indicators 2 was used to guide and shape the investigation. However, throughout the study process—from defining governance as a concept, through the evolution of the study methodology, to identifying the key issues and associated indicators—the stakeholder perspective was of primary concern and so shaped both the process and the outputs. This resulted in a study that sought to avoid abstract governance defini- tions, complex composite measures, 3 and theoretical solutions, because they are per- ceived by stakeholders to be significant obstacles to understanding, ownership, and action. 4 More practically, the study adopted a broad and shared understanding that governance is best understood as the “exercise of authority with discretion and integ- rity� and that, as a corollary, poor governance is about the “abuse of power for inap- propriate purposes and often personal gain.� 5 This working definition facilitated communication and did not limit the scope of inquiry. Moreover, it neither con- 2 Second-generation indicators complement an earlier generation of work that helped identify the link between governance and development. Second-generation indicators are selected as those most likely to lead to practical reform because they are generated through a transparent process available in many countries that is accurate and specific (see Knack et al. 2002). 3 A composite indicator or index is an aggregation of indicators into a single measure, intended to simplify a complex reality but often criticized for obscuring or smoothing those realities. 4 A considerable literature covers the intellectual challenge of measuring an inherently abstract concept—see, for example, Thomas (2007). 5 This is in line with definitions of public sector governance—see, for example, World Bank (2007b). It is also consistent with OECD’s DAC Guidelines and Reference Series: Good Practice Guidance for Development Cooperation, which uses a similar definition for public sector gov- ernance (OECD 2006). Also see World Bank (1989) and IMF (2007). 1 Transport governance indicators in Sub-Saharan Africa strained discussion nor limited the selection of indicators to those that might broadly be referred to as “supply side� governance. 6 Indeed, the critical role played by civil society in relation to the state was acknowledged by all stakeholders. Ultimately, the approach adopted focused on the search for indicators that stakehold- ers agree offer the best (albeit not perfect) opportunity to gather evidence of the ex- tent to which authority is being exercised with integrity—or evidence of the results of efforts to resist the abuse of power. It also led to a focus on how stakeholders believe such indicators could (not should) be measured and on their suggestions for ways in which targets might be set and reached through a program of meaningful change. Inevitably, because of the range of subsector stakeholders interviewed, some of the indicators and intervention options identified are not equally relevant to all transport subsectors; some, however, apply to the full range. What the proposed indicators have in common is that they are widely understood to reveal something critical about gov- ernance in the transport sector, as learned from the stakeholders across the countries involved in the study. Furthermore, because they are actionable 7 in the view of these stakeholders, they appear to offer the potential for impact on transport sector per- formance in ways that would have positive consequences for both national develop- ment and confidence. The rest of the paper is organized as follows. Chapter 2 explains the evolution of the technical approach incorporating stakeholder feedback. Chapter 3 presents indicators the most advocated based on the “governance cycle framework.� These indicators are assessed for quality using the study criteria, and the link between these indicators and 6 Many definitions of governance such as this one from the UN Development Programme, focus on the role of the state: Governance is the exercise of economic, political, and administra- tive authority to manage a country’s affairs at all levels. Good governance, thus, signifies the competent management of resources and affairs in a manner that is open, transparent, honest, accountable, equitable and responsive to people’s needs and problems. However, governance is also about the demands made by citizens, civil society organizations, and other non-state actors to hold the state accountable and to make it responsive to their needs. As the experience of many countries has shown, the demand side articulation has led to efforts by supply side actors to strengthen state organs in order to make them more transparent, accountable, and participa- tory. Thus governance is about the supply-demand relationship. 7 Governance and Anti-Corruption Strategy (World Bank 2007a, 34–35) defines the need for actionable governance indicators that are commonly characterized as “narrowly defined and disaggregated indicators that focus on relatively specific aspects of governance and could pro- vide guidance on the design of reforms and monitoring of inputs.� For details, see https://www.agidata.info/main/AboutAGI.ashx. 2 Introduction the various transport subsectors is clarified. Options for using existing indicators and data sets to substitute for or support these indicators are also considered. Chapter 4 suggests systems and institutional arrangements for monitoring and evaluation and demonstrates the proposed scoring system for each indicator. Options for interven- tion at both the institutional and indicator levels are explored. Chapter 5 reaches some conclusions and explores options and recommendations for the next steps, in- cluding wider rollout. Appendix A and B contain the long and short lists, respectively, of the indicators considered. Appendix C consists of worked examples of SSATP transport governance indicators. 3 The evolution of a methodology: technical approach and stakeholder feedback The “Inception Report for the Study� described a revised methodology for the three phases of the study: (1) design, (2) country visits, and (3) analysis and reporting. First phase The eight key steps in the first study phase (the design phase) are described here. Step 1. This step was a review of literature related to the study. 8 Step 2. A long list of over 170 candidate indicators was drawn from the literature re- view and organized using a framework of six governance dimensions (see figure 2.1) and definitions (table 2.1). The indicator long list appears in appendix A. Figure 2.1 Framework of dimensions for transport sector governance organization Anti-Corruption Effort Regulatory and Financial and Value for Administrative Money Controls Arrangements Transparency and Third Party Access to Information Engagement Accountability Equity of Benefits 8 Key reference documents appear in the bibliography. A list of the more than 210 documents reviewed as part of this study is available upon request. 5 Transport governance indicators in Sub-Saharan Africa Table 2.1 Definitions of Dimensions for the Indicator Organizing Framework Dimension Definition Financial management and value for Extent to which sustainable financial institutions and arrangements have money systems been put in place for maximum rural and urban transport service delivery Administrative and regulatory proce- Whether efficient and effective administrative units and regulatory ar- dures rangements have been made to maximize delivery of rural and urban transport benefits Third-party engagement Degree to which consultation with civil society and the private sector over transport sector development issues helps to ensure the identification of, and action to meet, priority needs Transparency & access to information Whether relevant and reliable information is available to all stakeholders Anticorruption effort Evidence of specific investment and demonstrable progress to reduce cor- ruption Accountability How leaders account for progress in delivering beneficial transport services using public money Equity of benefits Whether pro-poor and pro-growth transport sector policies deliver measur- able transport benefits to individuals and society Step 3. This step consisted of developing clearly defined and justified selection crite- ria. These criteria were then used to score and thereby reduce the indicator long list to a more manageable number and to steer the survey work during the country visits. During a midterm reflection and lesson learning exercise at the end of the second country visit, the criteria were revised slightly (see table 2.2). The shortened list of indicators generated by this process appears in appendix B. This process of indicator selection does not mean that country visits were constrained by a focus on indicators on the list. Rather, the purpose of the list was to offer a starting point for the country visits and to frame conversations that might otherwise range too widely. 6 The evolution of a methodology Table 2.2 Indicator Assessment Criteria and Definitions Revised criteria Definition Actionable  Indicator is narrowly and explicitly defined to provide clarity on the options to be considered in determining what steps can be taken to improve its score.  Knowing about the score will enable an organization or its key stakeholders to do things bet- ter or more effectively.  An institutionalized procedure is either in place or could reasonably be set up to collect data on the proposed indicator in the future. Credible  Indicator does not set direction for progress (and so is neutral) or say when change will be achieved.  “Lead� indicators located in the arena of formal rules will be avoided if possible.  Indicator is appropriate to the user’s need and unduly affected by exogenous forces. Nationally ownable  Indicator resonates with the intended audience and is sensitive to concerns of government.  Data are provided by politically acceptable sources and that can be embraced by reformers.  Indicator is defined in a way that permits meaningful discussion on the appropriateness of any given rating.  Data can be easily updated by country champions or members of the public with minimum specialist knowledge.  Indicator is as consistent as possible with those already in use. Relevant  Indicator captures a critical dimension of the quality of governance.  Indicator reflects important issues that warrant high-level policy advocacy.  Indicator has potential to advance constructive development policy in the transport sector. Sensitive  Indicator varies sufficiently to allow measurement of changes in the underlying phenomenon.  Unit of measurement is conducive to time-bound targeting.  Interventions can affect this indicator. Understandable  Indicator is easy to understand by people who are not experts.  Indicator is an unambiguous measurement that is intuitive in the sense that it is obvious what it is measuring and how it would be interpreted in practice.  Indicator makes the same sense to all; easy to communicate.  Potential user’s capacity to absorb information is respected. Available  Data source exists—as primary data (collected by in-country institutions such as the national statistics office) or as secondary data (other organizations).  Data collection is frequent or regularized without high cost or risk.  There is a minimal time lag between the collection and reporting of data to ensure that indica- tors are reporting current rather than historical information.  Information can be gathered while there is still time to act. Reliable  Data are trustworthy and defendable.  Data are replicable through a well-documented process.  Measurement process is methodologically sound.  Data do not change according to who collects. 7 Transport governance indicators in Sub-Saharan Africa Step 4. The reduced indicator list was scored. The highest scoring indicators were assessed as an “indicator set,� using basket criteria 9 (these basket criteria were also revised during country visits)—see table 2.3. Table 2.3 Basket of Indicator Quality Assessment Criteria Criterion Definition Balance between quantita-  Basket includes both quantitative and qualitative indicators. tive and qualitative  Some variables are suitable for rigorous quantitative analysis. measures  Some variables are included to allow measurement of subjective judgment capturing how key stakeholders perceive the governance environment and outcomes in the transport sector. Balance among policy, sys-  The basket of indicators contains indicators that represent all key stages in the tems, and outcome policy cycle, from policy formulation through systems to outcome in order to balance the mix of lead and lag indicators. Coverage across Subsectors  Basket covers all transport subsectors because a sufficient number of indica- tors can be easily adapted to measure the same or similar outcomes in subsec- tor domains. Equity for Stakeholders  Basket will generate data that can be disaggregated to allow measurement of progress or impact on different demographic groups, particularly men and women, ethnic and tribal, subnational, professional.  Variables will allow measurement of the underlying factors that shape power relations and changes in power relations between state and society and be- tween different groups in society.  Variables include direct or proxy measures of the empowerment of the poor. Regional and national appli-  A sufficient number of indicators are applicable at the regional level to allow cation comparisons between regions.  A sufficient number of indicators are consistent with those used in other Sub- Saharan Africa countries so that comparisons can be made between nations. Step 5. Local partners conducted a preliminary assessment of the availability of data to support indicators. An extract of one such assessment (Kenya) is provided in table 2.4. The work of refining the assessment of data availability continued throughout the 9 The purpose of the basket criteria was to avoid a situation in which the indicator set became simply a list of indicators that individually satisfied the indicator assessment criteria, but which collectively did not provide a balanced set. The criticism that many indicator assessment crite- ria (such as the mnemonic “SMART�) overly focus on the quality of the individual indicator at the expense of the collective is set out in the Department for International Development (DFID)-commissioned paper by ITAD (2011). 8 The evolution of a methodology country visit phase. The selection of an appropriate partner able to access this infor- mation and reach these judgments was critical to this assignment. Table 2.4 Snapshot of Preliminary Data Availability Assessment, Kenya Indicator Data /Documents Availability Possible Institutional Sources Requirements Dimension 1: Financial Management and Value for Money Systems Funds for maintenance released in Transport sector budget; Yes, disburse- Relevant minis- an appropriate and timely fashion transport sector disbursement ment of fuel levy try/department records (to the right agency) records (amounts and timing); funds (possible only available sub-sector disbursement rec- from Road Fund Board) ords (amounts and timing) (all disclosing maintenance figures). Composition (type of spending) of Transport sector budget down Yes, disburse- MoF,, relevant ministry / actual public expenditure in line by department and major ment of funds department records with original budget item; actual expendi- expenditure ture record by department and major budget item Transport agency receives regular Departmental / divisional finan- Yes, funds ex- MoT, MoR, sub-sectoral and accurate reports from its de- cial expenditure reports penditure returns departments partment / divisions on the use of from agencies to funds allocated to them. MoR, MoT & MoF For domestically financed expendi- Fiscal projection (grants / loans); Budget and ex- MoF, ministry of plan- ture (i.e. excluding donor financed capital and recurrent budget for penditure records ning and Vision 2030 elements) Capital expenditure / all transport sub-sectors; capital available total expenditure as a percentage and recurrent expenditure for of capital budget / total budget all transport sub-sectors Strategic national priorities are Medium term sector strategy / Yes, strategic MoF, ministry of plan- matched with budget allocations national development plan; national priorities ning and Vision 2030 (this may be covered above ministry, department or agency allocations Maintenance spent per km (on Maintenance expenditure by Yes, from agen- Agencies’ annual main network categories) broken sub-national area (province, cies but not readi- maintenance expendi- down by provinces / regions over district); maintenance expendi- ly available ture reports past three years ture by network category Note: MoF = ministry of finance; MoR = ministry of roads; MoT = ministry of transport; MTP = medium-term plan. Step 6. In this step, meetings were held for the local partners in each country in ad- vance of the arrival of investigators. A generic list of organizations and individuals 9 Transport governance indicators in Sub-Saharan Africa was circulated to local partners and used as the basis for making preliminary contact with stakeholders and scheduling meetings. Step 7. A short perceptions survey (eight questions) was developed to allow data collection to support comparison of the perceptions of the types of stakeholder con- sulted. It was to be used as a tool to frame the discussions, particularly among stake- holders where it might otherwise proceed less easily. The limited coverage possible during such short visits meant that only very tentative inferences could be made from the findings generated. The perception survey questions appear in table 2.5. Table 2.5 Perception Survey Questions To some extent Not very much I don't know Very much I disagree Survey question: To what extent do you agree with the following statements? 1. The government gets its priorities right with regard to transport sector spending. 2. The public is adequately informed on how government spends public money on transport provision. 3. There are reasonable opportunities for new market entrants to win transport concessions. 4. Major transport contracts are awarded on a best value for money basis. 5. It is now rare for a bribe to be paid in order to access or speed up a transport sector service (e.g., driving license or customs clearance). 6. I believe corruption allegations are generally properly investigated and can be successfully prosecuted. 7. If public sector officials in a "lead agency" override regulations and proce- dures they will be sanctioned. 8. Complaints from the public are generally handled fairly. Step 8. Arrangements were made for the one-week visits to each of the four pilot countries. Some reflection at the end of the design phase led to the following key observations:  Numerous indicators are already in circulation.  Identification of those with greatest potential will require adherence to the quality assessment criteria developed for the study—but stakeholder views will be critical. 10 The evolution of a methodology  Ultimately, the extent to which indicators are understood, can be monitored, and can help match solutions with “responsible� institutions will be strong signals of potential and productive use. Second phase The second study phase—the country visits—involved visits of one-week duration to four pilot countries. Discussions were held with 20–30 stakeholders in each country to gather perspectives on which aspects of governance are most critical to transport sector performance. Stakeholders included those from civil society organizations, the private sector, public administration, and political parties. Also during the visits, data were gathered on key current governance issues and solutions that work. Inevitably, it was not possible to produce a comprehensive report on governance issues across all subsectors after such short visits. However, country visit reports incorporating key findings from individual country visits and generated through cross-country compar- isons were produced. Figure 2.2 Phases of a “Governance Cycle� Appropriate to the Transport Sector Budget Discipline 11 Transport governance indicators in Sub-Saharan Africa Three important process issues emerged during the data collection phase as a result of stakeholder feedback, all of which led to changes in the overall methodology: 1. The emergence through conversation of the notion of a governance cycle. Alt- hough the governance dimensions and indicator list helped frame the scope of the inquiry at the design stage, during the visits conversations were inevi- tably shaped by stakeholders’ sectoral experiences and by their specific role in the management of that particular subsector. This role was often closely linked to a phase in the cycle of management activity (such as the procure- ment phase or the delivery phase). What then emerged quite naturally from the collated stakeholder interviews was a list of governance indicators related to “phases� in the management of the transport sector. Those phases for which indicators were identified appear in the “cycle� in figure 2.2, which presents a phasing of activity in which each phase requires the appropriate governance to secure good outcomes. 10 Figure 2.3 A Governance Cycle Framework Based on Transport Planning, Management, and Oversight Institutional and Strategic Framework Institutional Mandates and Strategic Priorities Responsibilities Sectoral Resource Allocation Framework Budget Discipline Sectoral Development Framework Programme Procurement Delivery Sustainability Design Accountability Framework Information External Scrutiny Accountability Dissemination 10 Although some of these phases are simultaneous or on-going rather than sequential, they are presented for simplicity as separate and distinct phases in figure 2.2. 12 The evolution of a methodology Figure 2.3 organizes these phases into a governance framework. The relationship be- tween indicators and phases of a governance cycle underscores a healthy perspective for at least some stakeholders—that governance is important at all stages in the trans- lation of budget to benefits and the delivery of best value for money in the transport sector. The option to select a key indicator for each phase offered the possibility of generating an indicator “set� in which the whole would be more than the sum of the parts. Although each indicator would be an important measure in its own right, there would also be interdependence between the indicators, suggesting that progress in one might promote progress in others. 2. Requests to see the report findings. Most stakeholders expressed an interest in receiving a copy of the list of governance indicators that might be used to demonstrate progress, to set targets for improvement, and to advocate for change. Perhaps these requests were related to some nervousness about the issues that might surface the report , but also appeared to correspond to a real interest in knowing a little more about what governance looks like, how different sectors “measure up,� and how stakeholders might take ownership of what until now has seemed to be an abstract, academic, or externally gen- erated set of standards. Options for institutional ownership of the indicators identified through this study, the monitoring arrangements that might sup- port them, and the potential for intervention to improve each appear in chapter 4 of this report. Encouragingly, that chapter reports clear opportuni- ties to engage local institutions in governance monitoring in each pilot country—both with and without further donor support. 3. Realization that the perception survey was less important than originally antici- pated. Most meetings held in-country were scheduled for less than one hour. To have framed each conversation around a limited range of questions would have prevented stakeholders from expressing their own views more widely on governance issues and possible responses, which in turn would have limited further refinement of the framework, reduced the opportunity to gather valuable information on stakeholder views about what governance means to them, and reduced the possibility of discovering indicators beyond the initial long list. In any case, because of the limited numbers of stakehold- ers to be interviewed, a focus on the survey would have narrowed the find- ings without offering appropriate robustness to justify the results. Therefore, in most meetings the perception questions were posed only during the last 13 Transport governance indicators in Sub-Saharan Africa five minutes of the meeting. In all, 67 stakeholders completed the survey.. 11 The results of the perception survey appear in chapter 3. The results of the data gathering during country visits and the stakeholder discussions were collated and assessed during the third study phase: analysis and reporting. The findings, conclusions, and options and recommendations for the next steps are pre- sented in chapters 3–5. 11 The survey was not conducted in Mali largely because of the more general “reluctance� to discuss governance shortcomings in that country. The study team thus judged that the survey might be more provocative than helpful if conducted. 14 Selecting and testing transport governance indicators This chapter summarizes the key governance issues raised by stakeholders in the four pilot countries and attempts to distil the indicators that have the greatest potential to “capture� the critical governance component at the heart of the issues debated. Critical governance indicators: Make or break decision making In what follows, 10 indicators are proposed, together with a rationale for each to sup- port the proposition that the indicator is critical to transport sector governance. This rationale is based on stakeholder perspectives that decisions related to these indicators are critically important to the successful functioning of the sector because they make or break performance targets. The location of each indicator within the governance cycle framework (see figure 2.3 in chapter 2) is highlighted. The rationale for the indi- cator is followed by a summary of specific issues related to the indicator as deter- mined by the study. In this chapter, a RAG (red-amber-green) scoring system is proposed as the basis for establishing baselines and targets for each of these governance indicators and for monitoring performance against targets. To demonstrate this process, one of the pilot countries or subsectors or ministries, departments, or agencies (MDAs) is assessed against each indicator to determine a baseline, suggest a target, and propose some options for policy advocacy and practical intervention. In all cases, actions in response to the indicator may be assigned to an organization other than the one proposed as responsible for collecting data to support the indica- tor or for the results that the indicator measures. This is particularly true for transpar- ency indicators that encourage public access to information, which may then lead to advocacy actions by external stakeholders. Thus the supply of information encourages a demand for governance. It is also important to emphasize that, although each indi- cator offers an important insight into governance in its own right—because each sits within the governance cycle framework—the entire set is interlinked. Thus an im- provement in one (e.g., clearer priorities) has the potential to leverage improvements in another (e.g., alignment of budget). 15 Transport governance indicators in Sub-Saharan Africa Indicator 1: Clarity of and distinction between mandates and responsibilities of key ministries, departments, and agencies in the transport sector Rationale: Institutional reform and sep- Institutional and Strategic Framework aration lie at the heart of transport sec- Institutional Mandates and tor performance in all four coun- Strategic Priorities Responsibilities tries. Examples of this reform in- clude separate road fund boards and Sectoral Resource Allocation Framework road agencies created through changes in the law that now governs Budget Discipline their establishment and operation; private sector concessions granted Sectoral Development Framework across all subsectors to privatize or Programme Procurement Delivery Sustainability create public or private partnerships Design where government had held the monopoly; and procurement and Accountability Framework anticorruption agencies established Information to regulate and review the perfor- External Scrutiny Accountability Dissemination mance of the sector on a project or an organizational basis. A clear description of the responsibilities of key MDAs is es- sential for transparency and effective planning and provides clarity for leaders, a focus for delivery, a basis to resist external interference, and a clear locus of accountability. What are the outstanding issues? This indicator was presented by stakeholders as a transparency, regularity, and accountability issue. In Kenya, road agencies were viewed as “too involved in policy.� In Tanzania, senior officials within the Ministry of Transport thought that mandates were “competing rather than complementary.� The Ministry of Transport in Zambia also thought there were “tensions between authori- ties.� There was considerable consensus on the value of separating the regulatory function from service delivery, but concern that this should not mean that institu- tions become “disjointed and dislocated.� There remains concern over duplication of mandate—such as in the road sector in Mali where the lines between responsibility for route types remain blurred and where MDAs are involved in direct procurement in ways that appear to be at odds with their mandated responsibilities. In Kenya, dis- tinctions between the Ministry of Transport and Ministry of Roads are blurred, and reportedly there is interference in the work of the road agencies responsible for the network. In Zambia, the procurement authority is in flux despite a clear shared vision of what needs to change. 16 Selecting and testing transport governance indicators Potential for action? Across the pilot countries there is scope for changes in the law to tighten mandates, perhaps supported by a corporate planning approach to determine what these responsibilities are and where and why the duplications and gaps prevail. For one thing, mandate mapping exercises could be undertaken. There is also scope for clearer indicators to define institutional performance targets in line with respon- sibilities, for sanctions against those who step outside their limits, and for service- level agreements that translate these clearer distinctions into functional relationships. Compliance with mandate could be included in external audit processes. Indicator 2: Coherence of transport sector policy and extent to which its prioriti- zation process is based on objective criteria Rationale: Transport sector policy must be Institutional and Strategic Framework aligned with the macroeconomic context in order to contribute to high-level na- Institutional Mandates and Strategic Priorities tional objectives. Within this framework, a Responsibilities consultative and needs-based approach to Sectoral Resource Allocation Framework prioritization is a fundamental basis for political and public sector accountability. Budget Discipline A cross-sector plan can help retain balance and integration across subsectors. A hier- Sectoral Development Framework archy of plans can then help a public ad- ministration to operationalize the gov- Programme Procurement Delivery Sustainability Design ernment’s agenda by assigning priority to objectives and outcomes at all levels. Pri- Accountability Framework oritization should rely on objective criteria to enable the selection of projects on a Information External Scrutiny Accountability Dissemination sound economic and social basis. A num- ber of investment decision tools (such as HDM in the road sector) and economic analysis approaches would be relevant. What are the outstanding issues? Across subsectors in Kenya the general view was that it is unclear how policy is developed. In Tanzania, policy was seen to be more about personal agendas than public priorities. And in Zambia, politicians referred to specific projects as examples of government policy. In that country, there was an ex- pressed need—by donors and public sector officials—for an integrated master plan and a call for clearer criteria on which to base prioritization decisions and to expose political interference, which reportedly were particularly evident in the road project selection process. There was a widespread view that roads have dominated the agenda at the expense of other subsectors—particularly the rail networks in Tanzania and 17 Transport governance indicators in Sub-Saharan Africa Zambia. Across the pilot countries, road maintenance has received less investment than is warranted by economic return estimates. There is scope for both wider con- sultation and better integration in all four pilot countries. Potential for action? Possibilities include facilitating a cross-government sectoral master plan; initiating wider consultation on priorities, including more decentraliza- tion and local participation; setting explicit standards for consultation and coverage; introducing economic analysis and investment decision tools; instigating wider publi- cation of government policy and priorities, including rationale, through more diverse media. Indicator 3: Budget allocations based on reliable financial forecasts and aligned to priorities based on objective criteria Rationale: Experience suggests that good Institutional and Strategic Framework progress on outcomes at the sectoral Institutional Mandates and level is achieved where links are well Strategic Priorities Responsibilities established among planning, prioritiza- tion, and realistic budgeting. Much de- Sectoral Resource Allocation Framework pends on the abilities of governments to agree with MDAs on ceilings and re- Budget Discipline source envelopes based on reliable fi- nancial forecasts (which include grants Sectoral Development Framework and loans). Budget alignment with na- Programme Procurement Delivery Sustainability tional priorities (objectively determined) Design should be a measure of both government effectiveness and aid effectiveness be- Accountability Framework cause it provides the basis for external Information scrutiny of public sector spending deci- External Scrutiny Accountability Dissemination sions. Uplifts in budget should be in- cluded in the alignment assessment, to prevent delayed allocations and late approval of “special projects� from becoming a vehicle for the diversion of funds away from national priorities. What are the outstanding issues? In the pilot countries, expenditure frameworks aligned to policy and sectoral plans are either not fully developed or not binding. In Mali, for example, the medium-term expenditure framework (MTEF) has limited influence on budget allocations. The Ministry of Transport in Tanzania is constrained by a planning cycle confined to one year. In Kenya, overcommitment of road funds is a major issue, severely curtailing the ability to plan for priorities based on potential 18 Selecting and testing transport governance indicators budget allocations. Indeed, in Kenya there are serious over allocation concerns—one stakeholder believed that “a reduction in pending bills would be a good way to show that government is dismantling the poor governance arrangements that allow for over commitment of the budget.� Potential for action? Potential actions include formulating medium-term sector strat- egies or expenditure frameworks; developing guidelines; pilot testing fiscal projection models and then introducing or improving them to close the gap between projected and actual revenue; and exploring new institutional arrangements such as parliamen- tary committees to scrutinize and improve budget management. Indicator 4: Quality and use of key performance and value for money indicators Rationale: An effective and accounta- Institutional and Strategic Framework ble public sector requires clarity about its objectives in order to better align Institutional Mandates and Strategic Priorities and justify the activities of MDAs and Responsibilities to improve the delivery of services Sectoral Resource Allocation Framework and programs. Key performance indi- cators (KPIs) need to be expressed Budget Discipline simply, clearly, and precisely in order to contribute to transparency, re- Sectoral Development Framework source allocation, and accountability. Clearly, KPIs are not an end in them- Programme Procurement Delivery Sustainability Design selves, and making the right choices about what is “key� is critical to their Accountability Framework real value. MDAs need to be able to identify indicators that measure both Information External Scrutiny Accountability performance and value for money— Dissemination the latter offering opportunities to introduce indicators that may relate specifically to efforts designed to reduce corrup- tion and waste. What are the outstanding issues? Across the pilot countries, the subsectors, and the MDAs, there is significant variation in the number and choice of KPIs, the clarity with which they are expressed, whether they are measured and reported publicly, and the extent to which they are felt to be binding. Some progress has been made in de- termining what the highest priorities are in terms of performance measurement, but monitoring and evaluation (M&E) systems remain weak and poorly integrated and the data unreliable. There is little commitment by leaders to determining levels of 19 Transport governance indicators in Sub-Saharan Africa responsibility or to facilitating coordination between institutions or a sense of owner- ship of the results. Potential for action? There is considerable scope for improving the selection of KPIs; introducing key value for money measures of economy, efficiency, and effectiveness; building capacity in monitoring and evaluation; and enhancing the standard of re- porting to governance bodies. Indicator 5: Comprehensive and timely public disclosure of transport sector pro- curement plans Institutional and Strategic Framework Rationale: A number of initiatives in the transport sector have been based Institutional Mandates and Responsibilities Strategic Priorities on the premise that greater transpar- ency—through greater disclosure of Sectoral Resource Allocation Framework material procurement and project in- formation—will yield benefits to gov- Budget Discipline ernment, industry, and civil society and to ordinary citizens. 12 Procure- Sectoral Development Framework ment processes are complex, and it is difficult to locate one process or link Programme Procurement Delivery Sustainability Design that—if it holds—reduces corruption significantly. After all, there are a great Accountability Framework many examples of how “binding� rules are circumvented by those determined Information External Scrutiny Accountability to secure inappropriate personal gain. Dissemination Although there are no quick fixes, transparency plays a key role in reducing corruption by increasing the risk of corrupt activity being detected. However, transparency only translates into such benefits where the capacity and will to scrutinize and expose wrongdoing have also been de- veloped. Caution is also necessary to ensure that the right information is released in order to avoid facilitating cartelization and price fixing. What are the outstanding issues? Despite significant investment in procurement in- stitutions and regulations in all four countries, there were many anecdotal examples of the circumvention of rules. Corrupt or collusive practices were initiated by both 12 See CoST, the Construction Sector Transparency Initiative at www.constructiontransparency.org. 20 Selecting and testing transport governance indicators the client and the contractor. Indeed, investigators heard numerous stories of ways in which procurement regulations were being flouted and side-stepped, 13 to the extent that the existence of regulations was deemed to be an inadequate indicator of good governance in procurement. However, there was general agreement with one public sector stakeholder in Tanzania, who believed that “procedures have reduced the amount that can be creamed off.� Nevertheless, as another Tanzanian public sector official stated, “Something is still radically wrong somewhere.� This view was en- dorsed by private sector engineers, who spoke of “a conspiracy of silence� to explain the limited number of objections made to contact awards. Potential for action? Possible actions include establishing a memorandum of under- standing with MDAs on the quality, quantity, and timing of the release of infor- mation; supporting the use of appropriate means such as websites, journals, national or regional newspapers, or public posting; building capacity for external scrutiny of procurement plans and awards; including press bodies and not just nongovernmental organizations (NGOs); ensuring timeliness of resolution of complaints to allow con- tract awards to be reversed if necessary. The Public Finance Management Perfor- mance Measurement Framework (initiated by the Public Expenditure and Financial Accountability partnership program, or PEFA) has already developed an indicator (PI-19) that could be used across the region and provide an overlap with the govern- ance framework proposed here. Indicator 6: Comprehensive time and cost reports on progress of work for major (top 10) transport sector contracts, disclosed to the public in a timely and accessible manner Rationale: Often neither MDAs nor contractors are under pressure—nor do they have a vested incentive—to prepare periodic progress reports and to establish a com- prehensive database that would help track progress and detect early signs of project delays and incompleteness. Without such information, there is no capacity to identify and analyze the root cause of incompleteness or overruns. A range of “evidence� of poor delivery phase governance was cited by stakeholders: pending bills, cost and time overruns, and contract variations. Both cost and time overruns are measurable, 13 Examples include previewing of tender documents by the preferred contractor, thereby allowing the preparation of a proposal and the building of a team before the requirement is announced; interference with bid documents; colluding on prices to squeeze out legitimate bidders on the grounds of price differentials; fixing the short list to create an “obvious winner�; and signing the contract before announcement of the award to reduce objections. 21 Transport governance indicators in Sub-Saharan Africa although time overruns were judged to be more accessible—and less politically sensi- tive—as a measure, as well as possibly slightly more time-sensitive in terms of data collection (i.e., discrepancies emerge sooner). What are the outstanding Institutional and Strategic Framework issues? Whereas a shrinking budget is sometimes cited as Institutional Mandates and Strategic Priorities Responsibilities the main cause of project delays, it rarely stands up to Sectoral Resource Allocation Framework the test when comparing planned and actual spend- Budget Discipline ing. Rather, the primary problem is inefficient man- Sectoral Development Framework agement and the political economy of rent-seeking. Programme Procurement Delivery Sustainability Design Delays in project imple- mentation and the at- Accountability Framework tendant costs overruns are a concern across the pilot Information External Scrutiny Accountability Dissemination countries and subsectors because they are attributed in large part to avoidable factors such as poor project de- sign, signing of contracts for which there is no immediate budget, delayed cash releas- es and payments, contractor competence, and inefficiencies. Time overruns on pro- jects are common, with some projects experiencing delays of more than a year. Many projects are left on hold without the appropriate acknowledgment that delays cost money in terms of penalty payments. Potential for action? Possible actions are analyzing the bid price against the engineer’s estimate; putting risk mitigation in place; analyzing patterns of poor performance against time and expenditure plans (easier to collect than indicators of quality and easier to standardize than unit cost information); undertaking evaluations to deter- mine lessons learned; providing enhanced information for audit; and establishing an improved basis for accountability (perhaps links to KPIs). 22 Selecting and testing transport governance indicators Indicator 7: Transport sector agency has established benchmarks for routine and periodic maintenance of assets and allocates the budget accordingly. Rationale: Good governance is criti- Institutional and Strategic Framework cal for safeguarding quality transport infrastructure through good budget Institutional Mandates and Strategic Priorities Responsibilities finance and a professionally compe- tent implementation agency that Sectoral Resource Allocation Framework recognizes the importance of maintenance. According to Gwilliam Budget Discipline et al. (2008), countries that devote a large share of their road funds to Sectoral Development Framework maintenance exhibit significantly better-quality indicators for their Programme Procurement Delivery Sustainability Design main road network (though not for rural roads). It follows that lower Accountability Framework maintenance spending is a problem, because it generates a rehabilitation Information External Scrutiny Accountability Dissemination backlog and the associated high capi- tal expenditure that further diminishes the amounts available for maintenance. In the road sector, countries with road funds and high fuel levies are substantially more suc- cessful at raising finances that translate into high road maintenance What are the outstanding issues? Across the pilot countries but particularly in Mali, the rehabilitation backlog is attributed to poor maintenance spending. In Kenya and Tanzania, the high level of pending bills poses a threat to future maintenance. In Zambia, upgrading projects appears to be crowding out both the rehabilitation and maintenance expenditures. This is not a new phenomenon. Political will has long diverted resources to building new roads, or upgrading (e.g., paving) existing roads, often where there is little economic justification. Funds are diverted from needed routine and periodic maintenance, thereby increasing the downstream maintenance obligation. Potential for action? A number of initiatives may be necessary before a consensus can be reached on benchmarks for maintenance standards and expenditures on an ap- propriate basis. This might include support for establishing or successfully operating a road maintenance fund; establishing locally sustainable road management systems; gaining political acceptance of the maintenance problem and the benefit that mainte- nance delivers; bringing stakeholders together for briefings; publishing a position paper; encouraging public comment; engaging civil society monitors; introducing 23 Transport governance indicators in Sub-Saharan Africa tools to enable the planning of maintenance based on reasonable estimates—for ex- ample, HDM-4 14 or a similar tool (although it is important that the results of tools such as HDM are then used in budgeting processes); commissioning independent surveys (but they can be expensive); building capacity to enable better analysis of needs against budget forecasts; setting up benchmarks for periodic and routine maintenance by establishing timing (e.g., every seven years) against road type (e.g., gravel roads) and prioritizing if necessary against volume of road use. Indicator 8: Transparency and timeliness of annual budget and expenditure dis- closures Rationale: Full disclosure of financial Institutional and Strategic Framework information to the public is central to Institutional Mandates and institutionalizing good governance, Strategic Priorities Responsibilities transparency, and accountability. Secrecy and lack of accountability breed corrup- Sectoral Resource Allocation Framework tion, whereas greater transparency can help to ensure the appropriate checks and Budget Discipline balances. There is mounting evidence that if civil society organizations are en- Sectoral Development Framework gaged on the ground as actors in the pro- Programme cess of budget consultation and expendi- Procurement Delivery Sustainability Design ture monitoring, it is likely to translate into more efficient use of resources, even- Accountability Framework tually leading to better service delivery Information and development outcomes. External Scrutiny Accountability Dissemination What are the outstanding issues? Some progress has been made in Zambia, but civil society representatives suggest that there is scope for wider consultation with and the enhanced credibility of civil society as a key player. Nevertheless, there are good examples of positive responses to civil society scrutiny and suggestion. This view was shared by Zambia’s Ministry of Transport: “The comprehensive disclosure of information is the primary governance mechanism which facilitates best outcomes across the transport sector.� This position was sup- ported in Kenya, where the view of one official was that “a culture of confidentiality leads to talk of corruption.� Progress in Tanzania rested largely on the disclosure of 14 The Highway Design and Maintenance Standards Model (HDM-4) was developed by the World Bank. 24 Selecting and testing transport governance indicators procurement rather than financial plans, but on this it was felt that information was presented in a way that obscured meaning. In Kenya, where there has been some at- tempt to release budget information to the public, the view was expressed that it has not been done in a way likely to be understood by the layman, in part because of the terminology used but also because “information needs to be linked to more pressing issues to excite people’s interest.� Several stakeholders recommended simplifying the language, disaggregating data, and using the appropriate delivery channels. The key words used to express views on this were accuracy, timing, and detail. Potential for action? Possible actions include building capacity for monitoring and evaluating units and government officials for their timely and accessible presentation of information; using more diverse media channels; building the capacity of civil so- ciety to analyze and interpret data; and supporting civil society monitors. Indicator 9: Rules applied to the membership and appointment process for key transport sector governance boards Rationale: Governance boards play a Institutional and Strategic Framework vital role in ensuring that services pro- vided are prioritized and offer the best Institutional Mandates and Strategic Priorities Responsibilities value for the money. Boards need proper procedures and policies to operate effec- Sectoral Resource Allocation Framework tively. Members of boards should bring skills and experience as well as vision to Budget Discipline enhance decision making. Effective boards understand their role, ensure Sectoral Development Framework delivery of organizational purpose, are effective as individuals and a team, exer- Programme Procurement Delivery Sustainability Design cise control, behave with integrity, and are open and accountable. Accountability Framework What are the outstanding issues? The Information External Scrutiny Accountability Dissemination political appointment of board members was a significant issue across all the pilot countries. Stakeholders in all four countries expressed the need for independent boards and shared a strong view that board places should be advertised and individuals vetted to ensure that they have the skills and competencies required to govern. There was considerable frustration that patronage dominates the appointment process and shuts out individuals who could make a dif- ference. As one Zambian official put it, “We don’t have a shortage of people with in- tegrity.� However, problems remain. For example, despite the design of many road 25 Transport governance indicators in Sub-Saharan Africa fund organizations to have, in principle, strong road user representation on their boards, a common feature of many is that the actual board members are �nominated� or �appointed,� and, although they may represent �road users� as defined in the ap- plicable road fund legislation, there is a perception that they may not necessarily be the most independent, or challenging, voice. Oversight boards exist in other parts of the sector such as in road traffic and licensing authorities, maritime transport, rail- ways, and so forth. To better represent the interests of the end users, these boards should demonstrate independence and their appropriateness for the role. Potential for action? Possible actions are revised guidance, support for lobbying and advocacy, and audit reports. Indicator 10: Percentage of recommendations from independent technical and financial auditor reports implemented within one year Rationale: Audits play a fundamental Institutional and Strategic Framework role in ensuring that organizations Institutional Mandates and Strategic Priorities function according to good governance Responsibilities and accounting and auditing stand- Sectoral Resource Allocation Framework ards, as well as adopt appropriate risk management arrangements. Critically, Budget Discipline the judgments of auditors on how well an organization has performed against Sectoral Development Framework these standards are independent. How- ever, if organizations are not required Programme Procurement Delivery Sustainability to respond to the findings of auditors Design or there is no monitoring of this re- sponse, then the value of the entire Accountability Framework process is undermined Information External Scrutiny Accountability Dissemination What are the outstanding issues? In Zambia, an independent audit of the Road Development Agency (RDA) brought such significant discrepancies to light that it stalled donor support, and significant delays are continuing in that support pending implementation of the auditors’ recommen- dations. In Mali, there are some issues related to some duplication of function with the Office of the Auditor General. Across the pilot countries, a review to determine the periodicity, findings, and responses to audits of MDAs would be of value. Potential for action? Donor support of implementation of the findings and recom- mendations of audit reports can be very effective (including withholding of funds if 26 Selecting and testing transport governance indicators necessary). Other possible actions are providing capacity support for audit officers (including ensuring no duplication); incorporating tested and refined governance framework indicators in audit reports; and using governance indicators as precondi- tions for donor funding. For ease of reference, the 10 indicators identified by the study are listed in table 3.1. Table 3.1 A Framework of Proposed Indicators Framework location Indicator 1. Institutional mandates Clarity of and distinction between mandates and responsibilities of key ministries, and responsibilities departments, and agencies in the transport sector 2. Strategic priorities Coherence of transport sector policy and extent to which its prioritization process is based on objective criteria 3. Budget discipline Budget allocations based on reliable financial forecasts and aligned to priorities based on objective criteria 4. Program design Quality and use of key performance and value for money indicators 5. Procurement Comprehensive and timely public disclosure of transport sector procurement plans (including bidding opportunities, contract awards, and data on resolution of pro- curement complaints) 6. Delivery Comprehensive time and cost reports on progress of work for major (top 10) transport sector projects, disclosed to the public in a timely and accessible manner 7. Sustainability Transport sector agency has established benchmarks for routine and periodic maintenance of assets and allocates the budget accordingly 8. Information dissemination Transparency and timeliness of annual budget and expenditure disclosures 9. External scrutiny Rules applied to the membership and appointment process for key transport sector governance boards 10. Accountability Percentage of recommendations from independent technical and financial auditor reports implemented within one year Quality assessment of the selected indicators using revised criteria So that they were consistent with the study methodology, the 10 indicators advocated by stakeholders were rated using the quality assessment criteria to determine whether they were actionable, credible, nationally ownable, relevant, sensitive, understandable, available, and reliable (table 3.2). 27 Transport governance indicators in Sub-Saharan Africa Table 3.2 Scores of the Proposed Indicators Revised criteria score (0–2) Total Governance cycle stage/indicator A B C D E F G H Institutional mandates Clarity of and distinction between mandates and responsibilities of 4 4 4 4 2 4 4 4 30 key ministries, departments, and agencies in the transport sector Strategic priorities Coherence of transport sector policy and extent to which it directs 4 4 4 4 2 3 4 4 29 the prioritization process Budget discipline Budget allocations based on reliable financial forecasts and aligned 4 4 4 4 3 3 4 4 30 to priorities based on objective criteria Program design Quality and use of key performance and value for money indicators 4 4 3 3 3 3 4 4 28 Procurement Comprehensive and timely public disclosure of transport sector 4 3 4 3 3 4 4 3 28 procurement plans (including bidding opportunities, contract awards, and data on resolution of procurement complaints) Delivery Comprehensive time and cost reports on progress of work for ma- 4 3 3 4 3 3 4 3 27 jor (top 10) transport sector projects, disclosed to the public in a timely and accessible manner Sustainability Transport sector agency has established benchmarks for routine 4 3 3 4 4 4 3 2 27 and periodic maintenance of assets and allocates the budget ac- cordingly. Information dissemination Transparency and timeliness of annual budget and expenditures 4 4 4 4 3 4 3 3 29 disclosures External scrutiny Rules applied to the membership and appointment process for key 4 4 4 4 2 3 4 4 29 transport sector governance boards Accountability Percentage of recommendations from independent technical and 3 4 4 3 4 3 3 3 27 financial auditor reports implemented within one year Note: A = actionable; B= credible; C = nationally “ownable�; D = relevant; E = sensitive; F = understandable; G = available; H = reliable. Assessors rated each indicator against the criteria as 0, 1, or 2, where 0 = poor; 1 = partial; 2 = good. 28 Selecting and testing transport governance indicators Scores were high across the indicator set; the strongest indicators were those related to institutional mandate and budget discipline, as well as prioritization, board mem- bership, and disclosure of budgets. Against the basket criteria test (table 3.3), the indicator set performed as follows:  There are both quantitative and qualitative measures; the scoring mecha- nism outlined in chapter 4 allowed each indicator to be scored in a quantifi- able way.  The set is fact-based (see what follows on the importance of perceptions).  There is a good balance between indicators that focus on policy issues and those related to systems and outcomes.  All of the indicators have whole sector and all subsector relevance.  There is significant scope for subnational application.  The indicators offer only limited potential for disaggregation by social group (unless these are geographically located) because of the high-level nature of the indicators and the generic relevance of the benefits that can be accrued from transport sector development. The table includes (Y) to indicate that there is a possibility for some disaggregation, depending on policy priorities. Table 3.3 Scores against the Basket Criteria Basket criteria Indicator 1 2 3 4 5 6 7 8 9 10 Balance between quantita- Qual. Qual. Quant Qual. Qual. Qual. Quant Qual. Quant Qual. tive & qualitative . . . Fact (F)/perception (P) F F F F F F F F F F/P balance Covers policy (P), systems P P P S S O O P S S (S), and outcomes (O) Coverage across subsectors Y Y Y Y Y Y N Y Y Y Equity for stakeholders N (Y) (Y) (Y) N (Y) (Y) N N (Y) National/subnational appli- Y Y Y Y Y Y Y N N Y cation Note: Y = yes; N = no. 29 Transport governance indicators in Sub-Saharan Africa Subsector relevance and indicator potential An unplanned but notable potential of the indicator set was highlighted by the basket criteria test and is elaborated in table 3.4. Many of the indicators can be used on a cross-transport sector basis for all of the subsectors (to allow a cross subsector com- parison), for individual subsectors, and for MDAs within the sectors. This flexibility in application would facilitate comparative analysis, assuming the methodology used for assessment could be consistently applied. 3.4 records the potential application of each indicator. Table 3.4 Potential Application of Proposed Indicators All subsectors Whole sector Waterways Indicator All MDAs Road Port Rail Air 1. Clarity of and distinction between mandates and responsi- bilities of key ministries, departments, and agencies in the         transport sector 2. Coherence of transport sector policy and extent to which its prioritization process is based on objective criteria         3. Budget allocations based on reliable financial forecasts and         aligned to priorities based on objective criteria. 4. Quality and use of key performance and value for money         Indicators 5. Comprehensive and timely public disclosure of transport sector procurement plans (including bidding opportuni- ties, contract awards, and data on resolution of procure-        ment complaints) 6. Comprehensive time and cost reports on progress of work for major (top 10) transport sector projects, disclosed to        the public in a timely and accessible manner 7. Transport sector agency has established benchmarks for routine and periodic maintenance of assets and allocates       the budget accordingly 8. Transparency and timeliness of annual budget and ex-         penditure disclosures 9. Rules applied to the membership and appointment pro-         cess for key transport sector governance boards 10. Percentage of recommendations from independent tech- nical and financial auditor reports implemented within one         year 30 Selecting and testing transport governance indicators Composite indicators: Complexity made simple? It was not the intention of the study to add another 10 indicators to the already over- whelming number of indicators in circulation; rather, once the most significant gov- ernance issues had been identified, the intention was to select indicators already in use that could be used to monitor and manage these issues. The 10 proposed indica- tors in large part meet this ambition. They are understood by stakeholders in-country and across countries and are already measured in some places and by some institu- tions—albeit not comprehensively, very well, nor necessarily with consequences when they reveal poor performance. These challenges are considered in chapter 4. However, in terms of indicators already in use, the study also considered those that have been developed by external players. These often composite indicators or indexes offer a number of arguable advantages, including being independently verified, and bring together a number of sometimes complex measures, transforming these, through a robust methodology, into a single figure. A question for the study was whether these composites would provide a better single measure of the priority gov- ernance concerns raised by stakeholders. Would they be more reliable and robust than any “home-grown� variety? A number of composites were examined in the light of these considerations to deter- mine whether these indicators would stand as better proxies for the governance issue and the associated indicators identified by the study—or perhaps offer an appropriate data source for institutions keen to monitor performance in this regard. Some of the possibilities are listed in table 3.5. Further consultation would be necessary to assess buy-in from stakeholders to these indicators or others. However, overall and somewhat perversely, the strengths of most composite indicators perhaps also best describe their weaknesses, because the meth- odology that makes them robust is poorly understood by stakeholders and so reduces transparency and ownership (something critical for advocacy indicators). The “smoothing effect� of combining indicators is perceived to obscure realities and com- promises credibility. 31 Transport governance indicators in Sub-Saharan Africa Table 3.5 Proposed Indicators and Associated Composites Indicator Composite options Comments Coherence of transport sector Mo Ibrahim Index: public man- Africa-owned; aggregates a num- policy and extent to which its agement, budget management ber of indicators, but detail is then prioritization process is based on lost for the subsequent policy objective criteria action required in each area Bertelsmann Foundation Sustain- Explicitly reviews whether gov- able Governance Indicators: man- ernment translates its objectives agement index: policy implemen- and priorities into implemented tationa policy; only undertaken in Organi- sation for Economic Co-operation and Development–Development Assistance Committee (OECD- DAC) countries Comprehensive and timely public Public Expenditure and Financial A dimension of one PEFA indicator disclosure of transport sector pro- Accountability (PEFA) has been selected for the study (PI- curement plans 19 (iii)) Transparency and timeliness of Mo Ibrahim Index: public man- Data collected in-country by civil annual budget and expenditure agement, budget management society and research institutes disclosures International Budget Project: Open Budget Index (OBI)b Unknown how long the OBI pro- ject will be funded and available. Rules applied to the membership Worldwide Governance Indicators: May not be focused enough be- and appointment process for key control of corruption cause it looks at wider elite cap- transport sector governance ture boards Percentage of recommendations Mo Ibrahim Index: public man- As above—at aggregate level from independent technical and agement, budget management includes emphasis on timely and financial auditor reports imple- focused audits and responses mented within one year a. See http://www.sgi-network.org/ for further details. b. Developed from a comprehensive study through the Open Budget Survey, the index evaluates whether governments give the public access to budget information and opportunities to participate in the budget process at the national level. The first Open Budget Survey index was developed in 2006 (now biannual). It works with civil society and research institutes in 85 countries to collect data for the survey. See http://internationalbudget.org/ for further details. The importance of perception As part of the process of identifying the main governance issues in the subsectors vis- ited, stakeholders were asked to complete a very brief questionnaire. it provided a snapshot of perceptions and allowed individuals to express their own views—often based on personal experience—of governance and reform. The questions were asked 32 Selecting and testing transport governance indicators Table 3.6 Survey Data Collated INTERNAL STAKEHOLDERS EXTERNAL STAKEHOLDERS To some extent To some extent Survey question: “To what extent do you Not Very much Not very much I don't dnow I don't know Very much Very much I disagree I disagree agree with the following statements?" The government gets its priorities right with 11 58 28 3 0 3 52 16 29 0 regard to transport sector spending. The public is adequately informed on how government spends public money on 3 33 39 25 0 3 16 39 42 0 transport provision. There are reasonable opportunities for new market entrants to win transport conces- 53 25 3 11 8 32 26 23 10 10 sions. Major transport contracts are awarded on a 25 42 11 17 6 6 26 19 32 16 best value for money basis. It is now rare for a bribe to be paid in order to access or speed up a transport sector service 6 28 25 36 6 6 13 26 52 3 (e.g., driving license or customs clearance). I believe corruption allegations are generally properly investigated and can be successfully 14 36 22 28 0 0 23 29 48 0 prosecuted. If public sector officials in a "lead agency" override regulations and procedures, they 33 39 8 17 3 13 23 42 23 0 will be punished or sanctioned. Complaints from the public are generally 22 44 14 17 3 0 23 39 32 6 handled fairly. at the end of the interviews, in part to elicit some more insightful observations as people expanded on their answers. Of the 67 persons interviewed in the four coun- tries, an average of 22 (per country) responded to the questionnaire in the three countries in which it was used. 15 The limited numbers of interviewees involved inevi- tably meant that the results would require far more robust testing to be statistically 15 The questionnaire was not used in Mali (although it had been translated) in part because of the difficulties of managing the delicately “‘nuanced� translation—which was extremely im- portant in the other countries as well. Primarily, though, it stemmed from a reluctance to dis- cuss governance issues as openly. The response that there were few governance issues to ad- dress was heard on numerous occasions. Because the questions were to act more as a catalyst for conversation than as a barrier to it, we did not press the matter. 33 Transport governance indicators in Sub-Saharan Africa valid. The survey does nevertheless add an interesting dimension to the study, partic- ularly in view of the dominance of fact-based indicators in the proposed indicator list. Although the sample size is insufficient to draw substantiated conclusions, some in- teresting patterns did emerge. Data supporting the findings appears in table 3.6, with headlines in table 3.7. Table 3.7 Headline Survey Findings Survey question: “To what extent do you agree with the fol- Results by stakeholder type lowing statements?" The government gets its priorities right with regard to Although the majority of both internal and external transport sector spending. stakeholders agreed �to some extent,� a higher per- centage of external stakeholders �disagreed.� The public is adequately informed on how government Among both internal and external stakeholders there spends public money on transport provision. was clearly a feeling that the public was inadequately informed; disagreement with statement expressed most strongly among external stakeholders. There are reasonable opportunities for new market en- Clear majorities of both internal and external stake- trants to win transport concessions. holders agreed �very much� or �to some extent� with statement. Major transport contracts are awarded on a best value for Greater sense of agreement among internal stake- money basis. holders; lesser sense among external stakeholders. It is now rare for a bribe to be paid in order to access or General view of both internal and external stakehold- speed up a transport sector service (e.g., driving license or ers was that this was not the case. Opinion most customs clearance). strongly felt among internal stakeholders. I believe corruption allegations are generally properly A range of views on this statement and no clear sense investigated and can be successfully prosecuted. of agreement or disagreement among internal stake- holders, although a stronger sense of �disagreement� was felt among external stakeholders. If public sector officials in a "lead agency" override regula- Greater sense of agreement on �very much� or �to tions and procedures they will be punished or sanctioned. some extent� among internal stakeholders; greater sense of �not very much� or �disagree� among external stakeholders—but no great polarization. Complaints from the public are generally handled fairly. More positive perception of how complaints are dealt with among internal stakeholders than external stake- holders. Note: The survey was anonymous, and respondents were classified in terms of whether they were public (inter- nal) or private or civil society organization (external) stakeholders. 34 A workable approach to measuring and managing transport sector governance The purpose of the proposed indicators is to allow policy decision makers to improve governance performance within the transport sector and to provide policy advocates with access to robust information as a platform for appropriate action. Those meas- uring governance often perceive the indicators being examined as broad in nature, with perhaps abstract institutional setups, structures, and opportunities for develop- ment. To help ground governance as a manageable concept, there needs to be a clear and objective basis for the scoring of indicators—and one that can better support decision making at both the policy and planning levels. A monitoring and evaluation system: Preliminary design and sample scoring This chapter begins by presenting a possible scoring framework that would allow in- dicators to be base-lined and then targets to be set as the basis for action. The scoring system is founded on a simple four-level RAG (red–red/amber–amber/green–green) rating approach, allowing assessment of performance along a continuum. The RAG ratings used are presented in table 4.1 and are followed by 10 worked examples to exemplify the indicators. The chapter then examines the institutional arrangements around these indicators and provides a preliminary “how-to� guide to support the monitoring and evaluation of the data to be collected. This examination is followed by consideration of the kinds of practical action that might be relevant in countries wishing to move their perfor- mance along the indicator “score line.� 35 Transport governance indicators in Sub-Saharan Africa Table 4.1 List of Indicators and Respective RAG Ratings Indicator Red Red/amber 1. Clarity of and distinction between Mandates and responsibilities have not Mandates and responsibilities have been mandates and responsibilities of key been defined or are unclear. defined, but there is significant overlap. ministries, departments, and agencies in the transport sector 2. Coherence of transport sector policy No transport sector-wide policy. Transport sector policy exists and identi- and extent to which its prioritization OR fies issues across some subsectors, but process is based on objective criteria Policy does not identify needs of subsec- not linked to macroeconomic context. tors. 3. Budget allocations based on reliable More than 50% difference between Less than 50% difference between sec- financial forecasts and aligned to pri- sector financial ceilings and actual tor financial ceilings and actual budget orities based on objective criteria budget allocation (by amount). allocation. OR OR Budget allocation by priority (based on Budget allocation by priority (based on top five projects by value) is so different top five projects by value) is significantly from sector strategy that a comparison is different (more than 50%) from sector not possible. strategies. 4. Quality and use of key performance No key performance indicators (KPIs) KPIs developed with targets but not and value for money (VFM) indicators developed for the sector, or some KPIs monitored or reported. developed but with no targets. OR Do not include VFM indicators. 5. Comprehensive and timely public Government lacks a system to generate At least two of the key procurement disclosure of transport sector pro- substantial and reliable coverage of key information elements are complete and curement plans (including bidding procurement information. reliable for government units represent- opportunities, contract awards, and OR ing 50% of procurement operations (by data on resolution of procurement Government does not systematically value). complaints) make key procurement information AND available to the public. Elements are made available to the public through the appropriate means. 36 A workable approach to measuring and managing transport sector governance Amber/green Green Mandates and responsibilities have been defined, but Mandates and responsibilities are clear, and there is no over- there is some lack of clarity or overlap. lap. Transport sector policy exists and identifies issues across Transport sector policy exists and is explicitly linked to mac- all subsectors, but does not prioritize them in an objective roeconomic policy with issues identified and prioritized in an way. objective way. Less than 20% difference between sector financial ceilings Sector financial ceilings and budget allocations are consistent and actual budget allocation. (less than 10% difference). AND AND Budget allocation by priority (based on top five projects There is little discrepancy (less than 20%) between sector by value) is not significantly different (less than 50%) from strategy and budget allocation in terms of priorities (based on sector strategies. top five projects by value). KPIs (including VFM indicators) developed, but under 50% KPIs (including VFM indicators) developed, and over 50% monitored and reported. monitored and reported. At least three of the key procurement information ele- All the key procurement information elements are complete ments are complete and reliable for government units and reliable for government units (90% of procurement oper- representing 75% of procurement operations (by value). ations (by value) . AND AND Elements are made available to the public in a timely Elements are made available to the public in a timely manner manner through the appropriate means. and through the appropriate means. 37 Transport governance indicators in sub-Saharan Africa Indicator Red Red/amber 6. Comprehensive time & cost reports Information does not provide sufficient Information does not provide sufficient on progress of work for major (top 10) detail for analysis, or detail for analysis, or information on projects, disclosed to the public in a information on actual budget variations variations or actual expenditure incom- timely and accessible manner or expenditure not disclosed. plete or in non-compatible format. 7. Transport sector agency has estab- Benchmarks not established. Benchmarks established, but routine lished benchmarks for routine and pe- OR and periodic maintenance not priori- riodic maintenance of assets and allo- No link between benchmarks and alloca- tized (less than 50% of needs met using cates the budget accordingly tion decisions. less than 50% of total subsector budget). 8. Transparency and timeliness of annu- Budget information does not provide Budget information does not provide al budget and expenditure disclosure sufficient detail for analysis, or infor- sufficient detail for analysis; information mation on actual expenditure not dis- on actual expenditure incomplete or in closed. non- compatible format. 9. Rules applied to the membership and Minority private sector representation Majority private sector representation appointment process for key on oversight boards and appointed on oversight boards but appointed transport sector governance boards directly. directly (e.g., by president or minister). 10. Percentage of recommendations from Recommendations not published, nor Technical and financial audits published independent technical and financial any details of any follow-up actions. but no details of any follow-up actions, auditor reports implemented within OR one year. follow-up actions published but less than 50% implemented. 38 A workable approach to measuring and managing transport sector governance Amber/green Green Information provided; information on variations or Information provided in appropriate level of detail; expendi- actual expenditure incomplete. ture information provided in comparable level of detail. Benchmarks established and significant evidence of Benchmarks established and more than 80% of maintenance prioritization (more than 50% of routine and periodic needs met or more than 80% of budget allocated to mainte- maintenance needs met or more than 50%t of total nance. subsector budget allocated to maintenance). Budget information provided; information on actual Budget information provided in appropriate level of detail; expenditure incomplete. expenditure information provided in comparable level of detail. Majority private sector representation on oversight Majority private sector representation on oversight boards; boards; appointed through competitive process— clear, transparent, competitive meritocratic appointment but process lacking transparency. process for private sector members. Technical and financial audits published, but incom- Technical and financial audits published; full information on plete information on follow-up actions published or over half of the follow-up actions published; over 75% of only between 50% and 75%nt of recommended recommended actions implemented. actions implemented. 39 Transport governance indicators in Sub-Saharan Africa A snapshot of 10 worked examples taken from the pilot countries follow in order to demonstrate how the indicator scoring system might be applied in practice. In each example, the indicator is applied to a particular country and sector or MDA, and then a RAG rating is provided for the current situation and as a proposed realizable target. Evidence supporting the current RAG rating is provided beneath each example, to- gether with potential interventions to support the target. 16 1. Transport Governance Framework: Institutional Mandates and Responsibilities (Kenya ) Indicator 1 Country Sector/MDA Current rating Target rating Clarity of and distinction between mandates and Kenya Whole sector Amber / green Green responsibilities of key ministries, departments, and agencies in the transport sector Evidence supporting the current rating Although various roads and other agencies (e.g., Kenyan Civil Aviation Authority) within Kenya 17 have been given clear mandates, there is some confusion in some sub- sectors, particularly in the ports and inland waterway arena, with overlapping respon- sibilities between the Kenya Maritime Authority, Kenya Ports Authority, Kenya Ferry Services Limited, and Kenya Railways Corporation. Potential policy actions in support of the target rating Kenya’s “Integrated Transport Policy: Moving a Working Nation� issued in 2009 provides a comprehensive overview of the institutional situation, recognizing areas of concern and potential action (Ministry of Transport, Republic of Kenya 2009). The document makes extensive recommendations for the reform and restructuring of the wider transport sectors. The steps to be taken have been set out in Kenya’s 2009 na- tional transport policy. They include: establishing a directorate of transport, consoli- dating transport functions under one ministry, separating policy, and strengthening regulatory and service provision functions. Clarity is required about which unit is responsible for taking forward the appropriate initiatives. 16 Appendix C contains these examples in more detail, with suggestions for improving the in- fluence of this indicator on transport governance in the respective country (if appropriate). 17 For example, the Kenya Roads Board, Kenya National Highways Authority, Kenya Urban Roads Authority, and Kenya Rural Roads Authority. 40 A workable approach to measuring and managing transport sector governance 2. Transport governance framework: Strategic priorities (Zambia) Indicator 2 Country Sector/ MDA Current rating Target rating Coherence of transport policy and extent to which its Zambia Whole sector Red / amber Amber/green prioritization process is based on objective criteria Evidence supporting the current rating Zambia’s national transport policy was last issued in 2002 (Ministry of Communica- tions and Transport, Republic of Zambia 2002). Many changes were made at that time, including reform of the National Road Fund (from the previous National Road Board) and the Road Development Authority. However, although the transport sec- tor policy exists and identifies issues across some subsectors, it is not linked to the macroeconomic context. Potential policy actions in support of the target rating The ministry responsible for transport—until September 2011 the Ministry of Com- munications and Transport—was very recently merged with the Ministry of Works and Supply. This merger provides an opportunity to rationalize the approach to transport subsector planning. It could include strengthening the capacity of the staff of the Ministry of Transport, Works, Supply, and Communications (MTWS&C) and building the capacity of the Ministry of Finance staff to work in integrated way with transport sector planners and clarify procedures for the preparation of short- to me- dium-term plans and a prioritization process. The national transport policy could also be updated, rationalized, and clearly prioritized, including improving integration of higher-level macroeconomic planning with the current level of Zambia’s 2010/11– 2012/13 medium-term expenditure framework. Other actions are to introduce objec- tive criteria to policy prioritization and project selection procedure in a transparent way. 3. Transport governance dimension: Budget discipline (Zambia) Indicator 3 Country Sector/ MDA Current rating Target rating Budget allocations based on reliable financial forecasts Zambia Whole sector Red / amber Amber/green and aligned to priorities based on objective criteria 41 Transport governance indicators in Sub-Saharan Africa Evidence supporting the current rating According to the auditor general’s report, there was a significant commitment by the Zambia’s Road Development Agency between 2006 and 2009. The document reports that in 2008 the RDA committed the government to contracts totaling K 1.643 tril- lion, despite having a total projected budget of K 1.200 trillion (K 685 billion from the Zambian government and K 515 billion from donors). This allocation resulted in an overcommitment of K 443 billion. This overcommitment was also based on the initial unrealistic budget expectations, particularly for donor funding. The total final ex- penditure was K 842.42 billion, or about 50 percent of the RDA’s original commit- ments. In addition, a further five road projects, totaling K 182.455 billion, were pro- cured outside the 2008 work plan. The authority to procure these projects was not available for audit (Auditor General’s Office, Republic of Zambia 2009, 7–8). Potential policy actions in support of the target rating Possible policy actions include: reviewing the fiscal projection and resource allocation mechanisms and considering the appropriateness of the procedures of spending agencies and those entities providing funding and control of the mechanisms govern- ing them. Capacity strengthening would be needed in several agencies. Also need to increase awareness of the importance of routine, regular, and timely checks. 4. Transport governance framework: program design (Tanzania) Indicator 4 Country Sector/ MDA Current rating Target rating Quality and use of key performance and value for Tanzania Whole sector Red / amber Amber/green money indicators Evidence Supporting the Current Rating Performance indicators are drawn up by MDAs in agreement with development part- ners, but do not routinely include the value for money indicators. Other indicators across the road and port subsectors are routinely reported (e.g., at the Annual Joint Infrastructure Sector Review, JISR, meetings). Some key indicators are in line with the development partners’ performance assessment frameworks for general budget support. Potential policy actions in support of the target rating The MDAs should seek to highlight the most effective examples of reporting—and use them as examples for others. Less effective reporting entities should be lobbied to 42 A workable approach to measuring and managing transport sector governance ensure full sector coverage. Other actions would include: seeking to institutionalize the collection of data in all key organizations; seeking to incorporate data collection processes within financial and management information systems; continuing to work closely with planning and finance ministries and development partners to improve the system over time; and incorporating anticorruption/VFM indicators as key to performance. 5. Transport sector governance framework: procurement (Mali) Indicator 5 Country Sector/ MDA Current rating Target rating 18 Comprehensive and timely public disclosure of Mali Whole sector Red / amber Amber/green transport sector procurement plans (including bid- ding opportunities, contract awards, and data on resolution of procurement complaints) Evidence supporting the current rating A report produced in June 2011 for the Ministère de l’Economie et des Finances of Mali (ECORYS 2011) focused on shortcomings in the settlement of complaints and dis- putes to establish a PEFA 19 rating, against the backdrop of the establishment of a Dis- pute Resolution Committee in March 2010 (Decision No.002/ARMDS-CR). Docu- ments provided for the reporting exercise did not indicate the consequences of com- plaints. Overall, the rating was red/amber because “a mechanism recording and pro- cessing claims relating to the process of procurement is in place but its design is poor and it does not work in a manner allowing a timely resolution of claims.� However, it was noted that the creation in 2008 of the ARMD (Autorité de Régulation des Marchés Publics et des Délégations de Service Public) represented a major advance and offered real potential to reduce fiduciary risk. Potential policy actions in support of the target rating 18 Since the indicator was chosen post visit, data to exemplify its use is taken from a whole country report produced in June 2011 for the Ministry of Finance and Economy of Mali. 19 The goals of the PEFA program are to strengthen the ability of partner countries and donor agencies to (1) assess the condition of country public expenditure, procurement, and financial accountability systems, and (2) develop a practical sequence of reform and capacity-building actions. PEFA is a World Bank initiative. 43 Transport governance indicators in Sub-Saharan Africa Government could assess itself using PEFA standards and methodology, and these could be verified by auditors. Access to information could be improved—in the short term government could use places where people congregate (e.g., courts and church- es) to post information. Local councils could publish information affecting the local area; newspapers could be used to further widen dissemination. 6. Transport sector governance framework: delivery (Tanzania) Indicator 6 Country Sector/MDA Current rating Target rating Comprehensive time and cost reports on progress Tanzania Whole sector Amber / green Green of work for major (top 10) transport sector pro- jects, disclosed to the public in a timely and acces- sible manner Evidence supporting the current rating The Tanzania National Roads Agency (TANROADS) holds the original road project start and end dates, together with information on revisions to completion dates. Vari- ations are also recorded. According to the data. of the 10 transport projects reviewed, all but one of the projects had time overruns, ranging from 3 percent to over 100 per- cent (TANROADS 2011). The financial information also available matches spending with progress against the contract in percentage terms; information is provided as well on the issues requiring action. The reasons for delays and cost overruns included rectification of defects, poor site management, delayed mobilization of staff, the re- surveys and redesign required, poor utilization of equipment, customs delays, con- tractor disputes, breakdown of crusher plant, rain delays, slow release of compensa- tion payments (mostly crops), and suspension because of a lack of working space. The presentation of information could be improved to make it easier to interpret. Potential policy actions in support of the target rating Overall, the reasons for overruns and overspending could be more thoroughly ana- lyzed in order to find better ways to mitigate risk. TANROADS should tighten its as- sessments of contractor capacity and track records (without introducing a prequalifi- cation phase), so that the best-qualified contractors are engaged to carry out works. Furthermore, the quality of initial designs could be improved—to reduce the scope for introducing variation orders once project teams have mobilized—where condi- tions on site do not match those anticipated. 44 A workable approach to measuring and managing transport sector governance 7. Transport sector governance framework: sustainability (Tanzania) Indicator 7 Country Sector/MDA Current rating Target rating Transport sector agency has established bench- Tanzania Whole sector Amber / green Green marks for routine and periodic maintenance of assets and allocates the budget accordingly. Evidence supporting the current rating Using HDM-4 program analysis in April 2011, TANROADS determined the periodic maintenance needs of paved roads over the five years beginning with fiscal 2011/12. The annual financial requirement is some T Sh 88 billion for paved roads. However, T Sh 52 billion was allocated for periodic maintenance of paved roads in fiscal 2011/12, or only 59 percent of the amount needed. The financing gap between maintenance needs and maintenance spending overall fell from 65 percent in fiscal 2000/01 to 45 percent in fiscal 2010/11 (Ministry of Transport, Republic of Tanzania 2011; Ministry of Works, Road Fund Board). Although the coverage of routine maintenance dropped from 82 percent in fiscal 2008/09 to 69 percent in fiscal 2010/11 (this is explained by the expansion of the road network managed by TANROADS), periodic maintenance needs remained at a steadier level, about 58 per- cent. In line with this situation, the proportion of roads in good or fair condition has fallen. The backlog for paved road maintenance is estimated at T Sh 442billion, and a plan is in place to carry out this work over the five-year period. The Road Fund Board rec- ognizes that it faces a number of issues, the most pressing of which is insufficient funds to meet the huge backlog of maintenance stemming from deferred mainte- nance. Other issues include the inadequate capacity of agencies and some contractors to deliver quality road maintenance and the expanding network as the government increases the upgrading and building of new roads. The strategic plan for 2011–14 includes increasing funds and raising efficiency through better monitoring of works and curbing overloading as key objectives. Potential policy actions in support of the target rating The growing backlog of maintenance needs could, in theory, be reduced over time by increased funding. However, if full funding is unlikely, then a manageable system for reprioritization is required, which might involve allocating a greater proportion of road funds to maintenance rather than to upgrading. To facilitate such an approach, a 45 Transport governance indicators in Sub-Saharan Africa more accessible and more easily communicated set of data might need to be devel- oped that demonstrates the outcome of various potential policy and associated in- vestment decisions over the longer term. 8. Transport Governance Dimension: Information Dissemination (Zambia) Indicator 8 Country Sector/ MDA Current rating Target rating 20 Transparency and timeliness of annual budget and Zambia Whole sector Red Amber/green expenditure disclosures Evidence supporting the current rating The Civil Society for Poverty Reduction (CSPR), a nongovernmental organization, has introduced a budget tracking and service delivery barometer that bases its assess- ment on government administrative data. Among other supporting points, the CSPR’s 2010 report 21 finds that there is generally a lack of strong structures on the ground to involve citizens or inform them about the available resources, which would then enable them to monitor how these funds are allocated. The use of media such as newspapers to advertise disbursements is perceived to be ineffective because many poor people do not have the money to purchase newspapers and most advertisements are not broadcast on radio. When citizens do receive this information, there is no evidence that those who do receive it even understand it. Furthermore, the govern- ment publishes the Yellow Book for Zambia budget, but many people do not have access—nor is the Appropriation Act published. Potential policy actions in support of the target rating In the short term, government could use places where people gather (e.g., courts, churches) to advertise disbursements and local councils could publish their reports accounting for local resources. In addition, newspaper reporting could be enhanced. The government could also review Uganda’s policy on the publication of financial disbursements, which has been cited as good practice. 20 Since the indicator was chosen post visit, data to exemplify its use is taken from a whole country report produced in June 2011 for the Ministère de l’Economie et des Finances of Mali. 21 See http://www.csprzambia.org for the latest annual report. 46 A workable approach to measuring and managing transport sector governance 9. Transport Sector Governance Framework: External Scrutiny (Tanzania) Indicator 9 Country Sector/ MDA Current rating Target rating Rules applied to the membership and appointment Tanzania Whole sector 22 Red Green process for key transport sector governance boards Evidence supporting the current rating The membership and appointment process of the Tanzania Roads Fund Board needs strengthening to increase private sector participation and reduce the proportion of direct appointments. Under the current arrangements, the chair is appointed by the president from people outside the public service, and four road user representatives are appointed by the roads minister from the road transport industry, the tourism industry, the Chamber of Commerce, Industries and Agriculture, the Confederation of Tanzania Industries, the National Cooperatives Organization, and any other or- ganization of road users with no potential conflict of interest. The roads minister calls for nominations, the list of candidates is published, and objections are invited. Potential policy actions in support of the target rating SSATP could establish good practice guidelines and encourage the adoption of a standardized approach to membership and appointment rules across its member countries. 10. Transport sector governance framework: accountability (Zambia) Indicator 10 Country Sector/MDA Current rating Target rating Percentage of recommendations from independ- Zambia Whole sector Amber / green Green ent technical and financial auditor reports imple- mented within one year. 22 Since the indicator was chosen post visit, data to exemplify its use is taken from a whole country report produced in June 2011 for the Ministère de l’Economie et des Finances of Mali. 47 Transport governance indicators in Sub-Saharan Africa Evidence supporting the current rating The 2010 report of the auditor general on the Roads Development Agency recom- mended action against 51 findings. By November 2011, 34 (66 percent) of these had been implemented. Potential policy actions in support of the target rating Findings and associated recommended actions suggested by the Office of the Auditor General should be prioritized (high, medium, low importance) to enable a more stra- tegic approach to implementation. This approach should also be recommended to subordinate auditing bodies. The relevant bodies should receive support to meet five further prioritized goals within the next six months. Availability of data Indicators are most likely to be used if the data to support them are already being collected and used. Although it was not possible in the short time available for this study to undertake a comprehensive survey of data availability and reliability, an ini- tial attempt was made to summarize availability in the four pilot countries, and the summary findings of this exercise appear in appendix C. These findings can be veri- fied and improved (see the recommendations in chapter 5). It is hoped that the prospect of data availability and indicator ownership has been maximized by means of the consultative approach adopted by the study team. Under this approach, stakeholders have played a significant role in identifying the indicators they believe will work in practice. Institutional arrangements: Balancing internal and external engagement in data gathering and advocacy For any monitoring system to work, a number of key questions need to be answered at the institutional level: Who owns the indicators? Where are the data? How reliable are they? How are they gathered? Who collects them? What is the frequency of data collection? Who checks the data? How do feedback mechanisms work to ensure ac- countability? There are potentially up to five institutions that could  Gather the information (who is the data collector?)  Analyze the information (who is the data analyzer?)  Ensure that the data gathering is done (who is the M&E policy advocate?) 48 A workable approach to measuring and managing transport sector governance  Ensure that results are acted upon (who is the performance improvement advocate?)  Assume accountability for the result (who is responsible for reacting to the results?). Figure 4.1presents this arrangement visually. Figure 4.1 Institutional arrangements for data gathering and advocacy PERFORMANCE ACCOUNTABLE IMPROVEMENT INSTITUTION ADVOCACY MONITORING & DATA ANALYSIS EVALUATION ADVOCACY DATA GATHERING Like the indicators themselves, the M&E findings need to be fully owned by those in the best position to interpret and act on them. For this to be achieved, the M&E sys- tem, as well as the indicators, needs to be credible. All of the proposed transport gov- ernance indicators must to be institutionally owned, and it must be clear who does what—although sometimes more than one of the five roles just listed may be com- bined in one institution such as data gathering and analysis. For any monitoring system to be sustainable, the data collection should be routine and accurate. If it requires particular effort, the enthusiasm for collecting it is likely to lapse. Therefore, the decision on which institution will be the primary “owner� of an indicator is to some extent guided by the nature of the indicator itself. Note that the “right� institutional home will also depend on the level of data being examined. It is important that the institution have the competence to do what is required of it in terms of M&E. 49 Transport governance indicators in Sub-Saharan Africa In line with these requirements, one of the observed strengths of the indicator set is that much of the data required to support it—as described in the examples—is al- ready being collected (e.g., annual subsector budgets, annual programs, annual in- come and expenditure reports, and technical and financial audits). There is, however, scope for improvement in process and transparency in many cases. Table 4.2 lists generic MDAs across the pilot countries and the wider SSATP member country context with ownership potential for both data collection and gathering and policy (M&E system and performance improvement) advocacy. The last two columns show, by way of example, which specific organizations might be the “collecting� and �advocating� MDAs in Zambia. 50 A workable approach to measuring and managing transport sector governance Table 4.2 MDAs with institutional ownership potential, with Zambia as an example Suggested MDAs for Zambia Indicator Key MDA Collect and analyze the data Use the data (owning the indicators) (advocating for change) Institutional mandates and responsibilities Clarity of and distinction between mandates Transport Transport, Works, Supply and Communications Ministry of Finance and National Planning and responsibilities of key ministries, de- Local government Departments of Civil Aviation, Road Transport, National Road Fund Agency partments, and agencies (MDAs) in the Departments responsible for regulation Maritime and Inland Waterways, Planning Zambian Parliament: Parliamentary Com- sector such as civil aviation (airline operators, air Road Transport and Safety Agency mittee for Communications, Transport, traffic), maritime transport, port operations, Road Development Agency Works and Supply rail network and operations, Railway Systems of Zambia Government of Zambia: President’s Office road safety, vehicle , driver testing, licensing Local Government and Housing Department of Housing and Infrastructure Strategic priorities Coherence of transport sector policy and Transport Transport, Works, Supply and Communications Zambian Parliament: Parliamentary Com- extent to which its prioritization process is Civil society organizations (CSO), Think tanks mittee for Communications, Transport, based on objective criteria Works and Supply Financial management Budget allocations based on reliable finan- Finance Finance and National Planning Office of the Auditor General cial forecasts and aligned to priorities based National planning Transport, Works, Supply and Communications Central Statistical Office Zambia on objective criteria Line ministries Road Development Agency Zambia Revenue Authority Road Transport and Safety Agency CSOs—such as Civil Society for Poverty Planning Department Reduction (CSPR) and others Departments of Civil Aviation, Road Transport, Maritime and Inland Waterways, Planning National Road Fund Agency 51 Transport governance indicators in Sub-Saharan Africa Suggested MDAs for Zambia Indicator Key MDA Collect and analyze the data Use the data (owning the indicators) (advocating for change) Program design Quality and use of key performance and Line ministries, departments, and agencies Transport, Works, Supply and Communications Office of the Auditor General value for money indicators Finance and National Planning Central Statistical Office Zambia National Road Fund Agency Zambian Parliament: Parliamentary Com- Local Government and Housing mittee for Communications, Transport, Department of Housing and Infrastructure Works and Supply Road Development Agency Government of Zambia: President’s Office Road Transport and Safety Agency Planning Department Departments of Civil Aviation, Road Transport, Maritime and Inland Waterways Monitoring and Evaluation Unit Procurement Comprehensive and timely public disclose of Ministries responsible for transport (includ- Transport, Works, Supply and Communications Office of the Auditor General transport sector procurement plans (includ- ing roads) Road Development Agency Central Statistical Office Zambia ing bidding opportunities, contract awards, Subsector agencies Road Transport and Safety Agency Parliamentary Committee for Communica- and data on resolution of procurement Procurement authority and regulator Departments of Civil Aviation, Maritime and tions, Transport, Works and Supply complaints) Inland Waterways Government of Zambia: President’s Office Monitoring and Evaluation Unit, Communica- Media groups (e.g., Daily Mail, Post, and tions and Transport Times of Zambia) Zambia Public Procurement Authority () Delivery Comprehensive time and cost reports on Ministries responsible for transport and Transport, Works, Supply and Communications Central Statistical Office progress of work for major (top 10) transport roads Department of Road Transport Media groups (e.g., Daily Mail, Post, and sector projects, disclosed to the public in a Road Transport and Safety Agency Times of Zambia) timely and accessible manner Monitoring and Evaluation Unit Communica- CSOs 52 A workable approach to measuring and managing transport sector governance Suggested MDAs for Zambia Indicator Key MDA Collect and analyze the data Use the data (owning the indicators) (advocating for change) tions and Transport Sustainability Transport sector agency has established Ministry responsible for finance, planning Transport, Works, Supply and Communications Ministry of Finance and National Planning benchmarks for routine and periodic Ministries responsible for transport (includ- Road Development Agency National Road Fund Agency maintenance of assets and allocates the ing roads) Local Government and Housing Office of the Auditor General budget accordingly Subsector agencies Department of Housing and Infrastructure Parliamentary Committee for Communica- tions, Transport, and Works Government of Zambia: President’s Office Donor groups Information dissemination Transparency and timeliness of annual Ministries responsible for finance, planning, Finance and National Planning Office of the Auditor General budget and expenditure disclosures national development National Road Fund Agency Donor groups Ministries responsible for transport & roads Transport, Works, Supply and Communications Zambian members of Parliament Subsector agencies Road Development Agency Parliamentary Committee for Communica- Civil society, media organizations Road Transport and Safety Agency tions, Transport, Works and Supply Departments of Civil Aviation, Maritime and President’s Office Inland Waterways District and council administrative officials Ministry of Local Government and Housing Media groups Department of Housing and Infrastructure Civil society organizations 53 Transport governance indicators in Sub-Saharan Africa Suggested MDAs for Zambia Indicator Key MDA Collect and analyze the data Use the data (owning the indicators) (advocating for change) External scrutiny Rules applied to the membership and ap- Ministries responsible for transport Transport, Works, Supply and Communications Office of the Auditor General pointment process for key transport sector President’s or prime minister’s office (if Road Development Authority Zambian Parliament—members and Com- governance boards involved) Finance and National Planning mittee for Communications, Transport, and Attorney general (chief legislative authority) National Road Fund Agency Works Subsector agencies President’s Office Transport users, civil society. media Media groups Accountability Percentage of recommendations from inde- Ministries responsible for transport Transport, Works, Supply & Communications Office of the Auditor General pendent technical and financial auditor Subsector agencies Road Development Agency Central Statistical Office Zambia reports implemented within one year Journalists, media Road Transport and Safety Agency Zambian members of Parliament Planning Department Zambian Parliament: Parliamentary Com- Departments of Civil Aviation, Road Transport, mittee for Communications, Transport, Maritime and Inland Waterways Works and Supply Private sector entities (e.g., railway) Government of Zambia: President’s Office Contractors and consultants (e.g., Association of District and council administration officials Building and Civil Engineering Contractors) Media groups Civil society groups (e.g., community-based church groups) 54 A workable approach to measuring and managing transport sector governance Figure 4.1 is now repeated as figure 4.2, using Zambia as an example. Figure 4.2 Indicator 4: Quality and Use of Key Performance and Value for Money MINISTRY OF TRANSPORT, OFFICE OF THE AUDITOR WORK, SUPPLY AND COM- GENERAL MUNICATIONS CENTRAL STATISTICAL MONITORING AND EVALUA- OFFICE supported by TION UNIT, MCT the PRESIDENT’S OFFICE SUB-SECTOR DEPARTMENTS AND AGENCIES To enable this particular institutional arrangement, support might be required to  Identify key performance indicators and indicators of value for money  Build capacity for data gathering  Provide clarity on what is needed in data for the Monitoring and Evaluation Unit (memoranda of understanding)  Establish and agree on reporting and risk escalation arrangements  Form and support working discussions. Although there will be similarities, institutional arrangements for each indicator will not be standardized across countries, and it will be important to map what is most appropriate to each case. A mapping exercise covering all 10 indicators in one pilot country and establishing frequency and direction of data flow (based on the outline methodology described in the next section) could form the basis for wider replication and comparison across the SSATP countries. 55 Transport governance indicators in Sub-Saharan Africa Methodology for data collection To perform their monitoring and evaluation and policy advocacy tasks well, institu- tions need clarity on how data should be gathered, baselines established, and targets set. Guidance might be needed on (sampling) methodology, periodicity of data col- lection, the reporting format, and dissemination channels. Because of the wide scope of this study and the time available, coupled with the fact that the final indicator set was selected after the visits, there has been only limited time and scope to consider methodologies for data collection within specific institutional contexts. However, table 4.3 suggests an approach and preliminary set of standards, proposing the follow- ing frequency of indicator assessment: Annual (institutional, strategic, and accountability indicators):  Mandates  Priorities  Budget and expenditure disclosures  Board membership  Audit recommendations Biannual (budget allocation and sustainability indicators):  Budget allocation  Benchmarks Quarterly (program development and delivery indicators):  KPIs  Procurement  Progress. A summary of the quarterly (management) and biannual (operational) indicator re- ports could be incorporated within an annual (strategic) governance assessment. This tentative proposal would need pilot testing before being formalized (see chapter 5). 56 A workable approach to measuring and managing transport sector governance Table 4.3 Data Collection Considerations Indicator Type of data Frequency of Collection method Who collects and Challenges collection analyzes data Clarity of and distinction Qualitative Annual Corporate planning Ministry of national Mandate overlaps can between mandates and re- methodology planning have implications for sponsibilities of key MDA in staff numbers. the transport sector Coherence of transport sector Qualitative Annual Stakeholder workshops Ministry of national Would require good policy and extent to which its planning facilitation of negotia- prioritization process is based tion and consensus on objective criteria building. Budget allocations based on Quantitative Biannual Database to record Ministry of finance Accurate data may be reliable financial forecasts and forecasts and alloca- difficult to access. aligned to priorities based on tions against priori- objective criteria ties—and priorities against criteria Quality and use of key per- Qualitative Quarterly Key performance indi- Transport sector Would need initial formance and value for mon- cator (KPI) progress departments submit assistance to ensure ey indicators reports based on RAG to monitoring and KPIs are meaningful. (red-amber-green) evaluation (M&E) Memorandum of ratings submitted quar- unit, ministry of understanding may be terly to ministries of transport, or central necessary to ensure finance and national statistical office compliance. planning Comprehensive and timely Qualitative Quarterly Database established to Procurement au- Accuracy may be public disclosure of transport allow easy generation thority collects from difficult to check. sector procurement plans of summary tables and spending agencies; Advocacy channels (including bidding opportuni- other supporting doc- analyzed by audit will need to be clear. ties, contract awards, and umentation such as office data on resolution of pro- press releases curement complaints) Comprehensive time and cost Quantitative Quarterly Database established to M&E unit Accuracy may be reports on progress of work allow easy generation Civil society moni- difficult to check. for major (top 10) transport of summary tables and tors Advocacy channels sector projects, disclosed to other supporting doc- will need to be clear. the public in a timely and umentation such as accessible manner press releases Transport sector agency has Quantitative Biannual Expert reports Ministry of transport Difficult to communi- established benchmarks for cate results meaning- routine and periodic mainte- fully; guidance neces- nance of assets and allocates sary the budget accordingly Transparency and timeliness Quantitative Annual Database established Civil society moni- Nil of annual budget and ex- for easy generation of tors penditure disclosures summary tables and documentation such as press releases 57 Transport governance indicators in Sub-Saharan Africa Rules applied to the member- Qualitative Annual Annual submission to Audit office Nil ship and appointment pro- audit office cess for key transport sector governance boards percentage of recommenda- Quantitative Annual Formal response to Audit office Donor group could tions from independent tech- Supported audit office based on support but may be nical and financial auditor by qualita- RAG rating tables to reluctant to delay reports implemented within tive infor- show response and new spending to promote one year mation rating recommendations. Interventions to support building M&E capacity A good indicator is only as good as the system that is designed to capture its data re- quirements, the quality of the data collected, and the capacity of its institutional own- er to process and use the data collected. Because of the low data quality and statistical capacity 23 of countries in Sub-Saharan Africa, further work might be needed on pro- ducing clearer mandates (and funding) for national statistical and monitoring and evaluation institutions. Binding agreements and related action on providing M&E units with data might be needed between MDAs, along with changes in procedures that lead to institutionalization of information requests and receipt and response pro- cesses. Although the methodology suggested for data collection and use is intentional- ly simple, additional capacity building of institutional personnel to improve data analysis may still be needed. Intervention options in the governance cycle From an institutional perspective, not only is monitoring the indicator important but also acting on the findings. As noted earlier, the owner of the indicator in terms of action may be different from the owner in terms of monitoring. The indicators pro- posed are all actionable, and so should be acted upon. The type of institutional “home� that is appropriate for an action will depend on the action itself, which will in turn depend in part on the nature of the RAG rating. If scores are generally toward the green end of the scale, only minor interventions will be needed (by the accounta- ble agency). If more serious shortcomings are identified, then intervention at a higher institutional level may be needed (by the performance advocacy agency). It is possible 23 For example, the institution’s ability to adhere to international standards on methods and data reporting in social and economic statistics, to collect data at recommended intervals, and to make data available for users in of international data sources. See World Bank (2010) 58 A workable approach to measuring and managing transport sector governance Table 4.4 Suggested actions to improve indicator rating and country performance SUGGESTED POLICY ACTION 1. Distinct institutional mandates I. Create the appropriate legislation, regulations, and procedures (or amend the existing ones). II. Improve the policy planning capacity of the ministry of transport to oversee the entire sector. III. Develop clear indicators of responsibility to ensure no interference by non-mandated bodies. IV. Introduce corporate planning methodologies to negotiate distinct mandates. 2. Integrated planning and prioritization I. Strengthen the capacity, within MoT in particular, to develop integrated sector plans, which can be used to guide resource prioritization; develop or improve MoT’s ability to link higher-level macroeco- nomic planning with a medium-term expenditure framework (MTEF). II. Introduce appropriate integrated transport sector coordination and prioritization; adopt a rational ap- proach to resource allocation across subsectors, with linked annual programs and budgets. 3. Budget alignment I. Seek a binding MTEF that includes linked, objectively verifiable indicators. II. Introduce a fiscal projection model. III. Improve monitoring capacity. 4. Improved KPIs: Design, data collection, and reporting I. Strengthen the capacity across relevant ministries and subsector agencies to develop the appropri- ate sector and subsector performance indicators. II. Identify value for money (VFM) indicators and build a consensus around selection. INDICATOR S III. Institutionalize quality data collection processes across key organizations. IV. Incorporate data collection processes within financial and management information systems. V. Lobby less effective reporting entities to ensure full sector coverage. 5. Transparent procurement I. Split procurement from the implementing authority. II. Institute external audit of the processes and key procurement performance milestones. III. Explore self-assessment by government using Public Expenditure and Financial Accountability (PE- FA) standards and methodology; these could be verified by auditors. IV. Improve access to information—in the short term, government could use places where people congregate (e.g., courts and churches) to post information. V. Consider having local councils publish information affecting the local area. VI. Use newspapers to further widen dissemination. 6. Reduced time and cost overruns I. At a higher level, make project financial commitments—and contract awards—only when funds are actually available and not expected (pending bills). II. Tighten assessment of contractor capacity and track record to ensure that only qualified contractors are invited to tender. III. Improve the planning, quality, and detail of initial designs.a IV. Apply better systems and procedures and adopt some key procurement performance indicators.b V. Better apply the International Federation of Consulting Engineers (FIDIC) or a similar approach so contracts are managed more effectively from start. VI. As a longer-term intervention, increase the local contracting capacity; by increasing the size of the pool of qualified contractors, competition can be enhanced and better value for money achieved.c 59 Transport governance indicators in Sub-Saharan Africa 7. Appropriate investment in maintenance I. Agree on a standard for maintenance spending and prioritization and set it as the benchmark. II. Establish specific benchmarks for routine and periodic maintenance by classification of road. III. Maintain a database on network length and conditions where possible. IV. Support periodic surveys where necessary. V. Improve the accuracy of budget predictions and the revenue forecasts that accompany them. VI. Provide technical assistance to second-generation road funds to improve budget forecasts of the level of demand for fuel and likely receipts. VII. Exert budget discipline at the line ministry level to ensure that funds are not diverted between cat- egories (e.g., from maintenance to rehabilitation). VIII. Make a clear distinction between capital expenditure and maintenance expenditure. INDICATOR S 8. Improved information disclosure I. Create a budget act that would allow for meaningful consultation on the budget-making process.d II. Encourage government to use gathering places (e.g., courts, churches) to advertise disbursements.e III. Speed up progress in implementing the Integrated Financial Management Information System (IFMIS) as a component of the Public Expenditure Management and Financial Accountability initia- tive Encourage local councils to publish their reports accounting for local resources. 9. Improved board membership and appointment I. Change the legislation governing the institution in question. II. Draw up good practice guidelines and adopt a standardized approach in the SSATP countries. 10. Transparent and timely implementation of audit recommendations I. Regularly review and publish progress reports by the auditor general and the relevant subsector agencies. II. Encourage media and civil society organization involvement and engagement in the process to act as an outside check. a. Thereby reducing the scope for introducing variation orders once project teams have been mobilized and where conditions on site do not match those anticipated. b. These measures could reduce the time between the preliminary design and the award of design and super- vision contracts if excessive time has elapsed. c. Steps to be followed could include: (1) building the local agency capacity to manage the procurement pro- cess; (2) introducing preferential procurement acts; (3) registering contractors; (4) introducing a training provi- sion for small-scale contractors; and (5) instituting equipment hire arrangements d. See Afrobarometer on citizen consultation and International Budget Partnership’s Open Budget Initiative. e. Reviewing Uganda’s policy on publication of financial disbursement, which has been cited as good practice. that the intervention of a separate, empowered institution or agency (e.g., parliamen- tary committee, ministry of finance, ministry of planning and development, or office of the auditor general) may be required to provide the independent push that will change the way things are done. Some changes may only be effected through the drafting and introduction of new legislation. Thus the indicator scores will lead to identification of the institutions that should be the focus of (1) intervention support and (2) advocacy support. Even though the interventions needed to raise the RAG rating of an indicator toward a target will depend both on the level at which the indi- cator is applied and the country score/rating, a non-exhaustive list of practical policy action options are suggested for each indicator in table 4.4. 60 Conclusions and recommendations This study has sought to identify indicators that offer the potential to credibly meas- ure governance in the transport sector. This potential is understood to be linked not only to the relevance of the indicator to key issues affecting the sector but also to the capacity of local institutions to embrace the indicators by monitoring and recording results and acting on findings. The study deliberately avoids generating “just another set� of governance indicators. Rather, it is firmly focused on the issues that emerged from the views of in-country stakeholders in order to generate indicators that are clear and measurable and that lend themselves to policy action. This focus is apparent throughout the paper, from the refinement of the framework around the governance cycle for transport (enabling the reduction of 170 plus indicators to a subset that makes sense individually to those involved in the cycle and yet operate collectively), through the development of indica- tors, RAG ratings, and suggested policy actions that themselves aim to track progress, which lies at the core of the governance problem. The process adopted and the findings produced by the study suggest some interesting first conclusions, as well as options for the consolidation of the approach and the findings as the basis for wider rollout. The key findings of the study are the following:  Presenting governance as a complex, abstract concept at best constrains stake- holder engagement and at worst provides a smoke screen behind which poor governance can be sustained. Thus indicators need to be very precise and prac- tically applicable, and ironically this excludes some of the more robust (inter- nationally recognized) indicator sets that are based on composite scores (or indexes).  Although it was initially suggested that a transport governance indicator be proposed per subsector, because of the many cross-sectoral issues identified in the master indicator list developed in the early stages of the study, it was deemed more appropriate and more practical to use governance dimensions to organize the indicators.  There is an expressed and credible desire to own indicators, introduce a moni- toring process, and respond to the results, but for this to be realized, institu- tional considerations have to be taken seriously. In particular, decisions about where to locate “ownership� of the monitoring and data collection processes, as well as the target for policy advocacy, need to be carefully considered. No matter what MDA or other organization is selected, technical competence 61 Transport governance indicators in Sub-Saharan Africa needs to be balanced against political credibility when advocating particular policy actions.  The collection and sharing of information must be backed by strong policy di- rectives and service delivery agreements in order to facilitate the necessary flow of data to the institutions charged with data compilation. This finding is very country context–specific because of the ministerial rivalries and political un- dercurrents between MDAs, and so the paper presents possibilities and exam- ples of institutional arrangements rather than proposing a standardized insti- tutional configuration.  Indicators that are already understood and in use work best. Because of the poor performance against many of these in the countries visited, there appears little reason to embed governance measures in more composite indicator frames. In any case, composite indicators are perceived to obscure realities through artificial “smoothing� and carry the further risk of being seen as the contrivance of external players.  However, for even a simple set of indicators (and one that is appropriately monitored) to offer any added value in governance, the indicators would need to be acted on in ways that lead to measurable results. These actions may re- quire external advocacy or support—for example, to consolidate the introduc- tion of medium-term expenditure frameworks in Mali. In order to realize the potential benefits of the indicator set, the following next steps are recommended:  SSATP should receive the support needed to score one (or more) of the four countries in the study to refine the RAG ratings. Such comprehensive and de- tailed scoring was not possible during the study because indicators could not be finalized until the last visit had been completed.  Such a pilot project would also provide an opportunity to produce a first transport sector governance framework (TSGF) report. It would involve set- ting a baseline and prioritizing according to the needs in the country selected to determine which sectors and or MDAs offer the greatest potential for re- form. The pilot project would also help to determine how actionable the indi- cators are and how easy it would be to develop a program of support and self- help to shift the pilot country toward its targets in view of its priority indica- tors.  A scoring and needs assessment methodology could then be formalized for wider use across SSATP countries. 62 Conclusions and recommendations  The pilot country could be the focus of an institutional mapping exercise to determine whether the monitoring and evaluation, advocacy functions, and institutional arrangements to support the indicators are in place and what, if any, capacity-building support would be required. Consideration could also be given to whether it would be appropriate to identify an overall owner of the transport sector governance framework (and reports).  Later, SSATP could conduct a robust impact assessment to determine the re- lationship between improvements in the indicator scores in the pilot country and transport sector performance, as well as a cost-benefit analysis to com- pare the costs of consolidating M&E and advocacy functions around the framework with the potential benefits of improved performance. For gov- ernance to become further embedded in the transport sector, further evi- dence may be required to show the link (or otherwise) between the subset of indicators and tangible improvements in delivery across the transport sector.  Throughout the process, focused briefing papers and requests for feedback on the transport sector governance framework could be produced regularly for SSATP representatives to maximize engagement and ownership. Initially, SSTAP could receive support to identify and develop more accessible ways to present the proposed indicator set to key stakeholders in each pilot country (e.g., TSGF charts, graphics). For many stakeholders, their interest and potential ownership of these indicators may be best achieved not just by sharing this paper but also by using it as a basis for more accessible presentations, short summary guides, and spe- cific forums or meetings. 63 Appendix A. Long list of indicators The following long list of indicators was drawn from across the wide range of litera- ture reviewed (see bibliography). 1. Funding mechanism for maintenance and rural access development releases funds in an appropriate and timely fashion. 2. Composition of actual public expenditure compared with original approved budget. 3. Average annual expenditure per kilometer of main road. 4. Capital expenditure as a percentage of rehabilitation needs. 5. Alignment of national strategy/priorities with budget allocation. 6. Clearly defined and understood road fund allocation rules. 7. Average maintenance spending across different parts of the network. 8. Cost overruns. 9. Independently verifiable criteria applied to allocation of subsidies to support essential transport services that are not economically viable. 10. External audits of transport agency are conducted on an annual basis. 11. Transport agency receives regular and accurate reports from its departments or divisions on the use of funds allocated to them. 12. Parent ministry (or ministry of finance) has functional reporting systems to follow up on the flow of budgetary resources to the transport agency. 13. Predictability and accuracy of government financial forecasts. 14. Percentage of road spending allocated to capital projects. 15. Time overruns. 16. Fares are set according to a transparent formula and process. 17. Transport authority has established an adequate internal control framework, including financial audit. 18. Road fund has an independent board (i.e., private sector majority representation). 19. Average time for contractors to receive payments following submission of invoices (approved work). 20. Road agency’s external audit report reviewed by parliament (or other appropriate body). 21. Transport agency prepares budget execution reports in a timely fashion. 22. Average unit cost of supervision per kilometer of road. 23. Balance between investment and maintenance. 24. Capital budget execution rates. 25. Average unit cost per kilometer of road. 26. Road fund in existence. 65 Transport governance indicators in Sub-Saharan Africa 27. Transport authority implements all recommendations from external audit reports within one year. 28. Transport authority monitors and clears expenditure payment arrears on a regular basis, and so there are no arrears over 30 days. 29. Fuel levy relative to optimal requirements for maintenance and rehabilitation. 30. Commercial management of road funds. 31. Adequate scope, nature, and follow-up of external audit reports. 32. Road agency carries out effective cash flow planning, management, and monitoring. 33. Transport authority implements all recommendations from internal controls within one year. 34. Transport agency keeps aggregate budget overruns (if any) within a reasonable range. 35. Tariff cost recovery. 36. Fuel prices in cities. 37. Transport agency gives fiduciary staff sufficient authority on day-to-day execution of fiduciary functions and procedures. 38. Balance of long- and short-term public expenditure planning. 39. Sustainability of assets. 40. Transport agency allocates adequate administrative budget for capacity development of fiduciary staff. 41. Potential efficiency gains. 42. Transport agency has an effective payroll control system. 43. Wage bill as a proportion of government expenditure compared with benchmark. 44. Application of unit cost in identification of needs. 45. Perception that complaints from the public are handled fairly. 46. Transport sector agency periodically collects data on the level of service/condition of its road network. 47. Explicit criteria applied as the basis for agreement of all transport concessions. 48. Transport agency adopts clearly defined criteria to evaluate bids. 49. Transport agency provides bidders with sufficient information, clarification, and time to prepare bids. 50. Road agency conducts technical audits for large works contracts (or a sample of them). 51. Percentage of the requested budget actually received on average (over last three years) on an annual basis by the transport sector agency. 52. Transport agency has established service users' feedback mechanisms, including satisfaction surveys. 53. Transport agency adopts objective investment decision tools (such as HDM-4 in road sector). 54. Transport agency periodically updates unit prices used for cost estimates based on the latest contracts or other infor- mation available. 55. Delegated financial authority of transport agency staff commensurate with their level of responsibility. 56. System for allocation of routes to operators. 57. Transport agency appoints members of tender commissions who are competent individuals. 58. Availability of information on resources received by service delivery units. 59. Regulation of private rail operators. 66 Appendix A. Long list of indicators 60. Safety standards and enforcement. 61. Impartial contract enforcement procedures. 62. Transport agency uses procurement selection criteria that adequately consider technical, financial, managerial, and im- plementation issues and does so in a timely fashion. 63. Transport agency’s share of administration cost, as a percentage of the total budget, is considered reasonable. 64. CEO has authority to define organizational structure, set salary levels, and hire and fire staff based on functional needs and performance (in line with the applicable labor legislation). 65. Average number of months between bid opening and contract award. 66. Percentage of bid submissions out of the total that purchased bid documents. 67. Unit prices of works (e.g., the cost of a cubic meter of asphalt concrete) carried out by the road agency considered rea- sonable. 68. Average number of bidders per contract. 69. Use of objective planning criteria and analysis. 70. Competition, value for money, and controls in procurement. 71. Legal and contractual relationships between the road agency and bidders considered fair. 72. International firms generally allowed to participate in bidding for works contracts. 73. Road agency keeps its key positions properly filled with qualified individuals (or, in the case of vacancies, they are filled in less than a year). 74. Consistency of policy toward infrastructure across modes. 75. Adequate qualification a main factor in the selection of members of the management team (including board members) of the road agency. 76. Proportion of expenditures on road maintenance and expenditures on road construction considered reasonable. 77. Quality of processes, rules, and practices. 78. Functioning monitoring systems. 79. Reasonable difference between contract values and engineer’s estimates. 80. Transport agency’s management performance periodically evaluated against policy objectives. 81. Transport agency maintains a well-established and functional management information system. 82. Effective public sector management. 83. Transport agency’s management and staff regularly discuss skills development needs and service improvement. 84. Transport agency attracts and retains qualified employees. 85. Fair procedures. 86. Transport agency’s staff performance evaluated regularly against outcomes and professional behaviors. 87. Transport agency’s overall human resources policy linked with and reviewed against policy objectives. 88. Checks and balances. 89. External (civil society, media, or private sector) oversight of public sector resource allocation. 90. Public hearings are conducted for policy changes or large investment projects. 67 Transport governance indicators in Sub-Saharan Africa 91. Citizen engagement in policy, planning, and monitoring processes. 92. Results of public hearings taken into account in the decision-making process. 93. Independent venues (Internet, hotline) available for citizens and firms to raise concerns related to the road sector (e.g., large road contracts, quality of works, road condition). 94. Scope, nature, and follow-up of external audits. 95. Use of social audits. 96. Transport sector priorities are consistent with those highlighted through advocacy by civil society organization. 97. Decentralization and local participation. 98. Participation in the annual budget process. 99. Donor coordination. 100. Number and percentage of bids published in local newspaper. 101. Disclosure of annual budgets and expenditures. 102. Extent to which information is made publicly available—e.g., procurement publications. 103. Publication of progress against performance indicators. 104. Road work programs and procurement plans disclosed to the public. 105. Transport service user satisfaction surveys results disclosed to the public. 106. Disclosure of progress to public. 107. Publication of corrective action against poor performance. 108. Road agency's audit report made available to the public. 109. Public transport works contracts and names of winning bidders disclosed to the public. 110. Transport agency has a home page or information center that discloses road sector information (e.g., updated road condi- tion). 111. Progress on and costs of works contracts (particularly large roads) reported to the public. 112. Progress on performance indicators published in annual reports that are available to the public. 113. Disclosure of audit reports. 114. Freedom of information laws. 115. Road agency action to address concerns raised by the public made public. 116. Policy objectives of the transport agency disclosed to the public. 117. Annual reports of the transport agency published. 118. Disclosure of relationship between contract values and estimates. 119. Media regularly address an issue. 120. Transport agency issues regular press releases. 121. Public access to key fiscal information. 122. Annual budget and expenditures disclosed to the public. 123. Extent of unreported government operations. 124. Public opinion of key performance standards (e.g., levels of corruption, quality of works). 68 Appendix A. Long list of indicators 125. Perceptions of unnecessary delay/expense in the customs clearance process. 126. Average time of delays faced by truckers due to authorized or nonofficial "checks." 127. Clearly defined and mandated allocation of responsibilities related to planning and funding of transport infrastructure. 128. Transport agency publicly advertises all tenders. 129. For the selection of consultants, all short-listed firms qualified for the proposed job. 130. Road agency keeps records (including completion reports and communications with bidders) for all contracts. 131. Number of days elapsed from tender advertisement to contract award considered reasonable. 132. Average level of bribes and other facilitation payments paid by truckers. 133. Percentage of investigated anticorruption cases (by sector) leading to prosecution. 134. All bidders meeting published qualification criteria invited to bid openings. 135. Number of complaints for the procurement of goods, works, and services considered reasonable. 136. Explicit procedures in place requiring members of tender commission with potential conflicts of interest to declare these; procedures in place to prohibit noncompliance. 137. Issues of urban mobility—percentage of income spent on transport. 138. Measures to reduce corruption. 139. Percentage of national budget dedicated to anticorruption body. 140. Average number of delays for truckers. 141. Existence of regulatory institutions for rural and urban transport services. 142. Road agency has established a code of ethics. 143. Third parties allowed to attend bid opening sessions. 144. Number of tenders rejected in each bidding considered reasonable. 145. Government supervision focused on monitoring and controlling contracts. 146. Purchase of luxury vehicles for official use. 147. Bribe payments. 148. Distribution of high-level appointments. 149. Levels of administrative discretion. 150. Artificial creation of queues. 151. Key performance indicators have been developed, and performance against these are reported both to the public and to politicians. 152. Perception that public sector officials in "lead agency" override regulations and procedures with impunity. 153. Perception that politicians override regulations and procedures that govern the transport sector with impunity. 154. Implementation of recommendations from external audit reports within one year. 155. Concerns raised in public hearings taken into account in decision making. 156. Annual (or other) report discusses corrective measures taken to address poor performance (e.g., to address poor road conditions). 69 Transport governance indicators in Sub-Saharan Africa 157. Central authority ensures appropriate coordination of sector strategies according to agreed-on criteria and standards. 158. Capability of core planning ministry for strategic thinking, policy development, and control of plans. 159. Clear sanctions are in place and enforced against explicitly defined standards. 160. Investments in transport activities explicitly linked to economic growth and poverty reduction priorities. 161. Transport user satisfaction. 162. Average passenger fare per kilometer. 163. Percentage of households reporting transport costs as a major constraint to employment. 164. Percentage of paved roads network managed by the road agency in good condition. 165. Percentage of unpaved roads managed by the road agency in good condition. 166. Affordability of transport or portion of household budgets needed to provide adequate transport. 167. Existence of pro-poor and pro-growth transport strategy meeting explicit standards of consultation and coverage. 168. Percentage of schools with reliable access. 169. Percentage of households reporting constraints on access to education or health services because of transport difficulty or cost. 170. Availability of public transport in cities. 171. Number of buses per 1,000 inhabitants. 172. Percentage of population living beyond 2 kilometers of an all-season road. 70 Appendix B. Short list of indicators Dimension 1: Financial management and value for money systems 1.01 Funding mechanism for maintenance and rural access development releases funds in an appropriate and timely fashion. 1.02 Composition of actual public expenditure compared with original approved budget. 1.03 Average annual expenditure per kilometer of main road. 1.04 Capital expenditure as a percentage of rehabilitation needs. 1.05 Alignment of national strategy and priorities with budget allocation. 1.06 Percentage of the requested budget actually received on average (over last three years) on an annual basis by the transport sector agency. 1.07 Average maintenance spending across different parts of the network. 1.08 Cost overruns (amounts). 1.09 Fares are set according to a transparent formula and process. 1.10 Average time for contractors to receive payments following submission of invoices (for approved work). Dimension 2: Administrative and regulatory procedures 2.01 All bidders meeting published qualification criteria invited to bid openings. 2.02 Explicit criteria applied as the basis for agreement for all transport concessions. 2.03 Access to market for bus and truck operators (for new entrants). 2.04 Transport agency adopts clearly defined criteria to evaluate bids. 2.05 Transport agency provides bidders with sufficient information, clarification, and time to prepare bids. 2.06 Clearly defined and mandated allocation of responsibilities related to planning and funding of transport infrastructure. 2.07 Transport agency has well-established service users' feedback mechanisms, including satisfaction surveys. 2.08 Transport agency adopts objective investment decision tools (such as Highway Design and Maintenance Standards in road sector). 2.09 Transport agency periodically updates unit prices used for cost estimates based on the latest contracts or other information available. 2.10 Regulatory body exists and functions effectively within subsector (air, rail, road, waterways). Dimension 3: Third-party engagement and transparency and access to information 3.01 Disclosure of annual budgets and expenditures. 3.02 External (civil society, media, or private sector) oversight of public sector resource allocation. 3.03 Majority of private sector/civil society representation on road fund board. 71 Transport governance indicators in Sub-Saharan Africa 3.04 Public hearings conducted for policy changes or large investment projects. 3.05 Key performance indicators have been developed, and performance against these are reported both to the public and to politicians. 3.06 Transport sector work programs and procurement plans disclosed to the public. 3.07 Transport service user satisfaction surveys results disclosed to the public. 3.08 Transport agency's audit report made available to the public. 3.09 Number and percentage of public transport works contracts and names of winning bidders disclosed to the public. 3.10 Clearly defined and understood road fund allocation rules. Dimension 4: Anticorruption effort 4.01 Perceptions of unnecessary delay or expense in the customs clearance process. 4.02 Average time of delays faced by truckers because of authorized or nonofficial "checks." 4.03 For the selection of consultants, all short-listed firms are qualified for the proposed job. 4.04 Transport agency keeps records (including completion reports and communications with bidders) for all contracts. 4.05 Number of days elapsed from tender advertisement to contract award considered reasonable. 4.06 Average level of bribes and other facilitation payments paid by truckers. 4.07 Percentage of investigated anticorruption cases (by sector) leading to prosecution. 4.08 Perception that public sector officials in "lead agency" override regulations and procedures with impunity. 4.09 Perception that politicians override regulations and procedures that govern the transport sector with impunity. 4.10 Perception that complaints from the public are handled fairly. Dimension 5: Accountability 5.01 Public opinion of key performance standards (e.g., levels of corruption, quality of works). 5.02 Implementation of recommendations from external audit reports within one year. 5.03 Concerns raised in public hearings taken into account in decision making. 5.04 Legally accountable and responsible (capital) city or metropolitan transport authority in place (with clearly defined mandate and responsibilities). 5.05 Independently verifiable criteria applied to allocation of subsidies to support essential transport services that are not economically viable. 5.06 External audits for transport agency are conducted on an annual basis. 5.07 Transport agency receives regular and accurate reports from its departments or divisions on the use of funds allocated to them. 5.08 Parent ministry (or ministry of finance) has functional reporting systems to follow up on the flow of budg- etary resources to the transport agency. 5.09 Publication of corrective action against poor performance. 5.10 Transport sector agency periodically collects data on the level of service or condition of its road network. 72 Appendix B. Short list of indicators Dimension 6: Equity of benefits 6.01 Investments in transport activities explicitly linked to economic growth and poverty reduction priorities. 6.02 Transport user satisfaction. 6.03 Average passenger fare per kilometer. 6.04 Percentage of households reporting transport costs as a major constraint to employment. 6.05 Percentage of paved roads network managed by the road agency in good condition. 6.06 Percentage of unpaved roads managed by the road agency in good condition. 6.07 Affordability of transport or portion of household budgets needed to provide adequate transport. 6.08 Percentage of schools with reliable access. 6.09 Percentage of households reporting constraints on access to education or health services because of transport difficulty or cost. 6.10 Availability of public transport in cities. 73 Appendix C. SSATP transport governance indicators INDICATOR 1: Clarity and distinction between mandates and responsibilities of MDAs in the transport sector Indicator Red Red/amber Amber/green Green Clarity of and dis- Mandates and re- Mandates and re- Mandates and re- Mandates and re- tinction between sponsibilities have sponsibilities have sponsibilities have sponsibilities are mandates and re- not been defined or been defined, but been defined, but clear, and there is sponsibilities of key are unclear. there is significant there is some lack of no overlap. MDAs in the sector overlap. clarity or overlap. 1. Indicator applicability: All subsectors; all MDAs 2. Example country: Kenya 3. Example subsector: Cross-sector 4. Example agency: Review across ministries or agencies 5. Year: 2010/11 6. Documentary source: Legislation or statutes, MDA websites, Kenya’s Inte grated Transport Policy 7. Finding or data: Review of institutional frameworks, assessment of overlap of responsibilities Situation Issue Ministry of Transport Has overall transport sector responsibility. Ministry of Roads and Public Works Has primary responsibility for roads. Ministry of Local Government Is responsible for unclassified roads . Difficult to balance interest of roads against competing transport modes. Kenya Roads Board Clear mandate for managing finance of highways (established 2000) Kenya National Highways Authority Clear mandate to manage national highways (established 2007) Kenya Urban Roads Authority Clear mandate to manage urban road network (established 2007) Kenya Rural Roads Authority Clear mandate to manage rural roads (established 2007) Kenya Civil Aviation Authority Regulation of air transport subsector—operations and air navigation Kenya Airports Authority Manages nine public airports (of total 156 public aerodromes) Kenya Ports Authority Management of Mombasa Port (plus several smaller ports) Kenya Wildlife Service Park roads 1Kenya Railways Corporation Lack of investment, lack of management and commercial autonomy 1Inland waterways Overlapping responsibilities among the Kenya Marine Authority, Kenya Ports Authority, Kenya Ferry Services Limited, and Kenya Railways Cor- poration for different aspects of inland waterways and ports 75 Transport governance indicators in Sub-Saharan Africa 8. RAG rating (baseline) Amber/green 9. Target (plus six months) Green 10. Examples of implementation options to achieve the target To achieve this target, the following actions would need to be complete within two to five years: What Improve policy planning capacity of MoT so that it could oversee entire sector Create department of transport within MoT to oversee road subsector To improve the influence of this indicator on governance in Kenya: Steps to be taken have been set out in Kenya’s National Transport Policy 2009. Implementa- tion of these steps will improve clarity and distinction of mandate:  Establish a directorate of transport.  Consolidate transport functions under one ministry and separating policy.  Make regulatory and service provision functions.  Enhance the role of the private sector in transport infrastructure development and management.  Integrate non-motorized and intermediate means of transport into the transport systems.  Consolidate urban public transport. INDICATOR 2: Coherence of transport sector policy and extent to which its priori- tization process is based on objective criteria Indicator Red Red/amber Amber/green Green Coherence of No transport sector Transport sector Transport sector Transport sector transport sector policy. policy exists and policy exists and policy exists, explic- policy and extent to OR identifies issues identifies issues itly linked to macro- which its prioritiza- Policy exists but across some subsec- across all subsec- economic policy tion process is does not identify tors, but not linked tors, but does not with issues identi- based on objective needs of subsectors. to macroeconomic prioritize them in an fied and prioritized criteria context. objective way. in an objective way. 1. Indicator applicability: Across sector 2. Example country: Zambia 3. Example subsector: Across subsectors 4. Example agency: Ministry of Transport, Works, Supply & Communications 5. Year: 2011 6. Documentary source Ministry, department, and agency websites 7. Findings/Data: 76 Appendix C. SSATP transport governance indicators The national transport policy was last issued in 2002. Many changes were made at that time, including the reform of the National Road Fund (from the previous National Road Board), and the Road Development Authority. However, currently there is a lack of an overall transport master plan, which has meant there is no overall strategic framework for prioriti- zation. Reports of political interference in project selection continue to create perceived gaps between policy, priority, and expenditure. The ministry responsible for transport—until September 2011 the Ministry of Communica- tions and Transport—was recently combined with the Ministry of Works and Supply. This provides an opportunity to rationalize the approach to trans-sector planning. At present, however, the transport sector policy exists and identifies issues across some subsectors, but it is not linked to the macroeconomic context. The RAG rating therefore is red/amber. 8. RAG rating (baseline): Red/amber 9. Target (2011/12): Amber/green 10. Examples of implementation options to achieve the target: What Who Improve integration of development plans across transport subsectors. MTWS&C Improve integration of development plans between poverty reduction MTWS&C/Ministry of strategy paper, medium-term expenditure framework, and transport Finance sector. Update national transport policy. Ministry of Transport Introduce objective criteria to policy prioritization and the project MTWS&C selection procedure in a transparent way. To improve the influence of this indicator on governance in Zambia: As above. 77 Transport governance indicators in Sub-Saharan Africa INDICATOR 3: Budget allocations based on reliable financial forecasts and aligned to priorities based on objective criteria Indicator Red Red/amber Amber/Green Green Budget allocations More than 50 per- Less than 50 per- Less than 20 per- Sector financial based on reliable cent difference cent difference cent difference ceilings and budget financial forecasts between sector between sector between sector allocations are and aligned to financial ceilings financial ceilings financial ceilings consistent (less priorities based on and actual budget and actual budget and actual budget than 10 percent objective criteria allocation allocation. allocation. difference). (amount). OR AND AND OR Budget allocation Budget allocation There is little dis- Budget allocation by priority (based by priority (based crepancy (less than by priority (based on top five projects on top fove pro- 20 percent) be- on top five projects by value) is signifi- jects by value) is tween sector strat- by value) is so dif- cantly different not significantly egy and budget ferent from sector (more than 50 per- different (less than allocation in terms strategy that a cent) from sector 50 percent) from of priorities (based comparison is not strategies. sector strategies. on top five projects possible. by value). 1. Indicator applicability: Across sector 2. Example country: Zambia 3. Example subsector: Whole sector 4. Example agency: Road Development Agency (RDA), National Road Fund Agency 5. Year: 2010 6. Documentary source: Published budget, published RDA expenditure re- ports, road fund reports 7. Findings/data: According to the auditor general’s report, 24 there was a significant commitment by the RDA between 2006 and 2009. The document reports that in 2008 the RDA committed the gov- ernment to contracts totaling K1.643 trillion, despite having a total projected budget of K 1.200 trillion (K 685 billion and K 515 billion from the Zambian government and donors, respectively). This resulted in an over commitment of K 443 billion. This over commitment, too, was based itself on initial and unrealistic budget expectations, particularly for donor funding—see table below—with a total final expenditure of K 842.42 billion, or approxi- mately 50 percent of the RDA’s original commitments. 24 All data in this section were taken from Auditor General’s Office, Republic of Zambia (2009, 7–8). 78 Appendix C. SSATP transport governance indicators Funding source Budget (K, billions, Releases (K, billions, Expenditure (K, billions, 2008) 2008) 2008) Government 685 670.40 675.11 Donor 515 264.19 167.31 Total 1,200 934.59 842.42 In addition, a further five road projects totaling K 182.455 billion were procured outside the 2008 work plan; the authority to procure these projects not available for audit. The report itself suggests that these figures do not reflect a sufficient and statutory duty of care by the controlling officers responsible for planning and controlling the expenditure of public funds. 8. RAG rating (baseline): Red/amber 9. Target (2011/12): Amber/green 10. Examples of Implementation options to achieve the target: What Review the appropriateness of procedures of spending agencies, those entities providing funding, and the control mechanisms governing them. Strengthen capacity where it is needed, in several agencies. Increase awareness of importance of routine, regular, and timely checks. INDICATOR 4: Quality and use of key performance and value for money indicators Indicator Red Red/amber Amber/green Green Quality and use of No KPIs have been KPIs developed with KPIs (including VFM KPIs (including VFM key performance developed for the realistic targets, but indicators) devel- indicators) devel- (KPI) and value for sector. not monitored or oped, but less than oped and over 50 money (VFM) indica- OR reported. 50 percent moni- percent monitored tors Some KPIs developed OR tored and reported. and reported. but with no targets. KPIs do not include value for money indicators. 1. Indicator applicability: All subsectors; all MDAs 2. Example country: Tanzania 3. Example subsector: Road 79 Transport governance indicators in Sub-Saharan Africa 4. Example agency: All main sector ministries or agencies 25 5. Year: 2010/11 6. Documentary source: Individual ministry or agency reports presented at 5th Annual Joint Infrastructure Sector Review (JISR) meeting ( October 31–November 1, 2011) 7. Findings/data: Based on data presented at the recent 5th Annual Joint Infrastructure Sector Review, particularly those set out in the �5th Joint Infrastructure Sector Review 2011: Policy Analysis Paper� (Ministry of Transport, Republic of Tanzania 2011), it is apparent that performance indicators had been developed and approved across subsectors. The JISR report also highlights recent European Union support of a functioning transport sector monitoring system through improved fiber optics and installation of a database. Although the report states that the database is available online at the Minis- try of Transport website, it was not accessible for review for this SSATP study. Nevertheless, progress appears evident in this area. Indeed, a recent study supports a move by implementing agencies to set realistic targets for indicators that will be used for assessing subsector performance. This study is currently in draft form, and thus only in later years will evidence of improved indicators targeting emerge. 8. RAG rating (baseline): Red/amber 9. Target (6 months): Green 10. Examples of implementation options to achieve the target: To achieve this target, the following actions would need to be complete within 12 months: What Who Highlight the most effective examples of reporting as an example to others. Government Lobby less effective reporting entities to ensure full sector coverage. Media, civil society Seek to institutionalize the collection of data in all key organizations. Across MDAs Seek to incorporate data collection processes within financial and management infor- Across MDAs mation systems. To improve the influence of this indicator on governance in Tanzania:  Continue to publish performance criteria and results on an annual basis. 25 Ministries represented: Works and Transport. Agencies: TANROADS (Tanzania National Roads Agency) , TAZARA (Tanzania-Zambia Railway Authority), RALG (Regional Adimin- istration and Local Government), RAHCO (Railway Holding Company), SUMATRA (Surface and Marine Transport Regulatory Authority), and Road Fund. 80 Appendix C. SSATP transport governance indicators  Routinely make results available on ministry or agency websites; promote media coverage to alert public.  Continue cross-sector conferences to disseminate and discuss results.  Continue to review how effective indicators are in measuring the key areas of per- formance.  Feed results into medium- or long-term sector planning and prioritization pro- cesses and public sector management monitoring frameworks. INDICATOR 5: Comprehensive and timely public disclosure of transport sector procurement plans (including bidding opportunities, contract awards, and data on resolution of procurement complaints) Indicator Red Red/amber Amber/green Green Comprehensive and Government lacks a At least two of the At least three of the All key procure- timely public disclo- system to generate key procurement key procurement ment information sure of transport substantial and relia- information ele- information ele- elements are com- sector procurement ble coverage of key ments are complete ments are complete plete and reliable plans (including procurement infor- and reliable for gov- and reliable for gov- for government bidding opportuni- mation. ernment units repre- ernment units repre- units (90 percent of ties, contract awards, OR senting 50 percent of senting 75 percent of procurement oper- and data on resolu- Government does procurement opera- procurement opera- ations (by value . tion of procurement not systematically tions (by value). tions (by value). AND complaints) make key procure- AND AND Elements are made ment information Elements are made Elements are made available to the available to the pub- available to the pub- available to the pub- public in a timely lic. lic through the ap- lic in a timely manner manner and propriate means. through the appro- through the appro- priate means. priate means. 1. Indicator applicability: Whole sector 2. Example country: Mali 3. Example subsector: not available 4. Example agency: Ministry of Economy and Finances of Mali 5. Year: 2011 6. Documentary source: “ Évaluation de la gestion des finances publiques au Mali selon la méthodologie PEFA,� ECORYS. 7. Findings/data: A report produced in June 2011 for the Ministry of Economy and Finances of Mali (ECORYS 2011) focused on shortcomings in the settlement of complaints and disputes to establish a Public Expenditure and Financial Accountability (PEFA) 26 rating—against 26 The goals of the Public Expenditure and Financial Accountability program are to strengthen the ability of partner countries and donor agencies to (1) assess the condition of a country’s 81 Transport governance indicators in Sub-Saharan Africa the backdrop of the establishment of a Dispute Resolution Committee in March 2010 (Decision No. 002/ARMDS-CR). Documents provided for the reporting exercise did not indicate the consequences of complaints. Overall, the rating was red/amber because “a mechanism recording and processing claims relating to the process of procurement is in place but its design is poor and it does not work in a manner allowing a timely reso- lution of claims.� However, it was noted that the creation in 2008 of the ARMDS (Auto- rité de Régulation des Marchés Publics et des Délégations de Service Public) represented a major advance and offered real potential to reduce fiduciary risk. 8. RAG rating (baseline): Red/amber 9. Target (2011/12): Amber/green 10. Examples of implementation options to achieve the target: What Who PEFA self-assessment Procurement authority Verification of PEFA findings Office of the Auditor General Publish information in more accessible ways Local councils Widen dissemination Spending agencies and newspapers To improve the influence of this indicator on governance in Mali:  Ensure that there is external capacity to review performance. public expenditure, procurement, and financial accountability systems, and (2) develop a prac- tical sequence of reform and capacity-building actions. PEFA is a World Bank initiative. 82 Appendix C. SSATP transport governance indicators INDICATOR 6: Comprehensive time and cost reports on work progress against major transport sector projects disclosed to the public in a timely and accessible manner Indicator Red Red/amber Amber/green Green Comprehensive time Information does not Information does not Information provided; Information provided and cost reports on provide sufficient provide sufficient information on budg- in appropriate level of progress of work detail for analysis; detail for analysis; et variations or actual detail; expenditure against major (top 10) information on budg- information on budg- expenditure incom- information provided transport sector pro- et variations or actual et variations or actual plete. in comparable level of jects disclosed to the expenditure not dis- expenditure incom- detail. public in a timely and closed. plete or in a non- accessible manner compatible format. 1. Indicator applicability: All subsectors; implementing MDAs 2. Example country: Tanzania 3. Example subsector: Road 4. Example agency: Tanzania National Roads Agency (TANROADS) 5. Year: 2010/11 6. Documentary source: TANROADS paper for 5th Annual Joint Infrastructure Sector Review meeting 7. Findings/data: Project Value debut date Original Revised % overrun Comments (T Sh, millions)a completion completionb Mbwemkuru–Mingoyo 118,171 25.02.2003 24.10.2005 10.12.2007 81 Rectification of defects design and build (95 km) Outstanding work, weigh- 354,067 15.10.2007 14.10.2008 n/a > 100 Work suspended due to bridge, Mtukula (to date) delayed payments Rehabilitation, TANZAM 68,900,504 19.09.2008 19.09.2011 n/a 33 (to date) Work delayed due to highway (149.6 km) equipment failure Upgrade, Katesh–Dareda 21,666,491,107 11.03.2009 10.12.2011 n/a 3 (to date) Work delayed by weather Upgrade, Tanga– 3,279557,069 22.10.2010 12.08.2012 n/a 18 (to date) Poor use of equipment Horohoro Upgrade, Songea– 2,555,141,115 14.06.2010 03.09.2012 n/a 11 (to date) Delayed mobilization of Namtumbo staff Upgrade, Peramiho– Poor site management. 3,749,877,445 09.08.2010 28.10.2012 n/a 7 (to date) Mbinga Upgrade, Resurvey and redesign 3,472,297,225 11.10.2010 07.09.2012 n/a 13 (to date) Tunduma–Ikana required. Resurvey and redesign Upgrade, Ikana– Laela US$11,137,558 04.10.2010 15.08.2012 n/a 18 (to date) required. Delayed mobilization of Upgrade, Laela– staff by the contractor US$20,310,898 01.06.2010 31.01.2013 n/a 0 (to date) Sumbawanga and delayed relocation of utilities by the employer. a. Or U.S. dollars are indicated. b. It is not clear whether the projects are completed. (See note below on improvement suggestions. If this information was available, it is likely that the indicator would be rated amber/red or red). 83 Transport governance indicators in Sub-Saharan Africa 8. RAG rating (baseline): Amber/green—but see note b to table. 9. Target (six months): Green 10. Examples of Implementation options to achieve the target: What Who Tighten assessment of contractor capacity and track record. Procurement authority Base contract awards on money available and not expected. Awarding authority Strengthen role of engineer’s survey in approval of designs. Awarding authority Reduce time between preliminary design and design/supervision contract. Awarding authority Better apply International Federation of Consulting Engineers (or a similar) approach Awarding authority so contracts are managed more effectively from start. To improve the influence of this indicator on governance in Tanzania: The indicator should cover only completed projects. To do this, a progress report must rec- ord whether a project is completed. INDICATOR 7: Transport sector agency has established benchmarks for routine and periodic maintenance of assets and allocates the budget ac- cordingly. Indicator Red Red/amber Amber/green Green Transport sector Benchmarks not Benchmarks estab- Benchmarks estab- Benchmarks estab- agency has estab- established. lished, but routine lished and signifi- lished and > than lished benchmarks OR and periodic cant evidence of 80% of mainte- for routine and No link established maintenance not prioritization (> nance needs met or periodic mainte- between bench- prioritized (less than than 50% of routine > than 8 % of nance of assets and marks and alloca- 50% of needs met & periodic mainte- budget allocated to allocates the budg- tion decisions. using less than 50% nance needs met or maintenance. et accordingly. of total budget). > than 50% of total budget allocated to maintenance). 1. Indicator applicability: Roads, rail subsectors 2. Example country: Tanzania 3. Example subsector: road 4. Example agency: Tanzania National Roads Agency (TANROADS) 5. Year: 2010/11 6. Documentary source: TANROADS paper for 5th Annual Joint Infrastructure Sector Review meeting 7. Findings/data: The periodic maintenance needs of the paved roads for the next five years beginning with FY 2011/12 has been determined technically using HDM-4 program analysis un- 84 Appendix C. SSATP transport governance indicators dertaken by TANROADS in April 2011. The annual financial requirement is some T Sh 88 billion for paved roads. When this amount is compared with the allocated amount of TSh 52 billion for periodic maintenance of paved roads in FY 2011/12, the needs covered only 59 percent. The financing gap between maintenance needs and mainte- nance spending overall, however, fell from 65 percent in 2000/01 to 45 percent in 2010/11 (Ministry of Transport, Republic of Tanzania 2011). Although the coverage of routine maintenance dropped from 82 percent in 2008/09 to 69 percent in 2010/11 (this is explained by the expansion in the road network managed by TANROADS), peri- odic maintenance needs held at a steadier level, at about 58 percent. In line with this situation, the proportion of roads in good or fair condition has fallen. There are also discrepancies between allocated budget and expenditure: Category FY2008/09∗ FY2009/10∗ FY2010/11∗ Roads fund maintenance budget 147,205 177,462 177,462 Maintenance, actual expenditure 134,821 146,043 152,117 Difference –8% –18% –14% ∗ In T Sh, millions The backlog for paved road maintenance is estimated at T Sh 442 billion. A plan is in place to carry this out over the five-year period. The Roads Fund Board recognizes that it faces a number of issues, the most pressing of which is the insufficient funds to meet the huge backlog of maintenance because of the amount deferred. Other issues in- clude the inadequate capacity of agencies and some contractors to deliver quality road maintenance and the expanding network as the government increases the upgrading and building of new roads. The strategic plan for 2011–14 includes increasing funds and raising efficiency through better monitoring of works and curbing overloading as key objectives. 8. RAG rating (baseline): Amber/green 9. Target (2011/12): Green 10. Examples of implementation options to achieve the target: What Who Improve ability to demonstrate potential outcomes of a range of Road Fund Board policy decisions based on benchmarks and allocation options. 85 Transport governance indicators in Sub-Saharan Africa INDICATOR 8: Transparency and timeliness of annual budget and expenditure disclosures Indicator Red Red/amber Amber/green Green Transparency and time- Budget information Budget information Budget information Budget information liness of annual budget does not provide suffi- does not provide suffi- provided; information provided in appropri- and expenditure dis- cient detail for analysis, cient detail for analysis; on actual expenditure ate level of detail; ex- closures or information on actu- information on actual incomplete. penditure information al expenditure not expenditure incom- provided in compara- disclosed. plete or in non- ble level of detail. compatible format. 1. Indicator applicability: Whole sector 2. Example country: Zambia 3. Example subsector: National 4. Example agency: Government of Zambia 5. Year: 2010 6. Documentary source: Budget execution and service delivery barometer; Civil Society for Poverty Reduction (CSPR), Zambia (July –December 2010); two-year pilot project 7. Findings/data: CSPR has introduced a budget execution and service delivery barometer that bases its as- sessment on government administrative data. Findings include:  There is generally a lack of strong structures on the ground to involve citizens or in- form citizens about the release of funds and the available resources that could ena- ble them to monitor how these funds are executed.  The media outlets selected to publicize disbursements such as newspapers are per- ceived to be ineffective. Many poor people do not have money to purchase news- papers, and most notifications about disbursements are not broadcast on radio.  When citizens do receive this information, there is no evidence that they understand it.  According to the 2008 Procurement Act and the Budget and Finance Act, the gov- ernment of Zambia procurement plans and contract award information must be published in the public media (Daily Mail, Post, and Times of Zambia) in the interest of fiscal transparency and accountability. However, there has been little conformity to publicizing plans and contract awards.  The government publishes the Yellow Book, but many people do not have access— nor is the Appropriation Act published. 8. RAG rating (baseline): Red 9. Target (2011/12): Amber/green 10. Examples of implementation options to achieve the target: 86 Appendix C. SSATP transport governance indicators What Who Encourage government to use places where people gather (e.g., courts, churches) Government to advertise disbursements. Review Uganda’s policy on publication of financial disbursements, which has been Ministry of Finance cited as good practice. Push forward plans to create a budget act that would allow for meaningful consul- Ministry of Finance tation on the budget-making process. Speed up progress in implementing the Integrated Financial Management Infor- Ministry of Finance mation System (IFMIS) as a component of the Public Expenditure Management and Financial Accountability initiative (PEMFA). Encourage local councils to publish their reports accounting for local resources. Local councils INDICATOR 9: Rules applied to the membership and appointment process for key transport sector governance boards Indicator Red Red/amber Amber/green Green Rules applied to the Minority private Majority private Majority private Majority private membership and sector representation sector representation sector representation sector representation appointment process on oversight boards on oversight boards on oversight boards; on oversight boards for key transport and appointed di- but appointed direct- appointed through and clear, transpar- sector governance rectly. ly (e.g., by president competitive pro- ent, competitive boards or minister). cess—but process meritocratic ap- lacks transparency. pointment process for members. 1. Indicator applicability: All subsectors; agencies 2. Example country: Tanzania 3. Example subsector: Road 4. Example agency: Roads Fund Board 5. Year: 2010/11 6. Documentary source: Roads Fund Board: “Corporate Information: What It Is, What It Does� (August 2010). 7. Findings/data: Membership and appointment arrangements: Chair appointed by president from persons outside the public service Senior public servant appointed by roads minister Four road user representatives appointed by the roads minister from  Road transport industry  Tourism industry  Tanzania Chamber of Commerce, Industries and Agriculture  Confederation of Tanzania Industries  National Cooperatives Organization 87 Transport governance indicators in Sub-Saharan Africa Any other organization of road users with no potential conflict of interest. Nominations called for by the roads minister, list published, and objections invited. 8. RAG rating (baseline): Red 9. Target (2011/12): Green 10. Examples of implementation options to achieve the target: What Who Changes required to the Road and Fuel Tolls Act Cap 220 (dating from January 1, 1986, but Roads Fund Board incorporating all amendments up to November 30, 2006)—Act No. 11 of 1998, Part III, The Attorney general Board and the Roads Fund Manager (see also Establishment of the Board Acts Nos. 6 of 1998 s. Parliament 2; 11 of 1998 s. 3, and 14 of 2011 s. 24) Good practice guidelines shared and standardized approach adopted across SSATP countries. SSATP INDICATOR 10: Percentage of recommendations from independent technical and financial auditor reports implemented within one year Indicator Red Red/amber Amber/green Green Percentage of recom- Recommendations not Technical and financial Technical and financial Technical and financial mendations from in- published nor any audits published but audits published, but audits published; full dependent technical details of any follow- no details of any fol- incomplete infor- information on over and financial auditor up actions. low-up actions. mation on follow-up half of follow-up ac- reports implemented OR actions published. tions published; over within one year Follow-up actions OR 75 percent of recom- published but less than Only between 50 and mended actions im- 50 percent imple- 75 percent of recom- plemented. mented. mended actions im- plemented. 1. Indicator applicability: All subsectors; all MDAs 2. Example country: Zambia 3. Example subsector: Road 4. Example agency: Road Development Agency (RDA) 5. Year: 2010/11 6. Documentary source: “Report of the Auditor General on Road Develop ment Agency 2010� 7. Findings/data: The report recommended actions in response to 51 findings. By November 2011, 34 of the actions (66 percent) had been implemented. 8. RAG rating (baseline): Amber/green 9. Target (six months): Green 10. Examples of implementation options to achieve the target: 88 Appendix C. SSATP transport governance indicators To achieve this target, five of the following actions would need to be complete within six months: What Who Develop a whistle blower policy and submit it to the RDA board of direc- RDA board tors for possible adoption and implementation. Harmonize provisions in the Public Finance Act and the Public Procure- Ministry of Finance and National Planning ment Act on multilayer contracts; they should also address linkages be- tween medium-term expenditure framework annual budgets and pro- curement plans. Establish a consolidated cash flow that enables identification of variation Short-term technical assistance support to orders or failure to adhere to contract requirements with a view toward RDA seeking reimbursements from contractors. Institute an annual work program adjustment based on review of contract RDA board status and commitments. Encourage RDA board to lobby government to remove the 10 percent RDA board capping on local resources in order to provide adequate operational funds to the three agencies. Address lack of proper handover of assets: RDA to ensure that the items RDA management purchased on contracts are handed over to the RDA by the contractor before the retention is released and the completion certificate issued. All vehicles and other assets purchased on projects will immediately be reg- istered in RDA’s name. The assets received will be added to the agency’s Register of Assets and Books of Accounts. Road matrix dissemination strategy: Develop a dissemination strategy Secretary of Treasury and ensure regular updates of the implementation of the action plan in the press at least twice a year to demonstrate the government’s efforts to be accountable in a transparent manner to citizens. Transport policy: Update the 2002 transport policy and the 2003 Letter of Public servant, Ministry of Communications Road Sector Policy to allow the policy and the strategy framework to take and Transport stock of high-level commitments. Prepare a transport master plan that includes all subsectors. Public servant, Ministry of Communications and Transport 1Review the Road Sector Management Plan of September 2009 during Ministry of Communications and Transport the next Joint Donor Forum. To improve the influence of this indicator on governance in Zambia: 1. Prioritize the findings and associated recommended actions of the Office of the Auditor General (high-medium-low importance) to enable a more strategic ap- proach to implementation. 2. Recommend this approach to subordinate auditing bodies. 89 Bibliography Auditor General’s Office, Republic of Zambia. 2009. “Report of the Auditor General on the Road Development Agency for the Period January 2006 to September 2009.� Lusaka. Bertelsmann Stifttung. 2009. BTI 2010—Kenya Country Report. Gütersloh. Germany. Bratton, Michael. 2010. “Survey of Citizens’ Perceptions of Local Government Re- sponsiveness in Sub-Saharan Africa.� Working Paper 119, Afrobarometer. Castillo. 2009. “Understanding, Measuring, and Stopping Corruption in Construc- tion: A Foundation Paper and Case Studies.� GHK Working Paper. CSPR (Civil Society for Poverty Reduction) Zambia. 2010. “Budget Execution and Service Delivery Barometer, July to December 2010.� Lusaka. DAC (Development Assistance Committee). 2006. DAC Guidelines and Reference Series: Good Practice Guidance for Development Cooperation. Paris: Organisation for Economic Co-operation and Development. ECORYS. 2011. “Evaluation de la gestion des finances publiques au Mali selon la methodologie PEFA.� ECORYS, Rotterdam, June. Freedom House. 2011. Freedom in the World 2011: The Authoritarian Challenge to Democracy Washington, D.C: Freedom House. Gwilliam, Ken, et al. 2008. “The Burden of Maintenance: Roads in Sub-Saharan Afri- ca.� Africa Infrastructure Country Diagnostic, Background Paper 14 (Phase 1), World Bank, Washington, D.C. Heidenhof, Guenter, and Doris Voorbraak, Team Leaders. 2008. “Rapport No. 44151-ML: Diagnostic sur la Gouvernance dans les Secteurs du Transport et de l’Energie au Mali.� Draft, Africa Public Sector Reform and Governance Unit, World Bank, Washington, D.C, June 2008. IMF (International Monetary Fund). 2007. Manual on Fiscal Transparency. Washing- ton, D.C. 91 Transport governance indicators in Sub-Saharan Africa ITAD. 2011. Governance and Conflict Indicators Report. http://www.dfid.gov.uk/r4d/PDF/Outputs/Mis_SPC/60797_ITAD-Gov-Conflict- Indicator-Rpt-Jan11.pdf. Heritage Foundation and Wall Street Journal. 2011. Index of Economic Freedom. http://www.heritage.org/index/. Knack, Stephen, et al. 2002. “Second Generation Governance Indicators.� MPRA Pa- per 25029, University Library of Munich, Germany (revised 2003). Messick, Richard. 2011. “Curbing Fraud, Corruption, and Collusion in the Roads Sector.� World Bank, Washington, D.C. Le, Tuan Minh, et al. 2010. “Public Investment Management in Zambia: Diagnostic and Recommendations.� Ministry of Communications and Transport, Republic of Zambia. 2002. “Transport Policy.� Lusaka. ______, 2011. “Road Sector Investment Programme II, 2004–2013, Bankable Docu- ment—Addendum 2.� Lusaka. Ministry of Finance, Republic of Zambia. 2010. “Medium Term Expenditure Frame- work 2010/11–2012/13.� Lusaka. Ministry of Finance and Economic Affairs, United Republic of Tanzania. 2005. “Na- tional Strategy for Growth and Reduction of Poverty (�Mkukuta�) 2005–2010.� Do- doma. ______, 2010. “National Strategy for Growth and Reduction of Poverty II.� Draft, Dodoma. Ministry of State, National Planning and Vision, Republic of Kenya. 2030. 2011. “Kenya Vision 2030: First Medium Term Update.� Nairobi. Ministry of Transport, Republic of Kenya. 2009. “Integrated National Transport Poli- cy: Moving a Working Nation.� Nairobi. Ministry of Transport, United Republic of Tanzania. 2003. “National Transport Poli- cy 2003.� Dodoma. 92 Bibliography ______, 2007. “Transport Sector Investment Programme (TSIP).� Dodoma. ______. 2011. “Fifth Annual Joint Infrastructure Sector Review Meeting, 31st Octo- ber – 1st November 2011 (Papers).� Dodoma. NEA Transport Research and Training. 2004. “Zambia—Trade and Transport Facili- tation Audit.� Paper produced for the World Bank, Washington, DC. Republic of Kenya. 2004. “Investment Promotion Act 2004.� Nairobi. Republic of Zambia. 2009.�Report of the Auditor General on the Road Development Agency for the Period January 2006 to September 2009.� Lusaka. Sub-Saharan Africa Transport Policy Program (SSATP). 2003. “Membership Com- mitment Statement.� SSATP, World Bank, Washington, D.C. ______. 2008. “Second Development Plan (2008–2011).� SSATP, World Bank, Wash- ington, D.C. TANROADS (Tanzania National Roads Agency). October, 2011.� TANROADS Paper for Fifth Infrastructure Sector Review Meeting (Roads Sub-sector).� Thomas, M. A. 2007. “What Do the Worldwide Governance Indicators Measure?� John Hopkins University, Baltimore. Transparency International. 2011a. “Corruption Perceptions Index.� Berlin. ______. 2011b. “The East Africa Bribery Index 2011.� Berlin. USAID (U.S. Agency for International Development). 2008. “Improved Road Transport Governance Initiative on Interstate Trade Corridors.� Improved Road Transport Governance Initiative. Washington, D.C. ______. 2009. “NGO Sustainability Index.� Washington, D.C. USAID/UEMOA (West African Economic and Monetary Union). 2011. “Improved Road Transport Governance Report.� Improved Road Transport Governance Initia- tive. USAID, Washington, D.C. World Bank, 1989, Sub-Saharan Africa: From Crisis to Sustainable Growth. Washing- ton, D.C. 93 Transport governance indicators in Sub-Saharan Africa ______. 2007a. The Governance and Anti-Corruption Strategy, Washington, D.C World Bank. ______. 2007b. Strengthening World Bank Engagement on Governance and Anticorrup- tion. Joint Ministerial Committee of the Board of Governors of the Bank on the Transfer of Real Resources to the Developing World. Washington, DC: World Bank. ______. 2011. Doing Business Index 2012, Washington, DC: World Bank. Whitworth, Alan. 2010. “Zambian Fiscal Performance, 2002–2010.� 94