54810 No. 154 March 2000 Road Sector Performance Indicators for African Countries In developed and developing countries, road systems are central not only to the economic health of the nation, but also to the quality of the environment and, in general, the quality of life. Public awareness of this fact has forced road administrations to be accountable for road system performance. Despite the efforts of many countries, there has been no transparent measurement or comparable evaluation standards for road systems performance. But in 1996 The Organization for Economic Cooperation and Development (OECD) embarked on an international effort involving thirteen countries and created a Scientific Expert Group to develop a set of Performance Indicators to address this need. Concentrating only on issues related to road systems, they set out to create Performance Indicators that would be comprehensive­­holistic­­ and would help road administrations to evaluate themselves, respond more effectively to decision makers and constituents, and even allow some comparisons with peer countries. The complete OECD Expert Group Report, 'Performance Indicators for the Road Sector" was published in 1998. The Performance Indicators in this paper, although slightly different from those proposed by the OECD Group, were developed in consultation with area specialists with the intent that they would apply to African conditions and circumstances. They reflect the Comprehensive Development Framework. The Starting Point: Three Critical Questions Every road manager, even those working in seriously compromised transport systems, must ask three critical questions: Is the road administration doing the right things? Is the road administration doing things right? And, what things done by others, including international partners, affect the road system? These questions promote a holistic approach to the development of Performance Indicators. In addition, they assume that, though the road administration handles day to day data collection, analyses and operations, government officials (elected and appointed) and the public do in fact influence the decision making processes. They are ultimately responsible that the data collected and analyses undertaken reflect their concerns and affect the project selection and the actions performed. To assist road administration management directly the three questions were translated into three functional areas of inquiry: what is the conceptual mental model to frame Performance Indicators for adaptation and quantification in different country contexts? what are the most important indicators to allow both assessment as well as the development of creative approaches? And, finally, a question that the Group saw as the most important of all: what is the application of Performance Indicators so as to create a learning organization'? The Performance Indicators The array of possible and viable Performance Indicators is vast, and therefore they must be classified. The classification recognizes the fundamental roles of transport and reflects them by seven groups of objectives ? namely, mobility, accessibility, safety, environment, equity, community and economic development. The management of the dynamic, goal-seeking transport system entails three major activities ? program development, service delivery, and system operation. These dimensions of performance are seen from three perspectives: government (including stakeholders), the road administration itself, and the user/community. The Performance Indicators are broad enough to capture the essential concerns of the government, administration, users and community; flexible enough for use across different country contexts, and specific enough to be measured. Table 1 presents the Performance Indicators applicable in African countries, using the above classification. Data collection to establish a baseline of initial conditions is essential at the outset. Finally, "what gets measured, gets done"; therefore, useful indicators are those that are used. So for the Performance Indicators to be truly useful, they should be subject to criteria that they are measurable, understandable; timely; relevant, reflective of geographic scale, useful to management, and useful as a multiple indicator of goals, a measure of achievement, and a diagnostic tool. Table 1: Proposed Performance Indicators for African Roads Perspective Government Road Administration Road User Comments Dimension Ministry Accessibility 1. Average road 2. Road Network 1. Three part: Mobility user cost (car, producer price, tax truck, trailer and tariff for road truck) administration 2. Km/sq. km of arable land or population by region; separately for functionally classified (FC) and community roads. Safety 3. Accident risk: 4. Unprotected 3. No. of fatalities fatality and injury road user risk and injuries accidents/veh-km 4. Nonmotorized fatalities and injuries Environment 5. Environmental Policy 5. Yes or No; or Program elaboration required (e.g. phasing in of non-leaded fuel; treatment of polluting vehicles; etc.) Equity 6. Percentage of 7. Processes in 6. Or within 2 hrs. Community population within 10 km place for walking time. of a classified road customer/road user feedback 7. Yes or No; a method to obtain information of social benefits and costs. Program 8. Rolling multi- 10. Data bank for FC 8. Yes or No; Development year program roads elaboration for construction, required maintenance, 11. PMS system and operations distribution of funds by 9. By program region, functional class, (construction/ 9. Percent and for prioritizing and maint./oper.) completion of rehabilitation and annual work maintenance actions 10. Yes, or no: program elaboration required on data collection methods and updating. 11. Yes or No; elaboration of principles Program 12. Forecast values of 14. Percent of 12. By FC and Delivery road costs vs. the actual gravel roads program costs formed twice or (construction, more times a maintenance, 13. Percent of work operations). year done by direct labor and parastatals 13. A measure of competition Program 15. Value of 16. Paved road 20. No . road 15-17. By FC Performance assets roughness (IRI) closings and 18. Gravel roads 17. Bearing road closing days only capacity/deflection 20. Percent links 18. Thickness of gravel and percent time surface closed by FC 19. Defective bridge deck area Final Possible descriptors are: (1) population (urban/rural); (2) GDP; (3) vehicle fleet by Conditions type; (4) fleet without emission control; (5) current road administration budget by program ; (6) veh and ton km of travel and traffic volumes by FC (weighted by link length); (7) modal split for passenger and freight (all road modes); (8) congestion: weighted road-km with Volume/Capacity >1 by FC The Mental Model A Mental Model is a prerequisite for developing Performance Indicators. As developed in the OECD Report, a generic Mental Model for Road Systems unbundles the many interrelated variables that help performance of a road administration. This sets the stage for the Performance Indicators that can apply to all road systems, including those in Africa. The model presents a self-adapting transport system in which evolving goals and objectives reflect human values and needs. These are expressed and made tangible in the results based management format as desired outcomes and consequences, achieved through inputs (e.g., materials, money), processes (e.g., maintenance work, data collection) and outputs (e.g., passable roads, bridges). These, and unexpected outcomes, are tracked and evaluated using data collected by the road administration and feedback from the road users and community. The model's central idea is to move the focus of planning from the alleviation of current symptoms to creation of future results through processes. This incorporates evaluation of the goals and objectives in a feedback loop. Evaluation results are defined in the quality and quantity of the outputs (e.g., condition of the road system) and in user satisfaction with outcomes. As a management-by-results model, it provides a concrete framework for self- evaluation. The model illustrates how the road administration carries out activities using resources allocated to it. Products and services ­­ outputs and concomitant outputs ­­ are judged in relation to pre-determined standards and criteria. Outcomes, derivatives of products and services through user interface, are judged relative to requirements for the objectives that the road users and the community desire. Those judging outcomes include: government (including "stakeholders" ? the regulators and investors, the suppliers of goods and services, developers ? who voice their concerns through the "government"), road administration (the professional transport organization traditionally responsible for "system performance"), and user/customer ?the users and operators of transport services, and the community. Qperationalizing the Model: A Rolling Multi-year Plan The Mental Model for utilizing Performance Indicators is operationalized in a road program cycle or Rolling Multi-year Plan (Figure 1). The road administration management employs several means to develop this road program: learning, information gathering, development of new options, public participation, evaluation, interaction with the market, and allocating resources to these activities. The cycle is a continuous feedback process, in which `bottom up' technical practices combine with 'top down' management and most importantly, incorporation of input from government and the public. The Rolling Multi-year Plan in Figure 1 has Performance Indicators as the observing eye at center. It originates from individual and societal drives, and goals. These generate objectives that, augmented by policy guidance from the government and the appropriate political entities and the civil society, are reflected in performance indicators for the road system. The objectives are translated, again with policy guidance, into an (annual) road program whose key inputs and processes are evaluated with performance indicators. The delivery of the road program outputs and processes are monitored with the assistance of performance indicators. Finally, the outcomes of the road program, the program performance, are analyzed with the help of performance indicators. These outcomes co-evolve with influences from outside the road transport system and are observed with unspecified time lags and lead to complex societal consequences. These consequences in turn stimulate new goals and start a new cycle. Figure 1 Using Performance Indicators Effectively The use of Performance Indicators has multiple dimensions, going well beyond evaluating the degree to which goals and objectives have been achieved or attempting to identify those variables associated with the achievement of goals and objectives. The effective use of Performance Indicators helps road administrations to: · periodically evaluate road system goals and objectives; · develop alternative courses of action or means to achieve desired goals or avoid unintended ones; · evaluate the degree of achievement of goals and objectives; · assess the efficiency and effectiveness of alternatives and of the road administration; · as a guide for both program and project management; and development or a periodic re- evaluation of goals and objectives. There is an implicit sixth use: to assist road administrations to be learning organizations. The Learning Organization Performance assessment depends on values that are largely unmeasurable. While there is a long and useful tradition among transportation professionals of using mathematical models to both assess performance and chart change, it is often symbolic communication and actions that portray the situation. These communications lay a common ground for understanding circumstances and possibilities­­and the desirability­­of change. Acknowledging and encouraging this kind of communication and assessment is a most productive use of Performance Indicators. Road administrations can do this by becoming a 'learning organization'­­organizations in which reflection and inquiry are encouraged and in which the emotional resistances to the question 'why?' are dealt with. Often the question, "why do we do this?", "why do we do this, this way?", "why not try a new idea?" and so forth, create an array of real and important negative feelings. These emotional resistances often result in unintended side effects and negative outcomes in transport planning; dealing with them allows staff and management to cope better with future problems and function more maturely. Further, the resistances often prevent an organization from trying new approaches for developing, recommending and implementing new plans or policies that may require a departure from extant practice. Performance Indicators can act as the cornerstone for studying resistances on which learning experiences can be built. Figure 2: Conclusions The overall conclusion is that Performance Indicators which are selected after reflection and consultation over a period of time, do provide a useful broad-based portrait of the road sector from different perspectives. More specific conclusions are that: · The views of road users should be solicited and incorporated into the activities the road administrations­­even more extensively than is currently done. · Road administration professionals want their professional knowledge to be shared with the public, and this desire for the public to know is clearly reflected the road program cycle Performance Indicators. · Road administration professionals feel that more can and should be known about the issues reflected in the Performance Indicators. Lack of public knowledge in the areas of equity, community and economic development should be addressed by increased research efforts. · Last and perhaps most important, Performance Indicators can and should encourage an institution to learn. The OECD Report notes that no indicator tells one story and that the interpretation of Performance Indicators is an opportunity for investigation, learning, creativity and solid specific problem solving. Refinement of Performance Indicators for Application in African Transport Systems Currently, an International Field Test is underway to test the OECD Performance Indicators in the concrete. Measurement of those indicators not yet precisely defined will be refined in the Field Test. To be useful, the measurement of Performance Indicators should be adapted and made flexible for application in individual county contexts. It will therefore be desirable for African countries to create their own field test of Performance Indicators in Table 1, for refining indicators for their own specific use. A multi-day conference among African countries to determine technologies and techniques to measure appropriate Performance Indicators, comparable within their own country subgroups, would be beneficial and productive. This article was first published in March 1999 as SSATP Note No. 17 by UNECA and the World Bank. For more information, please contact Frusso@worlbank.org