22406 view point Utility Benchmarking Bill Kingdon and Vijay Public Reporting of Service Performance Jagannathan In some countries regulators routinely publish indicators of utility Bill Kingdom isaSenior service performance through the local media. Exposing the "worst in Water and Sanitation Specialist at the World class" has proven to be a powerful way of pressuring utilities to Bank. He ismanager of provide better services to consumers. By focusing political attention the World Bank's water utility benchmarking on service quality, benchmarking can also help to shield regulators programn. Vijay from political interference. This Note reviews the requirements for Jagannathan is a Lead Water and Sanitation effective benchmarking: choosing indicators that are unambiguous Specialist in the East and verifiable, consistent with long-term incentives for good Asian Region of the performance, and easy for the public to understand. WVorld Bank. Benchmarking empowers a broad section of Scorecard for water and sewerage civil society to ask why one service provider has companies in England and Wales, achieved demonstrably better performance Figure 1999-2000 than another, or why some companies choose to 1 0 50 100 150 200 250 300 350 400 ignore effluent discharge standards. Mobilizing I Anglian consumers in this way is likely to lead to Northumbrian demands on the regulated utilities-whether private or public-to improve performance. Performance benchmarking has become Southern standard practice in the regulated utilities of Severn Trent England and Wales-with considerable success. Thames There the water and sewerage companies pro- South West vide the regulator, Ofwat, with indicators of ser- Yorkshire vice performance covering water supply (water pressure, hosepipe bans, and quality of drinking Dvir tymru water), sewerage service, customer service (num- North West ber of complaints), and environmental impact Note: The scorecard is a composie index of performance far water, sewerage, and (leakage, pollution incidents). Ofwat publishes Sourcerw s. the indicators annuallv' in an easy-to-digest Stcrm: Ofwat (wwew.ofweat.goe.ukdpdffilesaso2tf5,pdf. UTILITY BENCHMARKING PUBLIC REPORTING OF SERVICE PERfORMANCE format accompanied by commentary. Figure 1 that enhance specific agency interests-such illustrates Ofivat's total "score" for water supply, as new investment programs-rather than sewerage service, and customer-service perform- those that will benefit consumers. ance by U.K. water and sewerage companies. * Low consumer expectations. In many countries These simple performance scorecards have basic services such as water supply, sanita- helped to pressure the "worst in the class" to tion, and rural electricity have been pro- improve their game and allowed the "best in vided at subsidized prices, and when class" to gain reputation. consumers are not charged economic costs 2 Some developing countries have adopted per- for services, their tolerance for inefficiency formance benchmarking techniques for sinmilar and poor service increases-"You get what purposes-and with similar results. For examiiple, you pay for!" Only when services are billed in Sao Paulo, since the introduction of pollution at their true economic value do consumiler s tests and public reporting of results in 1991, 95 begin responding to differentials in service percent of the industries that had been pumping quality. untreated effluents into the river have installed * Power/iil vested interests. In most counltries the waste treatment units to avoid paying fines and commercial and indtustrial elite have strotig seeing their names published. Indonesia's envi- lobbying powers. Those powers will be used ronmental regulatory agency, BAPPEDAL, has to resist any public reporting system that developed a simple, color-coded rating systen to threatens to expose poor-even illegal- monitor industry compliance with national envi- practices and lead to increased costs and ronmental standards. Publication of the ratings adverse public sentiment. has brought about spectacular improvements in C fionflicts of interest. Manly monopoly service pollution abatement (Woorld Bank 1999). providers are either public agencies or parts of Procuraduria Federal de Proteccion al local goverlnment. They not only provide but Ambiente, Mexico's environmental enforce- also regulate services-supposed(ly in the best ment agency, announced last year that it would interests of the consumer. Publicly reporting begin to publish information on the environ- the agency's performance therefore directly mental performance of 3,000 industries to pro- reflects on the action, or inaction, of the local vide incentives to reduce contamination. Ratings or national government. So there is a tendencv will inclu(le recommendations on how to achieve to cover ulp poor performance. acceptable environmental performance. * Culture of underachievement. Ideally, pointing the finger at poor performance should be a Barriers wake tip call for an institutioni to do better. The power of public reporting seems obvious. In many developing countries, however, So why is it not more widespread? Many of the poor performance is used by institutions to reasons stem from the legacy of decades of pub- justify demands for more resources. Rather lic sector operation of utilities, or from the pres- than promoting imiproved performance, ence of vested interests antagonistic to public public reporting can open a floodgate of reporting of performance. demands from under-achievers. * Insufficient data. The performance of public * Inconsistencies in policy framnework. The sector sernce providers is typically not closely problems of consumer apathy, conflicts of monitored by national policymakers and interest, and other organizational issues donor agencies in a format that identifies the sometimes arise because the basic rtles of a best in the class or the underachievers. Data sector are inconsistent with rational eco- are more likely to reflect either broad policy- nomic principles. In such cases utilities may oriented themes or specific information use- not be able to do much on their own to lift ful for public investment purposes. Sectoral performance. The policy framework within agencies may be unwilling to report or which the sector operates must be reformed record information on the extent of under- so that pricing, investment, financing, and achievement, preferring to focus on actions operational rules will combine to induce pro- clucers to provide services that custollmers Annual performance measures for wvant, are willing to pay for, and arc sustain- three water utilities in Melbourne, Box Australia able in the long term. 1 Water Quality Developing the scorecard * Coliform sampling and testing. The basis of an effective performance reporting * Water quality complaints. syste no is good data, the right indicators, clear * Sewage treatment compliance. presentation, and credible public debate. * Reuse of effluent and biosolids. Choose the right indicators Reliability of Service Measuirable and meanringfuil indicators of per- . Interruptions to, and restoration of, supply. *Unaccounted-for water. formciance should reflect obvious feattores of the * 5ewer water. product or service so as to allow customers to * Sewer blockages and spills. untlerstanid variations in service perfornmanice Response times to emergency phone calls. between different entities and over time. Affordability Indicators may foctLs on quality. efficiency, * Use of grant relief scheme. affordability, or comparative performance. * Restrictions for nonpayment. Some mav be expressed as indexes acdjusted for * Use of installment payment plans. dlifferent operating con(litions. * Legal actions. In NVictoria, Austr-alia, the Office of the Regulator-General (box 1) publishes annLal per- Customer Service Jormance restolts for the three regutlatedl water * Complaints. distriboution coimipanies in MelbouL-ne. The iiidi- toorce: Melbourne Retail Water and Sewerage tompanies Performance Report, cators include water qualitv (for example, tests January 2000, Off,ce of the Regulator-General Victoria, Australia. per 1,OOO coostoilelrs and the percentage of sam- ples meeting guoidelines). the number of inter- ruptions in service, amotont of water not resuolts. bhut in most countries newspapers, tele- accounlted for bv the utilit, response times to visiont. anti radio are likely to be more effective emergency calls, nutmber of complaints, antI in bringing the messages to the attentioni of the need for payment assistance. public. Indicators should draw on data that are reli- The sereice provider is not the only entity able. relativelv easy to collect (or of stoch impor- that will be scrutinized wheni perfoel-larace tance that they ought to be collected anyway), reports are isstoed. The bodv that prepar-es the antI not susceptible tom multiple interpretationis. report moist also be prepared for media atten- TlheT should reflect conditions over whiclh the tion. It miist be sto-e that its commilentarv is fair service providers have control. Ioidicators that and objective, and that it canl publiclv defelnd its offer an indlisputable basis forjoodgimnenit are not analysis if reqosired. easy to find. Even the process of measuring The ptoblicationi ofdata also may tempt some some indicators can lead to disputes. For exam- participating organizations to cheat on their pie, of the various methods of estimatinig water perfornmance rieports. Routinie or random inde- not accounIted for, whlich should be used? periclent atudits of the processes of compiling Performance results nmust be comml1uLnicated aiad rieportinig data may be required to solve the in a way, that will allow the ptublic to snake an cheatinig problem. informed assessment of relative performance and gauge realistically the extenit to which Who should compile and report results? improvements should be expected. Long lists of If a formal regLulatory agency already exists, it indicators are not necessary: theyt will be expen- should take the lead. Whiere the regulator is also sive to prepare and may obscure the message. the service provider, however, as in the case of Resuolts m.ay be publicized in many ways. The locally provided services, mesponsibility might Web is a convernient place to post performance fall to a specialized IJlUllicipal periformliianlce UTILITY BENCHMARKING PUBLIC REPORTING OF SERVICE PERFORMANCE monitoring nnit or to citizen groups. The their poor performance was due to factors organization responsible for data collection and beyond their control. At the other end of the reporting may change over time. A private spectrum "top of class" agencies shotild be organization might start an initiative that is later rewarded for their superior performance. adopted by government. Bangalore's non- viewpoint governmental Public Affairs Centre, for exam- Conclusion ple, produces a scorecard for performance of By exposing poor performance public reporting the city's public services. The group's presenta- makes service providers more accountable to the is an open forum to tions are discussed in well-attended town hall public and thus increases their motivation for en forumito meetings and followed up by the local media to improvement-to the benefit of the end user. encolicy ination or public policy innovations for pressure providers to improve services. The Governmnent and donor agencies can do more to private sector-led and Public Affairs Centre has now taken the initia- encourage public performance reporting and to market-based solutions for tive to extend its activities and benchmark the help guide the effective use of scarce resources. development. The views quality of basic services across 22 major states in "Worst in the class" performance must not be published are those of the India. seen as a rotute to additional resources but rather authors and should not be Consumer advocates and pressure groups as a clear challenge to do better. attributed to the World can play a powerftul role in mobilizing public Bank or any other affiliated opinion in response to published information. organizations. Nor do any of Independent analysis lends punch to the data. the conclusions represent The interinational donor commtnity can also Note official policy of the World help by compiling and presenting comparative 1. The Benchmarking Water anici Sanfitationi Bank or of its Executive performiance information. In the water sector Utilities Project facilitates the sharinig of cost and per- Directors or the countries the Asian Development Bank has published a forynance information between utilities ani between they represent. comprehensive assessment of utility perform- counitries by creatinig a network of linked Web sites, ance in its Second Water Uttilities Data Book through global partniership eftirts. Each Web site pies- To order additional copies (Mclntosh and Ynigtuez 1997). The WVorld Bank, ents saluies for a set of core cost and perforuiance idi- contact Suzanne Smith, working with many partners, recently launched cators for a utility, or utilities, in that partiCLnlar regioll or managing editor, an international benchmarking network for cottntrv. [http:/ /'wr.wworldbank,org/htmiil/fpdt/water/ Room 19-017, water and sanitation utilities on1 the Web.' topics/bench_network.htrnl] The World Bank, 1818 H Street, NW, Create the right incentives References Washington, DC 20433. The penalties and incentives of perfoirmance McIntoslh, Arthur- C. and Cesar E. Ynigtuez. 1997. reporting are clear enough in the case of private Serond I1'oler a Utilities Data Book. Manila: Asian Telephone: 00 1 202 458 728 1 providers: Bad press can affect stock prices and l)evelopment Bank. Fax: public perceptions, and good press can boost Officc of the Regulator-(General (Vi(ctor-ia, 001 202 522 3 181 market share and sometimes ease regulatory Atistralia). 2000. "Melbourne's Water anid Sewer-age Email: hterdens. But because of the institutional prob- Companies-Peiformnance Report, jttly 1998-Jine ssmith7@worldbank.org lems discussed earlier the picture is not so clear 1999." [vsasss'.reggeix.vic.gov.au/ docs'/water/wr(1121000. for the public sector. Certainly, doing well can pdtl. earti good ptiblicity. Doing badly, on the other World Bank. 1999. Greetntig IndustrD: NVezv Roles for Printed on recycled paper hand, can have several outtcomes, incltuding Communities, ilorkets, and Goverenments. Watshinigton, D.C. requests for more resources. In the enid the public sector must shoulder Bill Kingdom (tekiogrdom@u'Sorld6bank.org) and 1`i;oy the butrden of addressing how incentives and agsannatlhan (o/goagoannoilhatozo;ookl6blnk.org). penalties can best be arranged to enhance per- formance tising the results of public perform-n_ ance reports. How? "Bottom of the class" agencies seeking concessional financing from national government or international financial institutions could be requtired to prove that This Note is available online: www.worldbank.orglhtmilfpdinotesl