66067 Final Report Results Based Management in Thailand mayb Country Development Partnership on Governance and Public Sector Reform Implementing Results Based Management in Thailand January 2011 The World Bank 1 CONTENTS I. Executive Summary ............................................................ 3 II. Background ........................................................................ 4 III. Results Based Management Systems ................................. 6 A. Framework 6 B. Lessons from international experience 9 C. Findings & Recommendations 14 IV. Linking Performance with the Budget & Planning .......... 22 A. Issues 22 B. Lessons from International Experience 25 C. Findings 35 V. Creating Incentives for Performance................................ 38 A. Issues 38 B. Lessons from international experience 40 C. Findings 45 VI. Summary Recommendations ........................................... 48 VII. Appendix ........................................................................ 51 A. Terms of reference for technical assistance 51 B. Workshop Program 53 C. Workshop Participants 54 D. Mission Schedule 57 E. Results Based Management Tools in Thailand (2010) 59 The World Bank 2 I. Executive Summary Thailand has been pursuing the implementation of results based management techniques in the public sector for over a decade. Leading this task is the Office of the Public Sector Development Commission (OPDC) which has been supporting various agencies and departments in undertaking a wide variety of results based management reforms, including key performance indicators, balanced scorecards, and individual performance bonuses, among others. This report summarizes the results of a two week fact finding mission and a one day workshop conducted by the Office of the Public Sector Development Commission (OPDC) of the Prime Minister‘s Office in collaboration with the World Bank‘s Public Sector Performance Global Expert Team (PSP GET) held on September 24, 2010 in Bangkok, Thailand. Prior to the workshop, the team held meetings with officials from the OPDC, the Bureau of the Budget (BOB), the Office of the Civil Service Commission (OCSC), National Economic and Social Development Board (NESDB), the Ministry of Interior (MOI), and others, to better understand the results based management system with a focus on the opportunities and challenges of implementing various aspects of the system. This report covers the main areas to consider in implementing a results based management system. The report provides an overview of Thailand‘s Results Based Management (RBM) system, with an emphasis on lessons from international experience in leveraging performance information to deliver results. Chapters on how to link performance with planning and budgeting, as well as an overview of incentives to improve performance are included. Each chapter is loosely structured in the following way:  Issues identified during discussions;  Relevant lessons from international experiences; and  Overall recommendations/findings of the PSP-GET. The key points emerging from this report include: 1. On the Results Based Management system – improving the coherence of the system with respect to the tools, indicators, and reports used. Confusion in the terminology underpinning RBM to over-production of central KPIs has resulted in unintended consequences for ministries, departments, and units. 2. On strategic budgeting and planning – further developing the supply (inputs) and demand/space for analysis of performance information, plans, and budgets. 3. On incentives for performance – rationalizing and applying a strategic approach to the use of both monetary and non-monetary incentives. The material in this report was compiled by a team led by William Dorotinsky, Sector Manager, Public Sector & Governance, in his role as leader of the Bank‘s Public Sector Performance Global Expert Team (PSP-GET). Other members of the team were: Joanna Watkins, Public Sector Specialist, and Ms. Miki Matsuura, Public Sector Specialist. Additional support was provided by the Public Sector Team in the Bangkok Office: Mr. Shabih Ali Mohib (Senior Economist), and Ms. Nattaporn Triratanasirikul (Economist), Rob Boothe (Consultant). The team would also like to acknowledge the role of the World Bank‘s EAP team, in particular Mathew Verghis, Lead Economist. The work was carried out pursuant to a Terms of Reference with the Office of the Public Sector Development Commission (OPDC) under the Thailand Prime Minister‘s Office. The World Bank 3 II. Background The Royal Thai Government is making a considerable effort to improve public sector performance, as reflected in the Royal Decree on Good Governance 2003, the Thai Public Sector Development Strategic Plan 2008-12, and the new Civil Service Act of 2008. In this effort the Thai Government is following a Results Based Management (RBM) approach to public sector reform. The themes of the Thai Public Sector Development Strategic Plan (2008-12) focus on developing a civil service that is flexible, adaptive and responsive to the needs of citizens. Specifically, service quality improvement and developing high performing organizations are two of the plan‘s main themes. Within this general framework, there are number of RBM initiatives underway throughout the public sector. These include, the Office of the Public Sector Development Commission (OPDC)‘s work on annual performance agreements across various levels of government based on the balanced scorecard (BSC) approach and Public Management Quality Award (PMQA). It also includes the Bureau of the Budget (BOB)‘s implementation of Strategic Performance Based Budgeting (SPBB), including its Performance Assessment Rating Tool (PART). In addition, the implementation of the provisions of the new Civil Service Act relating to the merit principle through performance management (including appraisal) of individual civil servants and the development of an HR scorecard for individual ministries and agencies (MDAs) increasingly reflects an adoption of results based approach.1 Individual ministries, agencies and sub-national governments also have their specific internal RBM performance indicators and criteria in achieving sectoral targets. Appendix E provides a comprehensive mapping of the various tools by agency and purpose. Results based management initiatives are primarily driven from the center in Thailand – BOB, OPDC, OCSC, NESDB - and the degree to which bottom-up initiatives are actively supported is unclear. Thailand‘s civil service is accustomed to operating under a rule-driven and hierarchical environment and the transition to RBM requires a significant change in the behavior and mindset of individuals operating in the system. 2 After nearly a decade of experience with implementing results based management tools in the public sector in Thailand, the principal stakeholders in the system recognize the need to move from compliance with reporting requirements to utilizing these tools to deliver results through integration with planning, budgeting and human resource management. Toward this end, the Government established an ad-hoc Public Sector Audit and Evaluation Committee (PAEC), which includes the heads of BOB, NESDB, OCSC, PMO, CGD, MOF, MOI, 7-10 external members, with the Secretary General and Deputy Secretary General of OPDC performing the role of secretariat. The purpose of this committee is to set the agenda and policy direction for audit and evaluation. Under this committee stands an ad-hoc subcommittee on Public Sector Audit and Evaluation formed in 2009 to consider the possible rationalization/integration of the range of RBM requirements. However, this committee has 1 Adapted from David Shand, ―Results Based Management in Thailand,‖ Discussion Note, March 2010. 2 For an interesting discussion on the evolution of RBM in Thailand see: Ch. 11 ―Managing Performance in the a Context of Political Clientalism: The case of Thailand‖ by Suchitra Punyaratabandu and Daniel Unger in The Many faces of Public Management Reform in the Asia-Pacific Region Research in Public Policy Analysis and Management, Vol. 18, 279-306 (2009) The World Bank 4 only met four times on an ad-hoc basis and has not thus far functioned as a platform for harmonizing the Results Based Management system in Thailand. This report highlights some of the key issues that should be addressed in order ensure that Thailand‘s Results Based Management system delivers on its intended objectives to improve the effectiveness and efficiency of government. In response to questions asked by the government, and drawing on the experience of countries that have implemented results based management tools and techniques, the report is divided into four sections:  An overview of Thailand’s Results Based Management system. In particular, this section considers issues raised during discussions with agencies on how the current system is functioning, highlighting the roles different agencies play in setting, monitoring and evaluating performance measures. Relevant lessons from international experience on how other countries have dealt with these challenges are presented.  General guidance on linking performance with the budgeting and planning process. This section highlights the implementation issues faced by Thailand, as well as other countries in pursuing results or performance informed budgeting.  Creating incentive for performance. This section considers some of the new approaches for creating incentives to deliver results, which are particularly important to ensure the use of performance information for learning.  Summary Recommendations. This section provides a set of recommendations for improving the current Results Based Management system in Thailand. The World Bank 5 III. Results Based Management Systems This section presents a framework for thinking about RBM systems. In particular, this section considers issues faced by agencies in complying with mandates and directives from central agencies. The specific topics covered are:  International lessons learned from RBM systems of relevance for Thailand  Findings and recommendations for Thailand’s RBM system _______________________________________________________________________ A. FRAMEWORK RBM systems deliver value through a number of channels such as, aligning different organizations to move toward common objectives or creating an environment in which agencies learn and improve program design. Drivers for introducing a Results Based Management system include the Ministry of Finance/Budget Office need for a basis for the allocation of funds and desire to know what agencies are buying, a desire by elected officials to hold the bureaucracy to account, the desire to encourage agencies to think about their production function and efficiency, to link their outputs to impact, and to link service quality to citizen needs (responsiveness). Generally, one can think of RBM systems designed to emphasize relatively more of one of these objectives:  To learn;  To steer and control; and  To provide accountability. In pursuit of one of these objectives or some combination thereof, countries often employ a number of results based management tools. These include, strategic planning, internal management information systems, benchmarking, risk analysis, or business process reengineering geared toward facilitating an internal management dialogue. Other countries use monitoring and management scorecards,3 performance agreements, citizen charters, annual reporting, and performance contracts. The objective/purpose of the system has implications for which tools are selected and how each of them is designed –specifically in terms of the accountability arrangements surrounding the implementation of the tool. Different performance tools have been developed to meet different objectives and problems, and address different linkages and accountability arrangements among various actors within the public sector. Many of these are mutually reinforcing, creating a network of supporting arrangements. Figure 1 presents a number of tools for reinforcing the linkages between different public sector actors. 3 The Balanced scorecard is a strategic performance management tool that was first developed by Art Schneiderman in 1987 and the method was further developed for practical use by Kaplan and Norton in 1996. It enables organizations to translate their vision and strategy into implementation through four perspectives: financial, customer, business process, and learning/growth. The World Bank 6 Figure 1: Performance Tools in the Public Sector Cabinet/ Legislature Presidency ▪ Government Performance Front Line Reports (Uganda) Ministries Service Providers ▪ Balanced Scorecards (US, MoF/MoE/ Thailand) Ministers Treasury ▪ Performance Monitoring (Brazil) ▪ Performance Agreements (UK Public Service Front Line Agreements) Service Citizens ▪ Performance Contracts Providers (New Zealand, Denmark) ▪ Client Satisfaction Surveys (Philippines) Legislature Executive ▪ Citizen Charters (Kenya) Ministries ▪ Performance Auditing (Supreme Audit Institutions) There are a plethora of examples of different RBM systems from across the globe. Some emphasize more of one objective than another and often at different levels of government. The institutional and functional variability of the systems makes it difficult to make cross-national comparisons. Notable examples include Colombia‘s National System for Evaluation and Management for Results (SINERGIA), Mexico‘s National Council for the Evaluation of Social Development Policies, the United States Government Performance and Results Act (GPRA) and the Management Control Division in the budget department of the Ministry of Finance in Chile. Underpinning the use of these tools is often a set of incentives at the organizational and individual level, which include monetary and non-monetary rewards for organization/individual performance and delegation of managerial authority for human resources/budgeting. Chapter 5 discusses the use of incentives primarily at the individual level. Generally, the results/performance of a project/program is in part determined by where you sit. As a purchaser of a service, your interests are typically different from those of the supplier of the service. For example, as a buyer of a car your preferences/interests will differ substantially from those of the manufacturer of a car. As a regulator, your interests (e.g. compliance with the law) are typically different from those of the regulated – who are often concerned with the lightness of the regulatory burden. In some cases, parties may have interests as both purchasers and suppliers (e.g. co-operatives). Government performance is complex in part because of the variety of interests pursued by different actors. Common Limitations/Risks to all RBM systems The World Bank 7 RBM systems evolve over time to respond to various political needs and changes. With any RBM system, there are a number of critical limitations to be aware of – these relate to the measurability of certain outputs/outcomes, over- or under representation of measured values, the use of too many indicators, lack of clarity on definitions and understanding of the performance information being produced leading to misinterpretations, and the manipulation of data (in terms of unjustifiable aggregation or disaggregation). Such risks cannot be eliminated, but only mitigated by using caution in interpreting information derived from the RBM system and responding to the inherent complexity in the public sector production chain in the design of the results based management system. Techniques for mitigating some of these risks associated with performance measurement include, publishing results at various levels of aggregation for the public, citizen monitoring of government projects, decoupling performance results from budget allocations, and introducing qualitative measures to supplement quantitative data, among other techniques. Framework for thinking about RBM Systems When analyzing an RBM system, it is useful to look at a handful of important control functions (e.g. HR and budgeting), incentives (such as performance pay), and reporting lines to see where in the internal administration they intend to exert influence. Ideally the design of these central systems should be aligned to reinforce one level, rather than multiple levels of accountability. The basic premise is that entities can neither improve their performance nor be held accountable if they have no authority to actually manage their own resources, and tools to support this authority. Figure 2 takes a few of these control functions, incentives, and reporting requirements and maps where in Thailand‘s internal administration they exert influence. For example, an entity responsible for delivering its outputs may not receive regular financial reports from a central treasury, preventing the entity from being able to actively monitor their budget (unless they develop their own parallel financial reporting system). Figure 2: Interaction of processes & Thailand’s internal administration HR Budget Performance Planning RBM Reporting Controls Controls Pay Level OPDC BOB NESDB OCSC 4 yr Operational OCSC Plans/ Annual Plans Ministries OPDC Departments Offices Unit Teams OCSC Individuals The World Bank 8 As reflected in the diagram, the systems for HR control, budget controls, performance pay, planning, and performance reporting are not well aligned, reflecting a lack of system coherence either towards control or supporting management. Typically in British Westminster systems, the emphasis is placed on ministerial accountability, with controls, accountability, and reporting aligned at this level. Incentives are then tied to the accountability arrangements. In Thailand, HR controls are at the ministerial level, while budget controls are at the ministry and unit levels. Performance pay, currently managed by two separate agencies (OCSC and OPDC), is at the department and individual levels. Planning is done at the ministry and department level, as is reporting on RBM for the most part. Generally, countries with more mature RBM systems attempt to align some of the systems for accountability, controls, and incentives in order to establish a meaningful ‗unit of accountability‘ in the RBM system and avoid conflicts, which can arise from disjointed controls and accountability arrangements. For the incentives in the system to work properly, both authority and accountability need to be aligned. In conclusion, the actual impact of any performance system is a function of how much it changes behavior appropriately and achieves results. In other words, the impact of an RBM system is a function of desired behavior change divided by the complexity of the system. As the complexity of the system increases, the impact decreases. If the RBM system seeks to measure everything with no selectivity, the systems may actually end up measuring and achieving nothing. B. LESSONS FROM INTERNATIONAL EXPERIENCE Over the past decade, a wealth of information has been collected on the experiences of countries pursuing results based management reforms. The OECD‘s ―In Search of Results: Performance Management Practices‖ (1997) analyzed a wide variety of country experiences along the dimensions of objectives and approaches, institutional arrangements, performance measurement, financial management, and the reporting of performance information.4 A number of important lessons from OECD, World Bank, and country reports are addressed here of relevance for the situation in Thailand. Lesson 1: Over-collection of performance information and setting too many targets limits the effectiveness of an RBM system. The lessons from the UK‘s experience with high level targets illustrate this point well. In connection with the 1998 Comprehensive Spending Review exercise, Public Service Agreements (PSAs) were introduced setting around 600 performance targets for roughly 35 areas of Government. Over time these were refined and reduced, as a result of feedback from agencies. Setting the right PSAs was a challenge – in the first round only 15% of PSAs were outcome-based, and they were not matched to the budgeting cycle. Moreover, a lesson learned was that setting the PSA targets was not sufficient to achieve results without a more detailed action plan for the Ministry explaining how the PSA target would be achieved. By the Spending Review in 2000, the number of top level PSAs were reduced to around 160, supported by a range of Service Delivery Agreements (SDAs), which included a lower level set of targets for the purpose of departmental delivery planning. In 2001, the United Kingdom established the Prime Minister‘s Delivery Unit (PMDU) to help monitor progress on and strengthen the UK Government's delivery of public service priorities through a sustained focus on the performance of key services and 4 OECD, In Search of Results, PUMA, 1997. The World Bank 9 public sector management – including supporting the foundations of PSAs. By 2009, PSAs had been reduced to 30, signaling the major outcome-oriented priorities of government. The evolution present in the UK experience reflects the tendency when countries first start out to want to monitor the performance of everything in the system as illustrated in Figure 3. Generally systems will go through a period of intense data collection, but over time the number of indicators/targets reported to the national level tends decreases. Figure 3: Number of national level indicators used by central government 2,037 2,000 Goals Objectives Targets Indicators 1,500 1,178 1,000 562 545 500 454 400 200 164 153 127 54 46 4 30 4 13 4 13 - United Kingdom Netherlands Canada France Korea (1998) (2002) (2003) (2005) (2007) Source: OECD, International Database of Budget Practices and Procedures, 2007. In the case of US State of Oregon, the ―Oregon Shines II‖ program and evolution of the Oregon Benchmarks also reflects a similar evolution. At the start of the program, there were over 270 Oregon Benchmarks. The sheer number of indicators caused confusion and misunderstanding, and were basically ineffective for decision making purposes. After years of careful observation and refinement, Oregon narrowed its benchmarks to 90, to fit within seven major areas, which have been universally accepted as both ―meaningful and otherwise helpful.‖5 Many cases suggest that countries embarking on establishing RBM systems initially collect more performance information than they know what to do with or can effectively use and over time reduce requirements for reporting at the national level. Lesson 2: Simply collecting performance information doesn’t necessarily lead to improvements. 5 Young, Richard D. (2005). An Overview: Oregon Shines II and Oregon Benchmarks,‘ Unpublished manuscript, South Carolina, Institute for Public Service and Policy Research, University of South Carolina. Accessed from: http://www.ipspr.sc.edu/scip/publications/Final%20Oregon%20Shines%20II%20and%20Oregon %20Benchmarks.pdf The World Bank 10 Performance information is often collected in a wide variety of formats and for a number of intended users. No one system is likely to meet the information needs of all users. The usefulness of this information for any given user depends on what information is being collected, the quantity and quality of the information, how it is presented, the intended purpose, and the timeliness of the information for relevant decision-making processes (e.g. the budget or planning cycle). The non-use of performance information can stem from a misalignment in any of these areas, as well as its interaction with other inputs in decision-making (e.g. political influence). In addition, a well documented phenomenon is that of ‗gaming‘ or manipulation of indicators or results for reporting purposes. Such contamination in the data undermines its credibility and efficacy for decision-making purposes. Getting users to use the performance information for decision-making is therefore contingent on a number of factors and requires careful attention to quality assurance and usability. Box 1 describes the quality assurance framework in place to support the UK Public Service Agreements. Box 1 Improving the quality of performance information—the Case of the UK At the heart of the UK‘s performance management system are Public Service Agreements (PSAs), agreed between the Finance Ministry and line ministries. Introduced in 1998, their aim is to focus resources on improving outcomes for the public and to strengthen accountability for cost effective service delivery. Published alongside departments‘ three-year budget allocations, PSAs specify: i) the department‘s aim; ii) five to 10 supporting objectives; iii) performance targets, including a value- for-money target; and iv) standards to be maintained, monitored and reported. PSA targets have been refined gradually to focus on outcomes rather than the inputs or processes. The number of targets has been reduced, from around 400 in 1998 to 30 in the 2008-11 Spending Review. Biannual reports are published, which provide information on spending and performance against PSA targets. While the public and parliamentary oversight of performance has generally been weak, there has been a great deal of emphasis on quality assurance of the performance measures. The Office for National Statistics provides advice to ministries and agencies on methods, and on quality assurance for statistical systems through central government, certifying appropriate indicators. The National Audit Office, who has created a Directorate of Performance Measurement to co-ordinate work on performance measurement in financial audits, provides advice, training and reviews of central government bodies‘ governance (and performance) arrangements, including in value-for-money studies. Source: http://www.hm-treasury.gov.uk Lesson 3: Performance measures will vary by activity (e.g. not all are amenable to outcome measurement or the same type of measures) It is well known that the organizations and units within government serve highly different functions and therefore, when it comes to establishing measurement systems for organizations, some are easier to measure the outputs and outcomes of than others. To illustrate this point, Wilsons‘s 1989 typology of organizations is useful (Table 1). Among organizations/units, production organizations are the easiest to measure the outcomes and outputs of, for example in tax collection, sanitation, vehicle registration etc…. In such cases measuring and managing for results is easier to do than in coping organizations, such as diplomacy or research, where the outputs and the outcomes of the organization are difficult to measure. A good example of this is the US National Science Foundation (NSF), which funds cutting The World Bank 11 edge research. It is not possible to measure the impact of the organization meaningfully in the short run, rather only measures of outputs (e.g. number of papers written etc..) are available. A creative solution to this problem was the establishment of a panel of experts who examine the quality of the research portfolio on an annual basis. A more comprehensive, retroactive examination of the impact of the research portfolio is done on a 5 or 10 year basis. This suggests that the measurements and definitions of outputs/outcome should be tailored to the different profiles of government entities and interpreted with caution in the areas where such measurement is very difficult and imprecise. Table 1: Wilsons (1989) typology of organizations6 Outcomes Observable Yes No Outputs Yes Production Organizations. Procedural Organizations. Examples: mail services, tax Examples: mental health, Observable collection, sanitation, counselling, military vehicle registration, revenue (peacetime), youth collection penitentiary No Craft Organizations. Coping Organizations. Examples: Field inspections, Examples: Diplomacy, military (wartime), doctors, Intelligence, research forest rangers Lesson 4: Simple, less complex measurement and management systems with a single entity coordinating the process reduces transaction costs and improves functionality. Many countries suffer from overlapping mandates between actors/tools in an RBM system due to the nature of the way government is organized and the power relations between layers of government.7 This creates an overwhelming demand on ministries/units for reporting, monitoring, evaluation both upward and outward. Over time, actors at the center of government have realized that coordination is critical and that a clear understanding of objectives is necessary to create a well-functioning system. In the US state of Oregon, a Progress Board was established to oversee the implementation of the Oregon Shines II and Oregon Benchmarks. The Board provided governance and oversight for an integrated system. The 12-member board, chaired by the governor, was assisted by a staff of professionals who gather data, monitor progress, and prepare reports. The Board was tasked with clearly identifying the aims and audience for the establishment of an indicator system. 6 Wilson (1989) as described in Wouter Van Dooren et al, Performance Management in the Public Sector, (2010). 7 See the GET Note: Japanese Public Sector Assessment Processes ―Recently Asked Questions‖ Series August 2010, World Bank for a description of Japan‘s experience. The World Bank 12 In the UK, the establishment of the PMDU‘s in 2001 office helped to bring focus to the various performance management initiatives underway and to supplement and draw upon existing tools (audit, evaluation etc..) for use in reporting to the Prime Minister. The US state of Maryland‘s experience with Statestat – a performance measurement and management system – also underscores the importance of establishing a key actor at the center to oversee and manage the overall system. In use since 2007, StateStat was originally modeled after Baltimore‘s CitiStat as a way of capturing and monitoring the progress of government service delivery using frequently updated data. Through a process of continual re-evaluation by the Governor, his executive staff, and agency leadership during bi-weekly meetings, new and improved strategies emerge for delivering key public services effectively and efficiently. Located within the Governor‘s offices in Annapolis, a lean staff provides the logistical and analytical support for the operation.8 According to a report on insights from international practice on performance reporting, the six attributes of a good system for outcome and output reporting are: 1. When developing performance measurement systems, use a consistent, comparable, and structured approach to performance information reporting across all agencies and programs. 2. Include a good performance story to accompany the indicators. 3. Specify outcome indicators, and fully explain the results reported against the indicator. 4. Provide both target and baseline data. 5. Ensure effective use of technology in presenting the performance data collected. 6. Present agency performance information which includes output and activity indicators in addition to outcome indicators.9 Lesson 5: Proceed with caution when directly linking measurement systems to incentives Getting the measurements of outputs and outcomes correct is difficult and may take years to refine. Because you will get what you measure, often with unintended consequences, countries have been reluctant to tightly incentivize measures. This is why many countries have been reluctant to tightly link performance information with monetary incentives (particularly in the case of budgeting). Moreover, there are many good examples of how measuring the wrong thing can actually counteract real Figure 4: Frequency distribution of improvement, for example the case in Thailand of the ambulance response times for life- indicator on the number of patients seen per day by the threatening emergency calls in UK dentist. This quickly resulted in a substantial decrease in the time spent per person and an increase in repeat visits. If this were linked directly to budgetary allocation, it could have had disastrous spill-over effects. There is another good example of ‗gaming‘ from the UK in which a target was developed for response times of ambulances to life-threatening emergency calls. In order to incentivize a quicker response time the target was set at 8 minutes or less. Under pressure to record the ‗right‘ answer, data was manipulated to suggest improvements – as Figure 4 8 GET Brief, ―Ten Observations on StateStat,‖ December 2009, World Bank. 9 Richard Boyle, ―Performance Reporting: Insights from International Practice,‖ IBM Center for Business of Government (2009). The World Bank 13 suggests, calls were reassigned to have response times under 8 minutes.10 As a result of such cases, most countries use performance information alongside a number of other factors when making decisions. C. FINDINGS & RECOMMENDATIONS In the past decade, Thailand has implemented an impressive array of results based management tools and techniques, drawing largely from the private sector performance management literature and international practices. Thailand‘s approach includes the use of Balanced Scorecards, performance agreements, a performance assessment rating tool, total quality management techniques, among others. A holistic perspective on results management is evident in Thailand‘s system, with a focus on principles such as effectiveness, the quality of service delivery, the efficiency of operations, organizational development, leadership, strategic planning, customer and stakeholder focus, and financial and personnel management. There have been some notable successes in Thailand with RBM, including service delivery improvements in: (a) reducing the service time for issuing permits, passports and driving licenses and (b) reducing the time necessary for filing taxes. In order to capture and document the broad array of tools being implemented by various agencies in Thailand, the team developed a Results Based Management matrix, included in Appendix E. This matrix includes the RBM tools used in Thailand to assess, to provide incentives, and to establish service delivery standards. Each tool is then analyzed along the following dimensions: scope/coverage, main objectives, assessment criteria, indicators and questions, weighting, frequency of reporting, timing of reporting, use of information, number of key performance indicators, government plans, frequency of reporting etc… This matrix provides a way of systematically analyzing the RBM system in Thailand and could serve as a useful instrument for the Government to complete and update as the system evolves. As mentioned previously, how and whether the data and reports produced are used is perhaps the most critical indicator of the functioning of an RBM system. From a slightly different viewpoint, Figure 5 maps the tools to the agencies and captures the visual complexity of the current system. 10 Bevan, G. and R. Hamblin (2009), ‗Hitting and Missing Targets by Ambulance Services for Emergency Calls: Effects of Different Systems of Performance Measurement within the UK,‖ Journal of the Royal Statistical Society, 172 (1), 161-190. The World Bank 14 Figure 5: A bird’s eye view of Thailand RBM system OPDC BOB OCSC NESDB DOLA Service Value for BSC/PMQA PART HR BSC Standards/ Money LQM 141 18 73 Local 20 75 Departmen Provincial Academic Administra Ministries Provinces ts Clusters Institutions tive Org. Moving forward, it is important to establish a working group to review all of the RBM tools with a specific set of screening questions in mind. These include: o Who benefits from this data? o Who will it be useful for and how? o What will the data/reports tell you? o Will the right people have the information they need at the right time in order to make decisions? o What is the appropriate level for reporting? o How often should the data be reported? o What are the costs of collecting the data? o What is its intended purpose – for learning, steering, or accountability? Ultimately, the value in data produced from Thailand‘s RBM system is not in the data per-se, but rather in the corrective actions and strategic decisions made as a result of using data. Based on conversations with authorities during the mission and a review of documents and reports on Thailand‘s RBM system, a number of issues/challenges with the current system emerge, which are discussed below: 1. Coherence of the RBM system i. Overlapping tools with similar purposes As described in the background section, Thailand has developed multiple initiatives on RBM under many agencies for assessing performance, leading to high transaction costs for agencies. There are The World Bank 15 roughly twelve separate tools being implemented from the center of government at the moment, some of which assess similar organizational aspects, such as strategic planning in the both the PART and the BSC or use similar incentives, such as performance pay for individuals and departments. Part of the reason for the overlapping of tools stems from the legal duplication of mandates in the Royal Decree on Criteria and Procedures for Good Governance, B.E. 2546 (2003) which specifies roles and responsibilities for different central agencies in the RBM system. Recommendation: Convene a forum/steering committee (this can be existing or new, as long as it includes the relevant actors) to review the matrix and comprehensively document the various tools and their purposes, with a view to identifying duplication and areas for integration particularly in terms of underlying objectives/questions. Draft a clear Terms of Reference for committee members. A chair should be identified at the onset. Implementing some of the emerging recommendations may require legal amendments and it will be important to involve staff from a few lines ministries in these discussions.11 ii. Significant transaction costs as a result of the RBM system for agencies The different requests for performance information from the OPDC, BOB and OCSC, among others, imposes significant transaction costs on ministries and departments, academic institutions, provincial clusters, and provinces. For example, there are three different divisions within DOLA which are taking care of the various aspects of the BOB, PART, BSC etc.. Balancing the costs of collecting and reporting this information, in terms of both human and monetary resources, with the benefits derived from the information is necessary to maximize the usefulness of any tool. Recommendation: Conduct a series of focus groups/surveys/meetings with individuals from ministries, departments, etc.. to get a better sense of the transaction costs they face in reporting, what information they already collect and what new information they need to collect as a result of reporting requirements. Most importantly, assess the degree to which managers have the information they need from the system to make improvements. Use the results to do a cost/benefit analysis of the various questions addressed through the different tools and whether the information might be better derived from other sources. iii. Lack of consultation and coordination between central actors in charge of RBM at the center of government From meetings held with the various central agencies in charge of different aspects of the RBM system, it is clear that there is a lack of communication between agencies on results based management. As a direct result agencies are developing their tools using different RBM terminology. The establishment of the PAEC is a step in the right direction, but more frequent meetings or another established line of communication would aide this process. 11 In line with the development of an integrated financial management system, IT systems capturing reported results could be usefully developed and refined to allow diverse users the opportunity to access reports. However, prior to automation it is highly recommended that the system be consolidated with clear understanding on who will use this information and how and ensure that the appropriate levels of access to systems are granted. The World Bank 16 Recommendation: Increase the frequency of meetings of the ad-hoc subcommittee on Public Sector Audit and Evaluation to increase the lines of communication between the various central actors. iv. The use of performance information for actually improving programs and getting results seems weak. As described earlier, determining who uses the information derived from the RBM system is difficult and to a large extent the use of this information for actually improving programs does not seem well-documented, with some notable exceptions. See Appendix E for details. Those to whom the performance information is reported should have a clear responsibility or reason for using the information. At present, the Thai system seems focused on producing reports and data, rather than on the use of the information. Recommendation: Improve the alignment of incentives, reporting, and timeframes so that information is available to those who can best use it. Streamline reporting to what is essential. 2. Current RBM framework characteristics i. Over production of KPIs The number of Key Performance Indicators captured in the RBM system in Thailand is difficult to measure – as the concept is used in a number of tools (including at the individual level) and the scope of KPIs is quite broad. A rough estimate of KPIs reported to the national level - 20 KPIs per organization covered by the Balanced Scorecard for 20 Ministries and 141 departments suggests that over 3,000 KPIs are being reported to OPDC through the BSC process alone. There are also 37 KPIs under the five dimensions of the PART, which indicates that well over 3,000 KPIs are being collected centrally. Recommendation: Reduce the number of KPIs being collected at a central level through a number of different channels: (i) by formulating a small number of high-level targets the Cabinet/PM wants to keep track of, (ii) by determining what indicator information is most useful at the central level and (iii) by integrating KPIs from the BSC and the PART. The system might usefully emphasize performance information of relevance to managers for actual management rather than for central reporting. ii. Lack of emphasis on the users of performance information The majority of reporting information being collected through the BSC, PART, and HR scorecard is reported to a number of central actors, but it is not clear the degree to which actors at the center of government need what information, when, and in what formats. Typically, in a results based management system, actors in various parts of government are concerned with different aspects of performance information. See Table 2 for an illustrative list of different types of users and how and what type of performance information they use. Recommendation F: Match the roles of users in the RBM system to the reporting requirements, to ensure that those who receive the data can actually use it. This may mean that fewer KPIs are reported to the central level, and more are reported up to the ministerial level. The World Bank 17 Table 2: Users and the use of performance information12 Type of Performance Users Emphasis of uses Information Program information, Program managers Learning internal documents Senior officials in Program information, central agencies and Steering and control internal documents ministries Program performance, Ministers Accountability internal documents and reports on performance Reporing on performance, annual reports, forward Members of Parliament Accountability estimates, reports on performance Citizens Accountability Targeted publications iii. Timing and frequency of various evaluations and reporting processes A comprehensive assessment of the timing and frequency for the various evaluation processes and reporting would be useful to leverage any possible synergies and identify gaps. Box 2 describes the role of a central evaluation unit in Ireland. In particular, the Bureau of Budget raised the issue of setting appropriate timeframes for the use of performance information in the budgeting cycle. While inconsistencies between timeframes (budgeting, policy-making, evaluation etc..) will be present in any system, an attempt to get key users of performance information access to the information they need at relevant times is important. Box 2 Role of a Central Evaluation Unit—the Case of Ireland Ireland established a Central Expenditure Evaluation unit in 2006. The unit is located within the finance ministry with the mandate for the following:  To promote best practice in evaluation and project appraisal across government, notably through the use of guidelines, the promotion of common approaches and the development of networks of practitioners.  To ensure compliance with the Department of Finance‘s Value-for-Money Framework, including through spot checks for compliance.  Provides a Secretariat to the Value-for-Money Central Steering Committee.  Provides technical advice to Departments and Agencies and facilitates capacity building.  Oversees evaluation of the National Development Plan (NDP). Source: see European Commission Sourcebook: Capacity Building, available on www.er.europa/regional_policy/sources/docgener/evaluation/evalsed/sourcebooks/ 12 Adapated from Wouter Van Dooren et al, Performance Management in the Public Sector, Routledge, 2010. (pg. 118) The World Bank 18 On the frequency of reporting, typically, monthly and quarterly reporting are for internal ministerial management and include a large amount of process indicators. Reporting done on an annual basis may attempt to track some output indicators. Reporting done every 3-6 years is likely for impact evaluations. In Thailand, the OPDC is currently collecting information on the Balanced Scorecards every 6, 9, and 12 months. Recommendation: Document and analyze the timing & frequency of the various reporting and process requirements of the RBM system –including that of evaluation - and map this against key points in the budgeting and planning cycles to determine whether better timing can be achieved between the reporting and analysis of RBM data. This would increase the usefulness of performance information in these on-going processes. In addition, consider the appropriate frequency of reporting for the relevant users. iv. Confusion in terminology Many agencies are pursuing aspects of the RBM framework, but with remarkably different terms for similar ideas (efficiency, value for money, etc..). This is particularly evident in what qualifies as an output/outcome and a key performance indicator. A number of different terms and definitions are being used as the assessment criteria in the various tools, despite similar objectives. While this reflects a more generalized problem in the field of Results Based Management and Monitoring & Evaluation, it would be highly valuable to adapt and adopt a specific set of terms for Thailand. Recommendation: Develop agreement on a set of definitions that would be consistent across all agencies for all key terms used in the RBM systems (indicator, KPI, efficiency, effectiveness etc..) and publish this alongside tools and communications. This would be highly useful to do prior to discussions on duplication. 3. Monitoring and Evaluation: i. Lack of follow-up from management on evaluation and audit findings The systematic process to review audit findings (particularly internal audit reports), as well as follow-up action from management levels has been lacking (PEFA, 2009). Moreover, the National Assembly does not issue any follow-up recommendations on Office of the Auditor General‘s (OAG) annual report. Recently, the OAG initiated legislation to strengthen the audit follow-up process and this has been submitted to Parliament. Recommendation: Evaluate the informal and formal follow-up functions/processes for all audit and evaluation findings and try and strengthen weak linkages between key actors. This may also require Parliamentary training on how to follow-up on audit findings. ii. Program assessments are being done by various agencies– but not necessarily in a strategic way Generally various types of monitoring, review and evaluation are commonly conducted by different agencies within government. Careful consideration must be given to defining the units best suited to which type of review or evaluation (see Figure 6). For example, line ministries tend to manage programs and have access to most information, so are likely to lead program reviews for The World Bank 19 management purposes. However, line ministries may not be well placed to conduct efficiency reviews, where there is the threat of a spending cut. Impact evaluations may require more complex methods and might be contracted out. Summary measures and spending reviews are commonly coordinated by the central budget agencies, with the information provided mainly by the line ministries. There is a distinction to be made between ad-hoc assessments, which can be very useful, and impact evaluations. Figure 6 presents a few different tools for monitoring and evaluation. Each of these types of reviews/evaluations can be useful at different times, serve different purposes, and should be done at appropriate intervals (e.g. impact evaluations should not be done on an annual basis). Recommendation: Analyze the various types of evaluations being done by agencies to determine if there are gaps or if resources are better used centrally to manage certain types of evaluations. Figure 6: Different Tools for Monitoring and Evaluation Summary measures incorporate a wide range of performance information into one or more overall performance ratings for a program, e.g. US PART. Typically collated by central agencies Program reviews Impact evaluations Spending reviews Review consistency Assess program Assess consistency of in design, execution effectiveness on basis of portfolio of programs and reporting impact measures within and across sectors Methodology includes Based on logical extensive data Set ex-ante (multi-year) framework collection, sophisticated nominal expenditure evaluation techniques ceilings Often performed Often coordinated at internally within Often performed by the centre, but with line ministries experts - consulting strong input from firms, universities etc departments/ programs Value-for money /efficiency reviews – consider the scope for efficiency savings across public expenditure. Often centrally driven, but may use ‘independent’ resources, while supreme audit agency also considers on a case-by-case basis 4. Additional observations i. RBM Framework for Local Authorities Local Authority Organizations play a considerable role in the delivery of services, with 25% of total revenues being delivered to them. However, there is little information on their fiscal operations and performance. The Ministry of Interior‘s, Department of Local Administration (DOLA) has begun the implementation of their own set of RBM tools – adapted for the sub-national context. At the same time, though, Ministries, through the establishment of the RBM system, will also exert reporting pressures on local authorities. The World Bank 20 Recommendation: Strengthen the linkages of DOLA’s RBM tools with the overall RBM system and assess from the standpoint of the local authorities how reporting and data management is likely to function. It will also be important to use similar terminology for the RBM system. ii. Internal Control System The interaction between the internal controls system/framework and the RBM system is unclear. The Government internal control standards are based on the international benchmark of COSO‘s five components of (i) environment of the control entity, (ii) risk assessment, (iii) control activities, (iv) information technology and communications; and (v) assessment. Since 2001 each agency has an internal audit function tasked with reporting to the agency‘s top management on issues they need to address. However, one-forth of all agencies have not complied with reporting on the internal controls system due to capacity constraints. Recommendation: Consider using the internal controls system for quality assurance (through verification) of the performance information collected. The World Bank 21 IV. Linking Performance with the Budget & Planning This section addresses the issues faced by Thailand, as well as other countries in linking performance information with budget and planning. In particular, it considers the main sticking points in the process for Thailand and draws on relevant international experience for reference. These include:  The role of the central budget agencies in strategically aligning budgets with priorities and using performance information to inform budgetary decisions.  The reporting, incentives, and guidelines needed to successful implement strategic based performance budgeting. _______________________________________________________________________ Strategic budgeting is an effort to introduce a strategic element into budgeting, along several dimensions. These include in the areas of macrofiscal (revenue, expenditure, debt) sustainability, overall fiscal risk-management, multi-year planning, policy and objectives, multi-year capital investments, policy affordability, as well as in setting sectoral strategies and objectives. Any level of administration can have strategic elements: national, elected government, sector or ministry, subnational, or municipal. Various tools and techniques for strategic budgeting include: changes to the classification of the budget along functional, economic, and/or program lines, the introduction of multi-year estimates in the areas of macroeconomic, fiscal, functional, sector/ministerial, program forecasts, the establishment of a macroeconomic policy paper, a Fiscal Policy/Strategy paper, a Government plan/manifesto, a National Development Plan, a National Vision Statement, a Poverty Reduction Strategy Paper, Strategic Planning at the ministry, sectoral, etc. levels, and long-term planning (20, 30, 50 years) for the government as a whole. The introduction of budgets in the 18th century was itself intended to add a strategic, forward –looking element to fiscal management. A. ISSUES Thailand has a basis for offering performance information to the major players for use in the budget cycle and has been pursuing efforts to integrate strategic elements into the budgeting process.13 13 In the Thai Public Sector Development Strategy 2003 -2007, one of the six elements of the strategy called for reforming the fiscal and budgetary systems to: (i) Ensure budgets are spent in accordance with Cabinet policies, government strategies; ensure accountability for results through agreements concerning tangible outcomes, such that monitoring and evaluating can be conducted at every level. (ii) Adjust the budgetary system to link to government strategies; make public officials more responsible and results oriented. (iii) Develop integrated strategies or area development plans; allocate resources with focus on area conditions. (iv) Government agencies agree beforehand that unused budgets can be allocated to institutional development or training. (v) Prepare monthly/quarterly expenditure plans; prepare financial statements/reports for electronic funds transfer (vi) Improve the accounting system to international standards, so capital costs can be calculated for the provision of public services. Source: Ch. 11 ―Managing Performance in the a Context of Political Clientalism: The case of Thailand‖ by Suchitra Punyaratabandu and Daniel Unger in The Many faces of Public Management Reform in the Asia- Pacific Region Research in Public Policy Analysis and Management, Vol. 18, 279-306 (2009) The World Bank 22 The Bureau of the Budget also adopted the Medium Term Expenditure Framework (MTEF) and is in the process of refining and improving it. The MTEF is characterized by both a bottom-up and a top- down process and is situated within the annual budget formulation calendar. Moving forward, the BOB is looking to develop forward estimates of policies and to develop a budget strategy paper as well as considering providing indicative budget ceilings to ministries and agencies as a base for budget preparations. At this time the Bureau of the Budget does have indicative budget ceilings but these are not shared with ministries and agencies. Other technical reforms underway in Thailand include plans to introduce more robust integrated management information systems (Government Fiscal Management System –GFMIS) at the local level, strengthening the role of performance audit, and monitoring and evaluation of spending. Such a focus demonstrates the Government‘s commitment to adding more strategic elements to the budgeting process. However, there is a consensus that much still remains to be done in improving the budgeting process and linking this with both planning and monitoring and evaluation processes. Main Issues As noted earlier, Thailand has been implementing budgeting reforms for many years. Earlier reforms designed to increase managerial autonomy and performance such as Thailand‘s hurdle approach to budget reform may not have been fully successful.14 To improve the likelihood of success of any new reform, it would be useful to understand the reasons why previous efforts did not achieve their intended results. In particular, understanding the drivers of change in the Thai system and the incentives facing the actors would be important to know in designing future reforms. The main strategic elements that BOB currently uses in the budget process are: (i) the PART, (ii) Ministry and department annual performance/action plans, and (iii) Ministry‘s annual performance agreements. Each of these elements are mapped onto Thailand‘s FY 2009 budget calendar, in red, to demonstrate where in the calendar they appear. Each is discussed, in turn, below. Thailand‘s Performance Assessment Rating Tool (PART) was first piloted in 2005 in 20 agencies. FY 2007-2008 was the first year the Bureau of Budget used PART to evaluate the performance of every government agency. PART questions (roughly 30) cover five main areas: purpose and design, strategic design, action plan, output management, and output and outcome results. The performance of agencies on PART is used by BOB to analyze and approve budget requests. Based on discussions with BOB and other agencies, and a review of the results of the Fiscal Year 2009 PART for the Ministry of Foreign Affairs‘, Office of the Permanent Secretary, a few key points emerge. First, it is not entirely clear how well the PART meets the needs of BOB. Second, the quality and credibility of the PART results is highly variable, reflecting variation in measurement capacity among agencies and reasonable difficulties of measuring certain types of agencies (e.g the outputs and outcomes of the Ministry of Foreign Affairs are much more difficult to quantify than the Ministry of Transport). Given the lack of flexibility of the tool and the weighting assigned to the questions it is likely that the overall scores are not highly credible. Finally, it is not clear how well PART fits into the overall results based management framework in Thailand (addressed in depth earlier in the report). 14 World Bank, PREM Notes, Public Sector, Thailand‘s Hurdle Approach to Budget Reform, 2002. Available from: http://www1.worldbank.org/prem/PREMNotes/premnote73.pdf The World Bank 23 Another issue which emerged from discussions is the duality of the central development planning process. At the national level the National Economic and Social Development Plan (NESDP) covers five years and is prepared by the Office of the National Economic and Social Development Board (NESDB). The NESDP is approved by Cabinet and endorsed by the King (the current plan covers 2007- 2011). With every new Prime Minister (PM), the PM and their party must produce a Policy Statement of the Council of Ministers to cover a four year period (to coincide with the electoral term of the government). This plan is then translated into the Government Administrative Plan (GAP) covering a four year period, expanding on the policy statement. The GAP was first introduced in 2005. Frequent changes in political power and two separate planning processes - one chiefly managed by technocrats (the NESDP) and the second principally managed by politicians (the GAP) - suggests that the integration of plans into one coherent direction for the country does not necessarily emerge naturally. Below the national level, four year Operational Plans are required for Ministries and Departments, as well as an Annual Performance/Action Plan. Line ministries present plans for review by the NESDB, and once they are approved, it then goes to BOB for discussion. Below the level of ministries and departments, the planning process is more ad-hoc and less structured. According to the Ministry of Interior, there are no strategic plans (at this time) for the 19 provincial clusters and planning at the local level is inadequate and fragmented. According to the 2009 PEFA, the four year operational plans and annual performance/action plans required of Ministries and Departments do not reflect costed sector strategies. In other words, programs and activities are not necessarily costed during the planning phase. This significantly undermines BOB‘s ability to draw on quality information from the annual performance/action plans for input into the annual budget. The third element used in budgeting discussions are the annual performance agreements (managed by OPDC). BOB receives a copy of each agreement and KPIs which cover outputs, outcomes, and service quality are analyzed during budget discussions. More information on how exactly each piece of information is used in the budgeting process is needed to get a full a picture. One final issue, which has implications for the quality of planning within an appropriate resource envelope, is the accuracy of macro forecasts for the economy. NESDB has the office of macroeconomic forecast, but noted that the accuracy of the forecasts needs improvement. The World Bank 24 Budget calendar FY 2009 Process No. Date Activities Budget review 1 14 Oct 08 - The Cabinet approve the process of budget preparation and budget calendar in FY2010 2 - Government agents/ SOEs and other agents 15 Oct 08  Submit the out-turn budget expenditure report in FY2008 to BoB 15 Oct 08  Submit the Performance Assessment Rating Tool (PART) Report in FY2008 to BoB 3 - Collaboration between government agents SOEs and other agents and BoB Oct-Nov 08  Reconsider the strategic plan, output, outcome, activities and Key performance (KPI) in FY2009 for budget preparation in FY2010 Nov 08  Update the database to forecast Medium Term Expenditure Framework (MTEF)  Forecast of minimum budget necessary for Nov 08 current expenditure in consistent to assumptions from MTEF  Government agents/ SOEs and other agents Nov 08 together with BoB propose the preliminary budget expenditure framework in FY2010, consisting of minimum budget necessary for current expenditure, committed expenditures, basic needed activities as well as activities committed under strategic plans Implementation 4 Oct 08 - Ministry of Finance (MOF) prepare the FY2010 following to the revenue forecast and preliminary budget projection Royal Decree on N+3 in preparation for Government Administrative Principle and Plan procedure for Oct 08  The Secretariat of the Cabinet, The Good Public Secretariat of the Prime Minister, NESDB, Governance, BoB and OPDC altogether set the guideline B.E. 2546 in preparation for the Government (2003) Administrative Plan Oct 08  BoB and OPDC set the guideline for the 4- year Operational Plan as well as the Annual Operational Plan B. LESSONS FROM INTERNATIONAL EXPERIENCE Strategic Budgeting government – wide The World Bank 25 A number of countries have implemented a variety of approaches to create a connection between organizational performance and budgetary allocations. At one end of the spectrum are formulaic (or tight) approaches in which departmental performance is quantified and directly tied to changes in funding. South Korea is the best example of this approach – though it is uncommon. At the other end of the spectrum are non-formulaic (or loose) approaches, in which performance information is used as one input among many to influence funding allocations. Most countries take a non-formulaic approach with no automatic performance-to-funding link. Examples include the UK and the US. The performance information is used to inform budgetary allocations at different points in the process by different actors depending on the country. Such approaches often fall under the heading ‗performance informed budgeting.‘ Table 3 presents the OECD classification of performance budgeting categories. This section focuses on the processes and mechanisms used at the non-formulaic or ‗loose‘ end of spectrum to link performance with budgeting. The ‗loose‘ approach in which numeric performance targets are only one consideration in funding decisions is preferred because uncertainty over the behavioral incentives tied to specific indicators creates a high risk of unintended adverse consequences the tighter the formal link. Table 3: Performance Budgeting Categories Source: OECD (2007), Performance Budgeting in OECD Countries, OECD Publishing. doi: 10.1787/9789264034051-en (pg. 21) A key issue in strategic budgeting is whether the performance information collected is used in the budget process and especially in resource allocation. Performance information is linked to funding in two ways: through expected targets and actual results achieved in the previous budget cycle. As Curristine (2007) points out, in the majority of OECD countries there is no systematic approach to linking public expenditure to performance targets. As Figure 7 shows, spending ministries use the results of performance measures mainly to manage programs/agencies, redistribute resources, extend programs, and/or provide information for policy development and advice. Only about one third of OECD countries report that 50 or more percent of the allocated resources take into account the determined output or outcome targets. The World Bank 26 Figure 7: How are the results of performance measures generally used by spending ministries/departments? Source: OECD (2007), Performance Budgeting in OECD Countries, OECD Publishing. doi: 10.1787/9789264034051-en (53) The goal of performance budgeting should not be to provide the answers to inherently political choices in the budget process but rather to provide a new set of questions to enable more accountability for results and better inform decisions. The shift in the agenda for the budget process could be expected to bear fruit in a more informed debate that adds performance goals and results to the other important issues addressed in annual resource allocation debates. A good example of this in practice is the use of a preliminary stage in the budget preparation in Chile known as the internal evaluation of budgetary execution within the Ministry of Finance. Associated with this evaluation is a timetable, a work program, work procedures, as well as a methodology. All of the information from each ministry is comprehensively reviewed, including financial and all available performance information – performance indicators, results of evaluated programs and institutions, and the institutional reports on comprehensive management. The purpose of this exercise is to achieve a sound allocation of resources and to bring about necessary management improvements. To date, the exercise has had a positive impact on internal management through organizational learning.15 Resource allocation and objective setting is both a political and a technical exercise. Elected officials will make policy choices reflecting constituency interests. However, a good budgeting process will assure these decisions are fully informed, and on the margin will tilt the decisions towards greater efficiency and effectiveness through providing a number of options for achieving policy objectives. 15 Marcela Guzman, ―The Chilean Experience‖ in Towards the Institutionalization of Monitoring and Evaluation Systems in Latin America and the Caribbean, World Bank, 2006. Available from: http://www.worldbank.org/oed/ecd/docs/proceedings_la_eng.pdf The World Bank 27 Broadly speaking, adding strategic elements into the budget process can occur at a high policy level (in conversations with the Prime Minister, Cabinet, Parliament etc.) or with agencies and departments. Each of these actors play a unique role in the budgeting process and it is the responsibility of BOB to keep actors informed and to leverage their expertise and decision-making power with the underlying objective of producing a result-oriented budget in the available resource envelope. In the case of the UK, the Line of Sight Exercise sought to create a single, coherent financial regime to enhance accountability to Parliament and the public. There was significant misalignment between the different bases on which financial information was presented to Parliament. Government financial documents were published in different formats, and on a number of different occasions during the year, making it difficult to understand the links and inter-relationships between them. In July 2010, the Alignment reforms was approved and passed in the House of Commons.16 Depending on the country, the link between departmental performance and budgetary allocation mainly occurs at two different points in the budgeting process: 1. At the strategic phase – to identify areas which may require less or more resourcing based on priorities and trends17 2. By reviewing current expenditures to assess performance and continued relevance, and determine new budget allocations The review of current expenditures in the UK is called the Comprehensive Spending Review Process (led by the Ministry of Finance), in Australia it is called the Expenditure Review Committee (led by a Ministerial Committee), and in the US it occurs within the budget review process. Box 3 provides a brief overview of the UK, Canadian, and Australian review exercises. In most cases they entail detailed work by the Ministry of Finance, changes in roles for Ministry budget staff, the creation of teams/groups, and enlisting other ministries in providing supplementary documentation. They can be annual (as in the case of the US PART or Australia‘s Expenditure Review Committee), or semi-annual (as in the case of the UK Comprehensive Spending Review Exercise). The objectives of such exercises often include promoting a performance oriented culture, improving prioritization at the agency level, putting pressure on agencies to allocate their resources better, and/or improving the incentives to achieve savings. Ensuing policy decisions will reflect one or more of these objectives, as well as a country‘s economic and political context. Box 3: Improving Prioritization through Spending Reviews (UK, Canada, Australia)  Spending Reviews are centrally driven exercises focused on ways to improve the efficiency of spending across government (i.e. between sectors/programs) and in consideration of differing funding levels. They also serve to help identify and prioritize high priority programs. Thus, they go beyond the typical program evaluations. Amongst OECD countries the development of spending reviews, and the institutional mechanisms that support them, have tended to be driven both by the need to tackle fiscal stress (Canada, Australia, Netherlands) or to better manage a fiscal upturn (UK, France, Korea). The design of spending reviews has varied greatly—being ad hoc or systemic, comprehensive or narrow—to suit both their primary objectives and the country specific 16 HM Treasury: http://www.hm-treasury.gov.uk/psr_clear_line_of_sight_intro.htm 17 For more information on this aspect, please see the email from Shabih Ali Mohib on Resources on International Experience with Budget Ceiling and Strategy Papers. The World Bank 28 institutional arrangements.  Spending Reviews have emphasized the use of performance criteria on program effectiveness and efficiency. The link to performance-informed budget reforms is strong as expenditure prioritization is concerned with clarifying governments‘ key objectives and priorities and directing resources toward high priority areas and away from low ones. It includes the institutionalization of processes to ensure that spending cuts or additions are as rational as possible and that programs that best maintain developmental gains are prioritized. It is thus little surprise that fiscal stress or expansion have been the impetus for the introduction of OBB reforms aimed at improving expenditure prioritization in many countries. For example:  The 1994 Canadian Program Review was a one off exercise, which established a high level special committee under the Prime Minister. The Committee set the performance based guidelines and managed the review process that helped to generate substantial cuts (averaging 21.5 percent across departmental budgets). The process (recently institutionalized) helped lay the foundations for the fiscal management framework that has bolstered the Canadian economy ever since.  In the Australia the sub-Cabinet Expenditure Review Committee (ERC) was established to consider major new policy and savings proposals, and recommend to the Cabinet those proposals that it wants included in the Budget. The ERC uses performance information from Portfolio Budget Submissions and was particularly successful in reducing and re-orientating spending to high priority areas in the mid-1990s.  At the heart of the UK’s performance management system is the Departmental Spending Reviews (SR) agreed between the Treasury and line ministries. Introduced in 1998, the aim of the SR is to review current government priorities, the outcomes being achieved and at what cost. One important outcome of the UK SR is an agreed set of budget forward estimates for the next three years. SRs occur every three years, with three Comprehensive Spending Reviews being conducted across all spending ministries over the last decade. These have looked at the allocation between programs—i.e. allocative as well as technical efficiency—that can create fiscal space for new or higher priority initiatives by cutting lower priority or ineffective programs. 18 In reviewing current expenditures, the guiding questions include, (i) are objectives still relevant/a priority?, (ii) does the program deliver on those objectives?, and (iii) is there overlap/conflict with other programs? Based on the answers to these questions, four alternative scenarios exist: expenditures may continue unchanged; others may be cut (particularly if low priority and unsuccessful); some may be increased (if high priority and successful); or overlapping programs may be rationalized. Savings from such exercises are handled differently between countries and within countries over time. For example, during good economic times, some agencies may retain savings and invest in new initiatives over a defined period. At other times, the budget office may retain savings or insist on the augmentation of budgets for certain programs. The scope of the exercise depends on the capacity of key actors. Some countries take a comprehensive annual approach – likely the most challenging (example: zero-based budgeting), or a comprehensive semi-annual approach (UK). Others take a 18 For a detailed description of the UK‘s 2007 Comprehensive Spending Review, see http://blog- pfm.imf.org/pfmblog/2008/05/united-kingdom.html The World Bank 29 selective approach in which the process applies only to a specific set of expenditures each year. Still others undertake a broader review in conjunction with election cycles. Strategic Budgeting at the Ministry/Department level The section above focused on strategic/performance budgeting at the government-wide level. Next we turn to the ministry/department level. The process of gathering performance information requires the customization of performance measures by sector – due to the difference in the nature of outputs. In certain sectors with clear output measures, it is easier to see how performance information might be tightly linked with budgeting. Direct/formula performance budgeting is mainly applied in certain sectors (e.g. health and education). To do this requires the structuring of activities in programs (known as program budgeting19) and also requires information on unit and input costs. Table 4 presents a number of examples of how countries have used direct (or tight) performance budgeting in programs at the ministry/department level. A major challenge all countries face in implementing performance budgeting is how to handle administrative costs. Some countries use a percentage, others have a flat overhead, others cap it etc.. At the program level, outputs can be tied to budget allocations such as in the per capita funding for local clinics in Chile. 19 For more information on program budgeting and how this related to performance budgeting, see: IMF Working Paper, “From Program to Performance Budgeting: The Challenge of Emerging Market Economies,�2003. Available from: http://www.imf.org/external/pubs/ft/wp/2003/wp03169.pdf and http://blog-pfm.imf.org/pfmblog/2007/10/program-and-per.html The World Bank 30 Table 4: Ministries/departments and programs that use direct performance budgeting Source: OECD (2007), Performance Budgeting in OECD Countries, OECD Publishing. doi: 10.1787/9789264034051-en (47) Once a coherent set of programs is in place, it is important to build in processes to make improvements when performance information reveals problems. The first step is to create a dialogue with the relevant actors to review the information for accuracy and begin the process of analyzing the sources of poor The World Bank 31 performance. Figure 8 presents a decision tree to help identify these sources in the areas of design, staffing, preparation, budgeting etc.. With this information recommendations for strengthening various aspects of program can be made. Figure 8: Decision Tree for sources of poor program performance20 For case studies and further information on performance informed budgeting, the book Performance Budgeting in OECD Countries (OECD 2007) reviews the experiences of eight countries (Australia, Canada, Denmark, Korea, Netherlands, Sweden, United Kingdom, United States) which have developed and used performance information in the budget process over the past ten years.21 Below, two cases of how countries have attempted to introduce strategic elements into the budgeting process – Victoria, Australia, and Korea – are discussed in-depth. 20 Brazil: Planning for Performance in the Federal Government: Review of Pluriannual Planning, World Bank, 2001. 21 OECD (2007), Performance Budgeting in OECD Countries, OECD Publishing. doi: 10.1787/9789264034051-en . The book examines whether performance information is actually used in budgetary decision making and if so, how it is used. It explores the links between resources and results, the impact on improving efficiency, effectiveness and performance, and lessons learned from country experiences in applying the performance informed budgeting approach. It offers guidelines and recommendations on adapting budget systems to promote the use of performance information. The World Bank 32 Victoria, Australia One of the most effective means of connecting goals, outcomes, programs, output, activities and resources is the use of Cabinet budget submissions and published budget papers. In Australia strategic performance based budgeting, also known as Outcomes Based Budgeting, has produced some simple techniques to discipline the process of documenting the desires of government to achieve very large strategic goals and have those very large pieces of work broken down at various levels into outcomes, programs outputs etc… This ensures that the language and the aspirations of different parts of the system are consistent or common. It assists in the assignment of accountabilities for the delivery of components of the system. It also helps identify any duplication of outputs or activities in the system between agencies. The Budget papers and the process that goes with it is also a marker of the changes from year to year as the process is implemented. It is the one place where all parties can track how minor reforms to processes have been introduced or reprioritized. Developing a strong strategic budgeting practice in Victoria required the addition of many new capabilities and skills into budgeting. These include the techniques of business case writing, economic assessments of projects and project management tools. These are now essential to translating priorities into action because they form the empirical basis for the selection of initiatives. As elsewhere, political interests sometimes affect the order of priorities in the list, but even these non-quantifiable elements in decision –making can be given a weighting in a system that may take years to embed but are now commonplace. These techniques are now required in any proposal that wants serious attention by the central agencies and Cabinet for consideration. Central agencies and their budget units play a fundamental role in communicating the policy and budget settings for the forthcoming budget and, where relevant in the out years, for capital expenditure. This includes the major goals coming from the government and the indicative movement in the aggregate for expansion/contraction of State outlays. Agencies prepare business cases and submissions for ongoing and new initiatives, both operating and capital. Agencies prepare the draft monitoring and KPI regimes for existing and proposed initiatives for the relevant budget period and out years if required. At this point it is important to note that in Australia, the amount of detail required for the first consideration of a budget bid is not onerous. The budget process is a multi-stage one with increasing demands made on agencies the closer a bid gets to being a real contender for resources. A bid has to get through the first Cabinet gateway before an agency is asked to undertake the very detailed, time consuming and after expensive research into KPIs, business impacts and the like. The central agency‘s role is at all stages to check for quality, completeness appropriate to the stage of Cabinet consideration. Central agencies help to ensure that ministries adhere to the rules and guidelines when requesting a budget. In situations of non-compliance, the ―punishment‖ systems are reasonably subtle recognizing that mechanisms such as those that operate in the private sector could unreasonably hurt the public e.g. withdrawal of service provision funding in year 2 for reckless overspending in year 1. The main constraints on departments are:  reputational – funding is harder to get in the event that there is evidence that a department has not performed well in discharging its responsibilities;  performance plans and rewards for the head of the department and the senior executive – as the responsible senior managers, jobs may be a risk for serious non-observance of the rules. Similarly bonuses may be at risk; The World Bank 33  public disclosure through the Auditor-General and the parliamentary process – again a reputational risk;  intervention by the Department of Treasury and Finance (Victoria) or the Finance Department (national) to conduct a price review and potentially the imposition of new rules, or the withdrawal of previous freedoms, where non-compliance has been detected. Agencies have the primary role for delivering on the promises set out in the budget. The system is designed to ensure that the failure of an agency to deliver takes account of the factors that are outside its control or influence. These factors are usually set out in the budget process documentation approved by Cabinet. Agencies carry the main responsibility for establishing that public resources are being well used and that projects are being delivered. As a result Australia tends to operate large agencies that retain strong support functions in the areas of evaluations, project management and reporting. These functions are in the main not carried out through central agencies. Agencies may also play a role as lead agency for cross cutting outcomes. Some agencies, such as health, have developed methodologies for some activities that they then share with the remainder of the bureaucracy. This is encouraged to ensure that the central agencies appreciate that not all planning competence resides in the centre. In the event of budget cuts or the identification of redundant activities, the preferred approach is that agencies themselves notify these activities or savings. The incentive scheme to encourage this is still in development. A special Cabinet Committee, the Expenditure Review Committee (ERC) is the key and most powerful entity in determining the form and content of the State Budget in Victoria. It meets throughout the year to establish the standards for budget submission, set the criteria against which bids will be judged, assess the performance in delivery by agencies and Ministers. It has a highly structure review program culminating in the determination of the final annual budget. Along the way it interviews Ministers, agencies and the Treasury on the nature of proposals, the business case for them, their ―fit‖ with the objectives (thrusts) of the government etc. Korea In implementing strategic budgeting, the roles of the central budget authority needed to be changed. In Korea, the strategic planning bureau and the performance evaluation unit have been established to reinforce planning and evaluation capacity within the central budget authority. It even introduced a new organizational motto, ―think tomorrow‖ to emphasize that its mission moves beyond traditional budgeting. In developing program structure, the central budget authority also played a central role in setting standards and monitoring the process although there had been extensive involvement of line ministries. There can be a centralized approach where the central budget authority‘s role is pivotal and a decentralized approach where program structure is developed by line ministries. If you want to use program structure for the purpose of priority setting and resource allocation, the central budget authority should play a role of developing guidelines and communicating with line ministries to ensure each line ministries program structure layers are comparable. Central agencies play an important role in ensuring that ministries adhere to the rules and guidelines at the time they request a budget. In Korea, line ministries usually follow the guidelines issued by the central budget authority, but there are strong efforts to circumvent them. For example, for the budget ceiling agreed at the cabinet meeting, line ministries sometimes strategically behave by underestimating expected costs of new entitlement programs and they force the central budget authority to allow budget The World Bank 34 increase over the ceiling. In this situation, Korean central budget authority‘s hands are tied. However, usually line ministries observe guidelines issued by the central budget authority. Priority setting at the macro-level, which is at sector level in Korea, is done by the working groups consisting of the central budget authority relevant ministry and experts. Policy agendas are developed by the working groups and necessary cost estimates are done by the central budget authority to implement them. Based on the estimates, budget ceilings for each line ministry are discussed at the cabinet meeting and decided. At the micro-level of budget allocation, which is a program and sub-program level, performance information is more directly used by using evaluation results. For example, if particular programs are rated as ―ineffective‖ through program review process, they are more likely to be subject to a budget- cut. For the purpose of strategic budgeting, the strategic plan, the annual performance plan and report are the basic reporting forms. Strategic plans explain each line ministry‘s medium-term strategy, annual performance plan specifies specific goals each line ministry plans to achieve next fiscal year, and annual performance report explains what each line ministry achieved and important factors to consider interpreting the outcomes. In Korea these reporting forms became mandatory with the enactment of the National Finance Law in 2006. C. FINDINGS A. Build more opportunities in the budget process for strategic elements. These include:  Introducing a form of a Budget Strategy Paper at the beginning to set the fiscal rules and support a debate at the highest levels on forward estimates of existing and new spending;22  Introducing a cabinet discussion around the Budget Strategy Paper. For example, in Sweden they conduct a two day cabinet retreat, after which the approved levels are sent to Parliament for ratification. Each ministry is then bound to meet the ceiling set by Cabinet and Parliament; and  Refining the types of reviews necessary – annual vs. multiannual and the process to deal with programs which are not performing well. Maintain caution in overloading the annual budgeting process. Countries use a variety of review techniques and one case in particular is worth mentioning. That of the UK Spending Review Exercises which is conducted every three years. B. Continue the development of sector/ministry ceilings. Moving beyond the current guidance of the aggregate constraints, BOB could further develop sector or ministry/department specific 22 See Malcolm Holmes, Approach Note, ―MTEF in Thailand: Towards Strategic Performance Based Budgeting‖ for further elaboration on the usefulness of a Budget Strategy Paper. For good examples of types of budget strategy papers, see the papers from the Pre-accession economic program (PEP) for EU Accession States. All papers are available here: http://ec.europa.eu/economy_finance/international/enlargement/pre-accession_prog/index_en.htm The World Bank 35 ceilings. Agencies and departments would develop their budget proposals and annual/multi-year plans within these ceilings. As in the case of Sweden, such ceilings could also be agreed upon very early in the budget preparation cycle. This will incentivize agencies to better prioritize budgetary resources, and alleviate the burden of BOB from determining how and where to cut agency budget submissions to fit the expenditure envelope; C. Assess the effectiveness of the PART system and consider refinements. A key question to answer is how can the PART become a more useful basis for making strategic decisions? It will be important to address issues of the quality and credibility of PART results both in terms of the underlying questions and methodology, as well as on the input side - training agencies on how to properly describe outputs/outcomes etc.. This is not a problem unique to the PART. Through the Working Group on the harmonization of the RBM system, it will also be important to clearly define how the PART fits into the overall system and how it adds value. D. Undertake a skills assessment within BOB (formal or informal) with regard to the necessary skills that will be needed to undertake some of the proposed reforms. In some circumstances, the introduction of a Budget Strategy Paper, costed sector ceilings etc..will require upgrading some of the skills of staff. New skills include valuation, policy costing, business case analysis, policy analysis and development, forecasting skills, trends analysis, and evaluation, among others. E. Focus on selective areas/programs that are important and easily subject to performance budgeting and use these to provide a demonstration effect. Since producing meaningful performance information for every program takes time, it may be a good strategy to start with selective areas/programs to demonstrate the impact of performance budgeting. In order to promote the adoption of SPBB, the effort could focus both on programs that are of significant political priority and programs where there is a reasonable foundation, or existing systems and practices, for the production of performance information. Pilots might be conducted in several ministries while the framework is still under development or being refined. These pilots will be used to focus efforts on developing capacity in planning, reporting (e.g. PART), and evaluations. F. More clearly define the roles that key actors can play in the budget process so that performance information, and systems, can be developed to meet their needs. This would include for the Prime Minister, central agencies, line ministers, the Legislature, external audit, and the public etc... Information should be provided in a manner that allows policy-makers to connect it, generally loosely, with planning and managerial actions, with decisions informed by performance measurement and with other sources of information such as affordability, experience, qualitative information and political priorities. The authorities could consider introducing regular user surveys or other mechanisms to gain feedback on the process, particularly with regard to the type, timing, and delivery of performance information. Rationalization of the reporting system based on the needs for information by various actors – this would reduce costs and raise effectiveness. G. Develop a single planning process, allowing for professional/technical inputs and elected official inputs. As noted in the issues section, the two separate central development planning processes - one principally managed by technocrats (the NESDP) and the second principally managed by politicians (the GAP) - suggests that the integration of plans into one coherent The World Bank 36 direction for the country does not necessarily emerge naturally. For an effective national planning process, the consolidated inputs from both technocrats and elected officials are necessary. H. Strengthen planning capabilities at the sub-national level by establishing channels of technical assistance. Based on discussions, the planning process is more ad-hoc and less structured at the sub-national level. According to the Ministry of Interior, there are no strategic plans for the 19 provincial clusters and planning at the local level is inadequate. The World Bank 37 V. Creating Incentives for Performance This section considers the types of incentives that can be used to encourage performance improvements. Incentives can be used to influence organizational or individual performance. This section focuses particularly on the costs/benefits of using various incentives at the individual level. This includes both monetary and non-monetary incentives, such as performance pay, reward programs, and access to education, among others. _________________________________________________________________________ A. ISSUES The Government of Thailand has established, or is in the process of establishing, a number of both monetary and non-monetary performance incentives throughout the public sector. These include:  The HR Scorecard – each year OCSC allow ministries to nominate a person/project which is seen to be innovative/best practice in terms of career development. OCSC asks the Prime Minister to give the award  OCSC provides training awards for those executives of agencies and those who manage HR  OPDC Performance Bonuses at the departmental level  OCSC Individual Performance bonuses  OPDC Public Management Quality Award (PMQA) – each year OPDC nominates individuals/department who achieved high scores in the section four - Organizational Management - in the Balanced Scorecard.  Awards for good performance by the Prime Minister – however, the emphasis is less on results and more on behavior  Giving back portions of budgets saved to agencies and/or granting regulatory flexibility  Attending training courses for executives However, the overlapping of these incentives leads to confusion for line ministries and departments and may send mixed signals to public servants. A few of issues identified in discussions with the relevant agencies include: i. Various incentives with similar purposes are handled by several agencies. The most notable example is that of the OCSC providing individual performance bonuses, while the OPDC provides bonuses at the departmental level. This creates confusion for line ministries, departments, and other agencies and sends mixed signals to employees. The use of multiple incentive tools can increase motivation, but uncoordinated incentives may send a confusing message to staff – where some receive rewards in certain areas, but others do not. The World Bank 38 ii. Monetary and non-monetary incentives are not strategically used. Thailand uses a combination of monetary (e.g. departmental/individual performance pay) and non-monetary incentives (e.g. access to training courses), but it is not clear that a strategic approach to using these tools across the public sector exists. iii. Performance improvements do not directly lead to more managerial control over resources. Many countries often link improvements in performance with greater autonomy and flexibility in managing resources at the activity level, as in the cases of Sweden, Denmark, Canada, and Australia. In Denmark, in particular, agencies are given block appropriations to cover running costs and salaries, giving agencies substantial flexibility to allocate resources.23 In Thailand, it is not clear that improvements in ministry or departmental performance leads to enhanced flexibility. iv. The transparency of the process for the selection and reward of individual performance through monetary and non-monetary incentives is lacking. In the case of OPDC departmental bonuses, managers have discretion on how to allocate the money. Some departments allocate money according to good performance, but other departments may use different criteria. In the case of the OCSC individual performance bonuses, many employees raised issues of equity and lack of transparency when the system changed from a merit- fixed step increase to a percentage increase based on performance. Currently, supervisors determine the percent increase (0-6%) based on performance, but the OCSC does not have a database to track individual pay increase over time. There is no data on how these bonuses are being distributed. v. The individual performance appraisal system appears to require strengthening. Firstly, there are on-going training programs for directors and middle management on performance management, but the system has not been rolled out to all civil servants. Secondly, in establishing individual KPIs, managers usually set the KPIs with little discussion or negotiation with the employee (despite guidance which may be handed down). Thirdly, OCSC does provide guidance on how to flexibly set KPIs for different job functions, and from discussions, it seems that many people have little understanding on how to set KPIs and that attention by directors and managers to the individual performance appraisal system is low. vi. An internal government culture of collective benefits exists, but the current performance pay system is focused on individuals. Individual performance pay provides a payment to a particular staff based on his/her performance. Individual performance pay may be a useful tool to increase incentives for 23 OECD, In Search of Results, PUMA, 1997. The World Bank 39 individuals in some cases. However, it relies on two criteria: (i) being able to distinguish individual contributions and (ii) on a culture which places value on individual effort. From discussions, it seems that both of these criteria are not being met. First, the individual performance appraisal system is weak and within the Thai government a strong emphasis is placed on collective, rather than individual performance. B. LESSONS FROM INTERNATIONAL EXPERIENCE What is an incentive? Generally, an incentive is a management system provided through payments, concessions, awards and so on that: (a) encourages harder work or a particular choice of work or (b) something that arouses feeling, or calls one to action; an exciting cause or motive; a ‗spur‘.24 Incentives can be designed at both the organizational and individual levels and take the form of either monetary or non-monetary incentives. 1. Types of Incentives i. Organizational Incentives Incentive mechanisms are important tools for governments to change organizational behavior to achieve their outcomes and perform more efficiently and effectively. Organizational incentive mechanisms include: a. Punishment or negative enforcement b. Performance-related pay at the organizational/department level c. Changes in budget of institutions d. Disclosure of information to citizens e. Provision of information to line managers Many countries provide organizational information to line managers or delegate internal controls over to the department/unit level, in an effort to improve organizational performance. Countries may also disclose information to the public so that organizations can be held accountable for achieving their outcomes. ii. Individual Incentives Creating a set of incentives to encourage and reward employees to work at their optimal levels to accomplish objectives is crucial for providing effective public service delivery and achieving organizational goals. However, it is critically important that individual incentives are designed strategically in a way closely aligned with organizational goals and outcomes. It is widely acknowledged that individuals are both extrinsically and intrinsically motivated to work. Many governments have established a set of both monetary and non-monetary incentives throughout the civil service to encourage good performance. This is based on the critical assumption that performance 24 Oxford English Dictionary Definition. Accessed on Oct. 25, 2010: http://dictionary.oed.com/cgi/entry/50114143?single=1&query_type=word&queryword=incentive&first=1&max_to_show= 10 The World Bank 40 based rewards increase individual employees‘ motivation. Yet, because little evidence is available to verify this assumption, it is important to design the form of individual rewards carefully. 2. Mode of Incentives i. Monetary Incentives The purpose of monetary incentives is to financially reward employees who perform especially well and increase their motivation to achieve higher levels of performance. Monetary incentives include performance-related pay (PRP), where financial rewards are linked with the level of departmental or individual performance, as well as various types of bonuses. PRP became a popular tool in the 1980s for motivating employees in the private sector to improve their productivity. PRP is based on the following assumptions: a. organizations can accurately measure individual, team/unit or organizational outputs;25 b. individual and team/unit outputs contribute to organizational performance; and c. pay can be administered in a way which capitalizes on its expected incentive value for potential recipients.26 Trends in the use of monetary incentives from the OECD indicate that the use of such incentives is generally increasing in OECD countries, yet it represents rarely more than 15% of total pay for individuals. Two-thirds of OECD countries have implemented PRP or are still in the process of developing the system. However, there are large variations in the degree to which PRP is actually applied throughout civil services. Only a handful of OECD countries have developed an extended, formalized PRP policy (Denmark; Finland; Korea; New Zealand; Switzerland; the UK). Linking monetary rewards to performance appears to be a sound idea, but country experiences indicate that its implementation is complex and difficult. A handful of OECD studies concluded that many of the PRP schemes failed to satisfy key motivational requirements for effective performance pay because of its design and implementation problems, but also because performance assessment is inherently difficult in the public sector.27 This difficulty stems from attempting to measure individual and organization outputs and outcomes in diverse institutional settings. Many studies have concluded that the impact of PRP on performance is limited, and can in fact be negative.28 More recently, experimental evidence suggests that the effectiveness of PRP in practice is slim and that it may also diminishes other motivations for performing well. Economists at the London School of Economics conducted 51 experimental studies to examine if providing monetary rewards led to an improvement in overall performance. They found the evidence that PRP often does not encourage 25 See Table 1 on Wilson‘s typology for further information. 26 ―Performance-Related Pay Policies for Government Employees‖ OECD, 2005. 27 Annex A. Performance-related Pay Policies Across 12 OECD Countries ―Performance-Related Pay Policies for Government Employees‖ OECD, 2005. Paying for Performance: Policies for Government Employees. Policy Brief. OECD May 2005. 28 For further information see ―Performance-Related Pay Policies for Government Employees‖ OECD, 2005. Country cases include Denmark, Finland, France, Germany, Hungary, Italy, Korea, New Zealand, Spain Sweden, Switzerland, and the UK. The World Bank 41 people to work harder and sometimes monetary incentives could reduce an employee‘s intrinsic motivation to complete a task, which could result in a net reduction of motivation across a team or organization. The study showed that this was particularly true when ethical or other reasons for complying with workplace social norms such as fairness, incentives on engagement, cooperation, social preferences, social status, and reciprocal behavior of employees are strong.29 Another study conducted by Dr. Dan Ariely, in which individuals received cash payments that varied in the amount relative to the level of achievement in different set of games, revealed that performance of participants was always the lowest in high payment conditions when compared with the low and mid- payment conditions. 30 The study suggests that this may be because cash rewards went beyond a certain threshold and raised motivation to supra-optimal levels, which resulted in perverse effects on performance (a common example of this is stage freight). Finally, monetary incentives can also create problems in the workplace particularly in a culture which emphasizes team work or collective engagement. Monetary incentives may encourage employees to perform to receive the money rather than working for the collective goals/objectives of the organization. This can transform good relationships among co-workers into competitors, undermining one another‘s work, and disrupt an otherwise harmonious workplace environment.31 Turning from the literature to the practical experiences of using performance pay in the public sector reveals a number of interesting findings. In spite of the aforementioned challenges, many OECD countries have implemented various PRP schemes. A 2005 OECD report Performance-related Pay Policies for Government Employees, contains detailed case studies on pay for performance schemes implemented in Canada, Denmark, Finland, France, Germany, Hungary, Italy, Korea, New Zealand, Spain, Sweden, Switzerland, the UK, and Chile.32 In this report, as well as others, a number of considerations or prerequisites have been identified that support the introduction of performance related pay: - the existence of a supportive legal framework; - a strong personnel performance assessment system; - good management and administrative capacity and communication; - adequate monitoring systems; and - good records management (for HR, salary, bonuses). It is important for countries to strengthen the foundations for introducing performance pay before investing too much in actual pay/bonuses throughout the system. Management information systems designed to monitor and track bonus distribution improves the transparency surrounding the process. If not, the introduction of such a scheme may increase problems linked to trust and lead to corruption and 29 See http://www2.lse.ac.uk/newsAndMedia/newsArchive/archives/2009/06/performancepay.aspx 30 Ariely, Dan et al. 2009. Large Stakes and Big Mistakes. Review of Economic Studies 75, 1-19 31 Kohn, Alfie 1999 Published by Rewards The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes. Mariner Books; 2 edition. 32 OECD (2005), Performance-related Pay Policies for Government Employees, OECD Publishing. doi: 10.1787/9789264007550-en The World Bank 42 patronage. The culture of an organization must also be taken into account in designing the system. In some cases, group/team rather than individual incentives may be more effective/acceptable to staff. Such team-based incentives can emphasize cross-department or ministry collaboration; working across organizational boundaries to resolve problems or make progress on government priorities. There is no ‗right‘ size for a performance payment, but in general it should be a limited part of the base salary. In a report by Matkinson, 2000 – the author suggests that rewards be a minimum of 5% of the base salary.33 Table 5 provides an overview of the form and size of merit increments and bonuses across a select number of OECD countries (Canada, France, Finland, Korea). Merit increments are permanent increases to base pay. In other words, they are a permanent award until retirement, factoring into pension payments in systems where pensions are a function of pay levels while employed. On the other hand, bonuses are a one time payment and do not effect base pay or pensions. OECD experience suggests that there is no one-size-fits-all approach to allocating bonuses. Some countries do it on an individual basis, others on a team basis (or a combination thereof). Bonuses can be set centrally, or determined by individual departments (New Zealand, Denmark). Some target senior managers, others target mid or lower level employees. The range between bonuses is wide – from 1.7% (Finland) to 100% of base salary (Czech Republic). Table 5: Form and maximum size of individual PRP payments Country Merit Increments Bonuses Canada 10-25% bonus lump 5% per year, up to 3 years sum PRP for top level civil servants (director's France level) in six pilot ministries up to a maximum of 20% of the base salary) Team basis, Range Finland Max merit increase is between agencies was between 25-50% 1-8.3% Mid to lower level Korea employees. Varies from 100% to 40% of the Up to 7% of half of an annual monthly base salary per salary year. Source: Adapted from OECD (2005), Performance-related Pay Policies for Government Employees, OECD Publishing. doi: 10.1787/9789264007550-en 33 Makinson, John (2000), ―Incentives for Change: Rewarding Performance in National Government Networks‖, Public Services Productivity Panel, HM Treasury, London, www.hm-treasury.gov.uk./media/41E1F/240.pdf The World Bank 43 ii. Non-monetary Incentives Non-monetary incentives are also designed to reward employees for good performance, providing recognition and encouragement. Examples of non-monetary rewards include increasing levels of responsibility (new appointment and promotion), assignment to more professionally challenging or higher profile assignments, providing access to opportunities to learn and advance (through training programs, scholarships, and other educational programs), allowing flexible work hours, and developing a pleasant work environment. Such rewards increase employees‘ sense of accomplishment and social recognition in the organization and promote pride in one‘s work. Non-monetary rewards motivate individuals through their intrinsic value and translate to a potential increase in productivity and innovation, as well as increased loyalty to the organization.34 This section is divided in incentives related to appointment and promotion, access to educational programs, and other aspects. a. Appointment and promotion Providing opportunities for employees to advance their career or expand job responsibilities can promote employees‘ motivation. An employee‘s appointment to a higher position is recognized as ―advancement‖ from colleagues and others, giving the employee a higher sense of accomplishment for his/her responsibilities. This will also have spill-over effects in motivating other employees to work more efficiently and effectively for the organizational goals and outcomes. Nomination of such employees requires a transparent selection process based on specific criteria, since others regard them as models/examples. The shift from life time appointments to fixed-term, position-specific contracts within the public service is also an example of an attempt to motivate public service employees to perform well, by bringing them up for review every few years. b. Access to educational programs Many studies show that providing opportunities for employees to learn through access to training programs, scholarships, and other educational programs increases employees‘ incentives to gain new skills and use it to perform better. Such opportunities also include providing travel awards and paid leave. Such non-monetary opportunities and rewards are perceived to carry more value than the equivalent cash value. Employees consider such opportunities and rewards to be ―worth the effort‖ since receiving rewards are often associated with the higher public recognition from their colleagues/others and give higher sense of accomplishment. c. Other aspects Associated with the above two opportunities, intangible benefits such as providing more challenging assignments, flexible work hours, a pleasant work environment, job security (tenure), and social recognitions and privileges may also increase an employee‘s motivation to perform better. 34 For further details of country case studies, please refer to the GET Note: Public Sector Awards Programs Non-monetary Awards for Public Sector Programs and Institutions: Survey of Selected International Experience, April 2010 The World Bank 44 C. FINDINGS Monetary and non-monetary incentives vary in purpose, effectiveness, and appropriateness, depending on the context. It is crucial for relevant agencies to diagnose weaknesses and strengths of the existing incentive systems in Thailand so that such tools can be strategically used to improve employees‘ and organizational performance. A. Streamline existing incentive tools. The multiplicity of incentives developed for similar purposes, but managed by different agencies, may send mixed signals to employees and undermine the overall effectiveness of these tools. The OPDC and OCSC should consider mapping out the various incentive tools including PMQA, departmental/individual performance pay, HR scorecard, and discuss the purpose of the tools and ways to streamline overlapping tools. Simplified incentive tools will prevent from sending mixed messages to employees. An initial attempt to do this is available in the matrix of RBM tools in Appendix E. Box 4 provides a number of questions which might be answered for each incentive. B. Redesign existing monetary incentives. Consider focusing primarily on team (unit or Box 4: Screening Process to Evaluate department) based, rather than individual, Incentives performance pay. If outputs are more collective than individual, group incentive schemes are 1. What is the intended purpose of more equitable. Moreover, the individual the incentive? appraisal system needs substantial work in 2. Who does the incentive target? order for it to serve as a legitimate basis to 3. Is the incentive aligned with make performance pay assessments. The organizational goals/outcomes? implementation should be coordinated with 4. Are the criteria for identification staff and unions and appropriate planning and of good performance clear and is resourcing must be assured. Generally, group the process transparent? based performance pay systems are easier to 5. Is there any way to determine administer, but may dilute the incentives for whether the incentive achieves its high individual performers and allow poor intended purpose? performers to benefit more than they deserve. 6. What are the potential One way to overcome this is to allow teams to costs/unintended consequences distribute the pay increase among themselves. that might emerge as a result of Conduct regular monitoring and evaluations of the incentive? the scheme to see if it is having the intended 7. Do employees perceive it as effect. equitable, more objective, and genuinely rewarding C. Closely link incentive tools with improving performance? employees’ and organizational performance. Source: World Bank team Assess the strengths and weakness of existing methods to assess individual and organizational performance. It is important to design individual incentive tools that link employee performance to that of organizational goals and outcomes and track progress over time. D. Consider using more non-monetary incentives. Thailand largely emphasizes performance bonuses to improve employees‘ and departmental performance. Academic studies suggest The World Bank 45 that using monetary incentives alone does not necessarily increase individual performance. Agencies could design an incentive system with a combination of monetary and non- monetary incentives. E. Strengthen the individual performance appraisal system. This system provides the foundation upon which performance decisions will be made. This may entail improving the design/clarity of job or position descriptions and results agreements. The training programs for directors and middle management needs to be rolled out across the civil service. Information on how to have conversations with managers, how to set KPIs, etc.. needs to be disseminated widely so that staff are aware of how the system works. F. Consider recognizing positive organization performance by introducing more flexibility in budget and human resource management. Countries often link improvements in organizational performance with greater autonomy and flexibility in managing resources at the program/activity level as long as a certain threshold of capability is met. This could include the relaxation of some budgeting and human resource management constraints. G. Standardize the criteria used to recognize performance. It may be a good idea for OPDC and OCSC to standardize the criteria for assessing performance and allocating bonuses as part of the performance appraisal system. This will enhance the transparency and legitimacy of the process. H. Strengthen the information systems surrounding bonus distribution. The management information systems (MIS or HRMIS) must maintain accurate records of base pay, bonuses, etc.. by staff member and this information should be made available to OPDC/OCSC. The information can be used to monitor the fairness, equity of allocations, and their correlation with performance, to assess whether the system is working well. I. Establish mechanisms to get feedback on the system. This may include surveys of managers and staff to understand the perceptions of fairness, transparency, and equity of monetary and non-monetary awards. This will give agencies the information they will need to fine-tune the system. A few general principles are useful when implementing both monetary and non-monetary incentives. These include a fair and transparent selection process for rewarding recipients, variation and limitation of the types and frequency of awards given, attention to making sure that the rewards reflect the commitments and values of the organization, and finally that the program is supported by an effective communications plan.35 Getting the incentives right is more of an art than a science and it will be important to experiment to see what is working. At this point, pilot various types of incentives and try and determine, through interviews and surveys, whether they meet their intended objectives. 35 GET Note: Public Sector Awards Programs Non-monetary Awards for Public Sector Programs and Institutions: Survey of Selected International Experience, April 2010 The World Bank 46 The World Bank 47 VI. Summary Recommendations The move to harmonize the RBM system is expected to:  Better align the efforts of the central agencies;  Remove duplication and better manage the information from agencies;  Provide a framework for eliminating overlapping and redundant reporting requirements; and  Provide a basis for better integrating monitoring and evaluation systems. The preceding sections outlined a number of recommendations for Thailand in moving forward. The recommendations are presented here in sequential order to emphasize the need to establish a coordination mechanism first, to then review the overall system, and start to implementing findings. This section summarizes these recommendations for quick reference. 1. Recommendations on Thailand’s Result Based Management System: To improve the coherence of the RBM system: A. Convene a forum/steering committee (this can be existing or new, as long as it includes the relevant actors) to review the matrix and comprehensively document the various tools and their purposes, with a view to identifying duplication and areas for integration particularly in terms of underlying objectives/questions. Draft a clear Terms of Reference for committee members. A chair should be identified at the onset. B. Conduct a series of focus groups/surveys/meetings with individuals from ministries, departments, etc.. to get a better sense of the transaction costs they face in reporting, what information they already collect and what new information they need to collect as a result of reporting requirements. C. Increase the frequency of meetings of the ad-hoc subcommittee on Public Sector Audit and Evaluation to increase the lines of communication between the various central actors. D. Improve the alignment of incentives, reporting, and timeframes so that information is available to those who can best use it. Streamline reporting to that which is essential. To revise RBM elements: E. Develop agreement on a set of definitions that would be consistent across all agencies for all key terms used in the RBM systems (indicator, KPI, efficiency, effectiveness etc..) and publish this alongside tools and communications. This would be highly useful to do prior to discussions on duplication. F. Reduce the number of KPIs collected at a central level through a number of different channels: (i) by formulating a small number of high-level targets the Cabinet/PM wants to keep track of, (ii) by determining what indicator information is most useful at the central level, and (iii) by integrating KPIs from the BSC and the PART. G. Match the roles of users in the RBM system to the reporting requirements, to ensure that those who receive the data can actually use it. This may mean that fewer KPIs are reported to the central level, with more reported at the ministerial level. H. Document and analyze the timing & frequency of the various reporting and process requirements of the RBM system –including that of evaluation - and map this against key points The World Bank 48 in the budgeting and planning cycles to determine whether better timing can be achieved between the reporting and analysis of RBM data. This would increase the usefulness of performance information in these on-going processes. In addition, consider the appropriate frequency of reporting for the relevant users. To leverage Monitoring and Evaluation: I. Assess the informal and formal follow-up functions/processes for all audit and evaluation findings/recommendations and strengthen weak linkages between key actors. This may additional training such as Parliamentary training on how to follow-up on audit findings. J. Analyze the various types of evaluations being done by agencies to determine if there are gaps or if resources are better used centrally to manage certain types of evaluations. Additional observations: K. Strengthen the linkages of DOLA‘s RBM tools with the overall RBM system and assess from the standpoint of local authorities how reporting and data management is likely to function. It will also be important to use similar terminology for the RBM system. L. Consider using the internal controls system for quality assurance (through verification) of the performance information collected. 2. Recommendations on linking performance with the budget and planning: A. Build more opportunities in the budget process for strategic elements. These include: o Introducing a form of a Budget Strategy Paper at the beginning to set the fiscal rules and support a debate at the highest levels on forward estimates of existing and new spending;36 o Introducing a cabinet or high level discussion around the Budget Strategy Paper; and o Refining the types of reviews necessary – annual vs. multiannual and the process to deal with programs which are not performing well. B. Continue the development of sector/ministry ceilings. C. Assess the effectiveness of the PART system and consider refinements. D. Undertake a skills assessment within BOB (formal or informal) with regard to the necessary skills that will be needed to undertake some of the proposed reforms. E. Focus on selective areas/programs that are important and easily subject to performance budgeting and use these to provide a demonstration effect. F. More clearly define the roles that key actors can play in the budget process so that performance information, and systems, can be developed to meet their needs. G. Develop a single planning process, allowing for professional/technical, as well as elected official inputs. 36 See Malcolm Holmes, Approach Note, ―MTEF in Thailand: Towards Strategic Performance Based Budgeting‖ for further elaboration on the usefulness of a Budget Strategy Paper. For good examples of types of budget strategy papers, see the papers from the Pre-accession economic program (PEP) for EU Accession States. All papers are available here: http://ec.europa.eu/economy_finance/international/enlargement/pre-accession_prog/index_en.htm The World Bank 49 H. Strengthen planning capabilities at the sub-national level by establishing channels of technical assistance. 3. Recommendations on creating incentives for performance: A. Streamline existing incentive tools. The OPDC and OCSC may consider mapping out the various incentive tools including PMQA, departmental/individual performance pay, HR scorecard, and discuss the purpose of the tools and ways to streamline overlapping tools. This could begin with reviewing the matrix of RBM tools on incentives in Appendix E. B. Redesign existing monetary incentives. C. Closely link incentive tools with improving employees‘ and organizational performance. D. Consider using more non-monetary incentives. E. Strengthen the individual performance appraisal system. F. Consider recognizing positive organization performance by introducing more flexibility in budget and human resource management. G. Standardize the criteria used to reward performance. H. Strengthen the information systems surrounding bonus distribution. I. Establish mechanisms to get feedback on the system. The World Bank 50 VII. Appendix A. TERMS OF REFERENCE FOR TECHNICAL ASSISTANCE Terms of Reference for World Bank Technical Assistance to the Office of the Public Sector Development Commission (OPDC), Thailand I. Background: The Royal Thai Government is making a considerable effort to improve public sector performance, as reflected in the Royal Decree on Good Governance 2003, the Thai Public Sector Development Strategic Plan 2008-12, and the new Civil Service Act of 2008. This is reflected in their adoption of a Results Based Management (RBM) approach to public sector reform. The themes of the Thai Public Sector Development Strategic Plan (2008-12) focus on developing a civil service that is flexible, adaptive and responsive to the needs of citizens. Specifically, service quality improvement and developing high performing organizations are two of the plan‘s main themes. Within this general framework, there are number of RBM initiatives underway throughout the public sector. These include, the Office of the Public Sector Development Commission‘s work on annual performance agreements across various levels of government based on the balanced scorecard approach and Public Management Quality Assurance (PMQA). It also includes the Bureau of the Budget‘s implementation of Strategic Performance Based Budgeting (SPBB), including its Performance Assessment Rating Tool (PART). In addition, the implementation of the provisions of the new Civil Service Act relating to the merit principle through performance management (including appraisal) of individual civil servants and the development of an HR scorecard for individual MDAs increasingly reflects results based approach.37 II. Objectives and Scope of Work: The objective of this engagement is twofold: (i) the World Bank team will conduct two workshops on the ―Healthy Thailand‖ project and on Results Based Management to be held between September 13 -24, 2010 in Bangkok, Thailand; and (ii) based on the findings of the workshops, interviews with relevant authorities, and international experience, the team will present recommendations on how to sustainably implement performance reforms in Thailand. The following tasks will be undertaken: 1) The expert team will study any material sent by the authorities on the institutional structure and challenges to performance management ahead of their travel to Thailand. 37 Adapted from David Shand, ―Results Based Management in Thailand,‖ Discussion Note, March 2010. The World Bank 51 2) The expert team will engage with the authorities during a few days of fact-finding immediately preceding the workshops. This should assist in informing the team of the current institutional structure, challenges and constraints. 3) The expert team will deliver two one-day workshops to the authorities for senior Thai government officials from the central agencies (e.g. OPDC) and selected line ministries. 4) The expert team will prepare and present a 10-15 page summary report after the workshops, which will summarize the proceedings of the workshops and lay out the key recommendations. Through the workshops and the accompanying discussions with the OPDC, BOB, OCSC and relevant agencies‘ representatives, the World Bank team is expected to share their experience on the following questions of interest to the authorities: i) How can the links between politicians, bureaucrats, and citizens be enhanced to improve public sector performance? See Annex 1 for additional questions. ii) What are the main implementation challenges countries face when implementing a Results Based Management system? a) Can output/outcome indicators in the budget document be combined with the OPDC Balanced Scorecard (BSC) as they relate to measuring the achievement of strategic plans and missions (Perspective 1: Effectiveness)? b) Can the Government link the results of performance evaluation based on the Balanced Scorecard into the annual budget allocation process/cycle? iii) How can the Government link the Results Based Management system with performance management by cascading organizational targets (using the four perspectives in the BSC: effectiveness, quality of service, efficiency, and organizational development) to the individual level as the basis for performance appraisal? Is there any correlation between a sum of merit increase budget setting and performance evaluation based on BSC‘s scores? Do any other countries employ the approach that links a merit increase with performance evaluation based BSC‘s scores? Note: In fact, in a Thai public sector, the amount of money that each agency receives for merit increase on each year round (every April 1 and October 1) is equal to 3% of the total amount of civil servants’ salary within its agency. Noticeably, there is no relationship with the performance based on BSC’s scores. III. Expected outputs The following are the expected outputs of the technical co-operation: 1) Two one day workshops for senior Thai government officials from the central agencies (e.g. OPDC, BOB, OCSC and relevant agencies‘ representatives) and selected line ministries. The World Bank 52 2) Report on the findings of the workshops and key recommendations. IV. Logistics Staff: Bill Dorotinsky, Joanna Watkins, Miki Matsuura B. WORKSHOP PROGRAM Friday, September 24, 2010: Morning Session Senior Management Meeting and Discussions 10:00 – 10:15 a.m. Welcome and Introductions – Mathew Verghis, Lead Economist 10:15 - 11:15 a.m. International Experiences with Results Based Management Systems – William Dorotinsky, Leader, World Bank Public Sector Performance Global Expert Team  Overview of results based management systems – variations on a theme  International experience and challenges faced in implementing results based management systems in the public sector  Initial considerations for Thailand’s Results Based Management System 11:15 – 12:00 p.m. Discussion 12:00 – 13.30 p.m. Lunch at Cuisine, Pullman Hotel Afternoon Session Technical Meetings and Discussions 12:00 – 13.30 p.m. Lunch at Cuisine, Pullman Hotel 13:30 – 14:15 p.m. Overview of Afternoon Session – William Dorotinsky, Leader, World Bank Public Sector Performance Global Expert Team - Overview of results based management systems and key considerations for designing effective incentives, tools and processes. 14:15 – 15:00 p.m. Brainstorming Sessions – 5 groups - Each group will be assigned questions related to their experience with implementing a Results Based Management approach in Thailand. See handout for further details. 15:00 – 15:20 p.m. Tea/Coffee Break The World Bank 53 15:20 – 15:50 p.m. Presentation by Groups 15:50 – 16:30 p.m. Suggestions for Thailand’s RBM system – Group Discussion 16:30 – 17:00 p.m. Concluding Remarks – William Dorotinsky 17:00 – 18:00 p.m. Cocktails C. WORKSHOP PARTICIPANTS Results Based Management - Morning Session No. Name Position Organization 1 Mr.Avoot Wannvong Deputy Secretary General OPDC 2 Mrs.Supannee Pairuchvet Senior Executive Director of Public Sector OPDC Development 3 Miss Darat Boripanthakul Executive Director of Monitoring and OPDC Evaluation Bureau 4 Mrs.Vunnaporn Devahastin Executive Director of Change Management OPDC Suthapreda and Innovative Development Bureau 5 Mrs.Kittiya Khampee Senior Specialist of Public Sector OPDC Development 6 Mrs.Darunee Phaosuwan Director of Human Resource Management OPDC Group 7 Mrs.Areepan Charoensuk Director of Change Management Group 3 OPDC 8 Miss Surungluk Meakhaumnouichai Director of Public Sector Monitoring and OPDC Evaluation Group 9 Mrs.Sirinate Klaharn Director of Special Executive Reward Group OPDC 10 Miss Jatuporn Poolkaew Director of Academic Monitoring and OPDC Evaluation Group 11 Mrs.Somporn Sompadung Director of Joint KPIs Group OPDC 12 Miss Usa Punyawadee Director of Provincial and Local Monitoring OPDC and Evaluation Group 13 Mr.Tanasak Mangkarotha Director of Provincial Administration Group OPDC 14 Mr.Vichit Sangthongloun Director of Local Administration Group OPDC 15 Dr. Maka Phoochinda Human Resource Officer OCSC 16 Ms. Maneerat Phasittipol Human Resource Officer, Expert Level OCSC 17 Ms. Chiraporn Tantiwong Director of Budget Evaulation Office BOB 18 Mr. Thanasal Vidheecharoen Budget Officer BOB Results Based Management - Afternoon Session No. Name Position Organization 1 Mr.Avoot Wannvong Deputy Secretary General OPDC 2 Mrs.Supannee Pairuchvet Senior Executive Director of Public Sector OPDC Development 3 Miss Darat Boripanthakul Executive Director of Monitoring and OPDC Evaluation Bureau The World Bank 54 4 Mrs.Vunnaporn Devahastin Executive Director of Change Management OPDC Suthapreda and Innovative Development Bureau 5 Mrs.Kittiya Khampee Senior Specialist of Public Sector OPDC Development 6 Mrs.Darunee Phaosuwan Director of Human Resource Management OPDC Group 7 Mrs.Areepan Charoensuk Director of Change Management Group 3 OPDC 8 Miss Surungluk Meakhaumnouichai Director of Public Sector Monitoring and OPDC Evaluation Group 9 Mrs.Sirinate Klaharn Director of Special Executive Reward Group OPDC 10 Miss Jatuporn Poolkaew Director of Academic Monitoring and OPDC Evaluation Group 11 Mrs.Somporn Sompadung Director of Joint KPIs Group OPDC 12 Miss Usa Punyawadee Director of Provincial and Local Monitoring OPDC and Evaluation Group 13 Mr.Tanasak Mangkarotha Director of Provincial Administration Group OPDC 14 Mr.Vichit Sangthongloun Director of Local Administration Group OPDC 15 Miss Monvadee Juntima OPDC 16 Miss Chotima Sanguanphant OPDC 17 Miss Kritsana Kaewduang OPDC 18 Dr. Phaiboon Phosuwan Policy and Plan Analyst Ministry of Interior 19 Ms. Sasinee Phothong Policy and Plan Analyst Ministry of Interior 20 Mr. Somdet Choontanom Chief, Public Sector Development Group Ministry of Natural Resources and Environment 21 Dr. Rutaichanok Jingjit Trade Officer, Bureau of Public Sector Ministry of Commerce Development 22 Mr. Pornpong Tayanukorn Senior Budget Officer BOB 23 Ms. Malai Sasinvanich Budget Officer BOB 24 Ms. Doolyarat Nitikritanusorn Secretary of Public Sector Audit and Ministry of Tourism and Evaluation Committee Sports 25 Mr. Udom Oatyimprai Chief of Monitoring and Evaluation Ministry of Tourism and Sports 26 Ms. Prasopsuk Songpasuk Plan and Policy Analyst, Office of NESDB Development Evaluation and Communication 27 Ms. Nisawan Pitchdumrong Plan and Policy Analyst, Office of NESDB Development Evaluation and Communication 28 Ms. Achara Wongse-ek Department of Local Administration no. Name Position Organization 1 Mr.Bill Dorotinsky Sector Manager World Bank (Global Expert Team) 2 Ms.Joanna Watkins Consultant World Bank (Global Expert Team) 3 Ms.Miki Matsuura Consultant World Bank (Global Expert Team) The World Bank 55 4 Mr.Robert Taliercio Lead Economist World Bank 5 Mr.Shabih Mohib Senior Economist World Bank 6 Ms.Nattaporn Triratanasirikul Economist World Bank 7 Mr.Robert Boothe Consultant World Bank The World Bank 56 D. MISSION SCHEDULE WB GET Team: Results Based Management and Intergrating Performance Management In the Public Sector September 15-24 Date Time Description Person/Organization Venue Sept. 15 11.30 Lunch with OPDC OPDC team Royal Princess Hotel Sept. 16 14.00 GET team meeting on Healthy Thailand (exclude OPDC team OPDC researcher) 11.00-12.00 Meeting with Mr. Arkhom Termpitayapaisith, Deputy National Economic and Social Soonthorn Room, 3rd Fl., Secretary-General Development Board (NESDB) Sept. 20 15.30-17.30 Workshop on Healthy Thailand Novotel Hotel 18.00 Dinner with SG, OPDC Sept. 21 14.00 Discussion with Bureau of Budget (monitoring and Bureau of the Budget (BOB) team 2nd Fl., Siripraphasit Meeting room, evaluation ; budget strategy and policy groups) BOB Building 9.30 Meeting with Department of Local Administration, Mr. Dussadee Suwatvitayakorn, 6th Fl., Building 4, DOLA Ministry of Interior Director, Decentralization Promotion Sept. 22 Division 14.00 Discussion with Bureau of Budget (strategic center team) Mr. Dusit Khamasakchai, Deputy 2nd Fl., Siripraphasit Meeting room, Director and team BOB Building Sept. 23 9.30 Meeting with Office of the Civil Service Commission Dr. Surapong OCSC Nonthaburi Sept. 24 9.30-12.00 Workshop on RBM:GET-PSP team will present Senior management of the BOB, Alpha Room, Pullman Hotel international experience on strategic performance based OCSC, OPDC budgeting and results based management to senior management 12.00-13.30 Lunch with participants All Cuisine, Pullman Hotel 13.30-16.30 Workshop on the technical issues relating to linking the Technical officials from the BOB, Alpha Room, Pullman Hotel strategic performance based budgeting system with the OCSC, OPDC, and sector ministries results based management tools 17.30 Cocktails Wine Pub, 2nd floor, Pullman The World Bank 57 Team Mr. William Dorotinsky Sector Manager Ms. Miki Maatsura Public Sector Specialist Ms. Joanna Watkins Public Sector Specialist Mr. Shabih Ali Mohib Senior Economist Ms. Nattaporn Triratanasirikul Economist Mr. Robert Boothe Public Sector Specialist The World Bank 58 E. RESULTS BASED MANAGEMENT TOOLS IN THAILAND (2010) RBM tools implemented by central Government Agencies in Thailand (as of Oct 2010) Assesssment Tools Feedbac Agency Weightin Frequency Timing of Year Form of Users of the #s of Legal k Tools Scope Principles/Objectives Assessment Criteria Indicators/Questions Verification Responsible g of Reporting Reporting Introduced Evaluation Information KPIs Framework Government Mechani Plans sm (i) Achievement in targets in strategic plans (at different 4 dimentions: levels: ministry, department, cluster), Submitted to Cabinet i. Effectiveness (ii) achievement in developing a service center, annually (iii) achievement in department output targets Deputy Prime Public Sector 20 Ministries, 141 Improving results based (i) Achievement in citizen participation, Minister and Royal Decree on Development Departments, 19 management in the public sector (ii) achievement in information disclosure, Others-TRIS Ministers are Good 20 KPIs Strategic Plan Provincial clusters, Assessing the organizational ii. Quality of service delivery (iii) achievement in operation with regard to corruption Adacemic responsible for Governance 2003 per agency 2008-2012 Balanced Scorecard 75 Provinces, 73 performance through annual prevention, Every 6, 9, 12 Self- institutions- monitoring and (Section 12), the OPDC Yes Not Known 2003 (total of 4-year (BSC) Academic performance agreements (iv) level of customer satisfaction months Assessment Education accomplishment of State 3,540 Operational Plan institutions, and (i) Capital expenditure disbursement rate, Standards performance Administration KPIs) 4-year annual other public (ii) achievement in energy saving, Office agreement Act (No. 5), 2002 iii. Efficiency of operatation Performance/Acti organizations (iii) achievement in time spending for service provision, The agency head is on Plan (iv) achievement in preparing unit costing accountable for achieving the results (i) Achievement in developing PMQA, iv. Organizational development covered by BSC (ii) achievement in legal development 20 Ministries, 141 (i) Leadership, Departments, 19 Others-TRIS Measuring efficiency and (ii) Strategic planning, Provincial clusters, Major component under BSC Adacemic Public Management organizational development (iii) Customer and stakeholder focus, 75 Provinces, 73 perspective iv. Organizational Self- institutions- Part VI of the Quality Award OPDC Rewarding government agencies (iv) Measurement analysis and knowledge managmenet, Yes Annual Not Known 2005 Academic development Assessment Education Royal Decree (PMQA) that achieved high performance (v) HR focus, institutions, and Standards (vi) Process mangement, other public Office (vii) Outcome organizations Measuring HR management in 5 • HR policy/plan to support government agencies' targets dimentions: and objectives. i. HR Strategic alignment • Appropriate HR size/capability • Retention of talented civil servants a) To provide a framework for • Continuous plan for staff capacity building and assessment and evaluation of administration to inspire staff within the organization HRM service delivery b) To facilitate HR devolution to • Accuracy/timeliness in implementing HR transactions line departments and provinces • Updated database in supporting decision making in HR • Place the organization in a management ii. HR Operational efficiency learning process to facilitate • HR productivity and value for money performance improvement; • HR automation • Improve communication with 20 Ministries, 141 stakeholders, government • Retention of capable staff Departments, 19 ministries, other civil service • Government employees‘ satisfaction with HR policy 2009 Provincial clusters, implementation (started in agencies, community and so on; 37 KPIs HR Scorecard OCSC 75 Provinces, 73 • Facilitate performance reporting; iii. HR Effectiveness • Promotion of development and knowledge management No Twice a year 2003 with Self- OCSC varifies Managers of under 5 Civil Service Act Academic • Provide data for benchmarking, • Existence of efficient and effective performance BSC but Assessment government agencies dimentions 2008 institutions, and management system separated both internal and external to other public recently) agencies organizations • Assist with better resource • Accountability for HR policy decision making allocation through improved iv. HR Accountability • Transparency in HR management process analysis of client service delivery and costing models; • Facilitate reporting of HRM policies, programs and services to senior management, OCSC and • Government employees‘ satisfaction on working parliament environment v. HR Quality of work life • Provision of welfare and facility The World Bank 59 Fe e dbac Age ncy We ightin Fre que ncy Timing of Ye ar Form of Use rs of the #s of Le gal Gove rnme nt k Tools Scope Principle s/Obje ctive s Asse ssme nt Crite ria Indicators/Que stions Ve rification Re sponsible g of Re porting Re porting Introduce d Evaluation Information KPIs Frame work Plans Me chani sm 30 questions in 5 dimensions: (A-1) Having a linkage between national strategic goals, Scores are rated: green, yellow, ministries strategic goals and agency service delivery or red targets (A-2) Setting service delivery goals (A-3) Having a process for identifying needs, problems, or issues of interest of target groups A. Purpose and design (A-4) Identifying outputs of interest of the target groups, or contribute to achieving the ministry‘s goals or national strategic goals (A-5) Being the sole agency responsible for the output (A-6) Taking obstacles and limitations for delivering services into consideration. (B-1) The agency prepared strategic maps that present linkages from national strategies to the agency‘s outcomes, output, and activities (B-2) The agency prepared strategic plans that identify long- term service delivery goals (B-3) The agency prepared strategic plans that identify long-term output goals (B-4) The agency‘s strategic plan has annual output goals (B-5) The agency‘s strategic plan identifies ways to seek B. Strategic planning, cooperation from relevant agencies in public and private sectors (B-6) The agency prepares strategic plans containing comprehensive performance assessment plans that lead to continuous improvement (B-7) The agency has a process for analyzing changes in national strategies, the ministry‘s strategies, and relevant laws and regulations in order to review the agency‘s strategies Example: For (C-1) The agency sets annual output goals FY 2009 20 Ministries, 141 (C-2) The agency identifies key activities that link to budget departments, 19 required resources and that promote the achievement of the calendar provincial clusters, annual output goals (preparing for Pe rformance 75 provinces, 73 Strategically allocate budget with a C. Linkages to budget (C-3) The agency sets progress indicators for all key FY 2010 Ex-post Self- Asse ssme nt Rating BOB Yes Annual 2008 BOB verifies academic consideration of organizational activities budget), the Assessment Tool (PART) institutions, and performance (C-4) The agency calculates output unit cost PART report other public (C-5) The agency takes past performance into from 2008 is organizations consideration in setting annual output goals submitted to (D-1) The agency‘s output delivery units prepare and BOB on Oct. implement annual performance plans and budget execution 15, 2008 plans (D-2) The agency‘s output delivery units prepare reliable and timely feedback reports for each output (D-3) The agency‘s output delivery units make use of reporting systems for management and performance improvement purposes (D-4) The agency‘s output delivery units calculate output D. Management unit cost for management purposes (D-5) The agency‘s output delivery units are exempted from ―value for money‖ (VFM) evaluation based on the Office of NESDB VFM analysis guidelines (D-6) The agency‘s output delivery units have financial audit reports that show efficient financial management and compliance to relevant rules and regulations (D-7) The agency‘s output delivery units evaluate efficiency of persons responsible for delivering output. (E-1) The agency demonstrates progress by comparing its outcomes with KPIs related to long-term outcome goals. (E-2) With reference to KPIs which are related to annual output goals, output evaluation results indicate progress (E-3) Since the agency‘s output delivery units are exempted from VFM evaluation in fiscal year 2009, the E. Output and outcome results agency is not required to present VFM evaluation reports (referring to question D-5). (E-4) Duties of the agency are not comparable to those of other agencies (E-5) Results from independent assessors indicate that the agency achieves output and outcome goals. The World Bank 60 Agency Frequency Timing of Year Form of Users of the #s of Legal Government Feedback Tools Scope Principles/Objectives Assessment Criteria Indicators/Questions Weighting Verification Responsible of Reporting Reporting Introduced Evaluation Information KPIs Framework Plans Mechanism Measuring VFM in 3 dimentions: (i) Performance efficiency i. Efficiency (ii) Economy Pilot phase, NESDB Assessing value for money to input NEDB gives Value for Money 20 Ministries and (i) Objective/target achievement to be Self- varifies (if the Section 22 in the NESDB for budget allocation of public ii. Effectiveness N.A. Annual (tbd) NESDB N.A. National Plans feedback to (VFM) Evaluation 141 Departments (ii) Satisfaction implemente Assessment Cabinet gives Royal Decree agencies for the next fiscal year. ministries d in 2010 a mandate) Impact on (i) people, (ii) economy, (iii) society, (iv) iii. Impact environment, (v)politics ―Evaluation Measures 4 dimensions: teams‖ in each Reflected i. General management, Similar to OPDC PMQA, but slighty different: province, back to each 19 Provincial ii. Personnel management (i) Leadership, which includes division, clusters, 75 iii. Financial management, (ii) Strategic planning, 2007 Ministry of Assessing the organizational 4 experts on DOLA Hamburger Model Provinces, 878 iv. Public services (iii) Customer and stakeholder focus, (Piloted in Self- Results reflected in Interior, performance in the local Annual each of the officers of PMQA Districts, (possibly Various types of services are (iv) Measurement analysis and knowledge managmenet, one Assessment the each division DOLA administrative agencies four areas, and stationed in subdistricts, assessed: civil defense, (v) HR focus, province) DOLA provinces municipalities) education, public health, road (vi) Process mangement, officers come back to construction (expanded to other (vii) Outcome stationed in Bangkok areas over time) provinces 20 Ministries, 141 Departments, 19 Comptroler- Provincial clusters, Reporting of Unit Generals 75 Provinces, 73 Reporting of unit costs in every Costs Department, Academic agency MOF institutions, and other public organizations Office of the Performance Auditor- Auditing General 2003 Royal Decree - Part VIII - Performance Evaluation Sections 45-49- establishment of an independent BOB, OPDC inspection (M&E Bureau BOB, committee for and Public Sector Program Evaluation NESDB NESDB- VFM Evaluation each agency to Audit and OPDC etc.. evaluate the Evaluation Sub- performance of committee) etc…. the government agency related to the result of the mission, quality of service, pleasure of customer, and value for money. The World Bank 61 Incentive Tools Agency Frequency Timing of Year Form of Users of the #s of Legal Government Feedback Tools Scope Principles/Objectives Assessment Criteria Indicators/Questions Weighting Verification Responsible of Reporting Reporting Introduced Evaluation Information KPIs Framework Plans Mechanism Ministry of Finance allocate necessary Providing monetary rewards to National budget agencies that achieved high Institutte of Department level Agencies that scored over 3-5 Rewards based Government OPDC 141 Departments performance in the BSC-funds N.A. Annual 2003 Development Performance Pay point scale for the BSC on BSC agencies allocate used for individual salary rewards Administration monetary based on group performance (NIDA) rewards/bonuses to governmetn officials 20 Ministries, 141 Departments, 19 There are Provincial clusters, Assessing individual personnel 2009 (at the Directors and KPIs but Individual 75 Provinces, 73 performance management through Supervisors determine the pay increase 0-6% based on an same time Directors and Civil Service Act OCSC managers OCSC varifies does not Performance Pay Academic providing individual performance assessment of 5-10 individual KPIs with HR managers 2008 determine work institutions, and pay Scorecard) effectively other public organizations Tools to set standards 19 Provincial clusters, 75 Service Delivery Ministry of Provinces, 878 Providing 48 service guidelines for Self- Standards/ Interior, 2007 Districts, (possibly Local Authorities assessments Guidelines DOLA subdistricts, municipalities) The World Bank 62