GET Brief: Improving Performance February 2010 53466 Improving Performance: Foundations of Systemic Performance Nick Manning Executive Summary At its heart, a performance orientation in the public sector is a predisposition to make promises and an ability to deliver them. Some of the key ideas behind this are: 1. Responsiveness - reducing the time lag between changed political priorities and corresponding public policy actions. 2. Measurement - the quantification of outputs (and occasionally outcomes). 3. Managerialism - the relaxation of the enforced consistency in procedures to move towards flexibility with accountability in order to improve efficiency. It is often seen purely as an import from the private sector, but in fact there have always been areas of managerialism within the public sector. Using these ideas, this note describes some of the key technical foundations necessary for moving towards a performance orientation and outlines a pragmatic approach for improving performance, highlighting the part played by changing performance arrangements for senior management. There are three key insights offered here. First, there is now a reasonable consensus on the managerial foundations necessary for moving towards a performance orientation and that, without these, the focus of management attention will be at best on day-to-day managerial survival or on increasing the total resources available without improving the results delivered. Second, there is no simple boundary to be crossed en route to a performance orientation ­ the approach is both a state of mind and a set of institutional arrangements, and there are many gradual approaches that can begin to deliver results. Third, while the preconditions are often absent at the whole of government level, there are many contexts which provide a more secure basis for attempting reform ­ varying from the sector level to municipalities or to specific entities or organizations. The development of "performance enclaves" at anything other than the whole-of- government level raises challenges of sustainability but, in offering these simple insights, the intention of this note is to avoid a trap often set by guidance on this topic. Much advice is premised on the basis that nothing can be done until the basic institutional arrangements are functioning well at the national or federal level ­ a threshold which demoralizes rather than energizes. In challenging environments it is unlikely to be passed within the working life of many dedicated public servants. The note concludes with a particular emphasis on the role that performance measures and incentives can play in senior staffing arrangements. GET Briefs ­intends to capture the knowledge and advice from individual engagements of the World Bank's Global Expert Team on Public Sector Performance (PSP GET). The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank. For more information about the PSP GET, contact the GET team leaders Bill Dorotinsky (wdorotinsky@worldbank.org) and Nick Manning (nmanning@worldbank.org) or visit http://pspget. 1 Historical Antecedents in the OECD and Latin America Beginning with the whole-of-government public management reforms in the OECD, many countries embarked on laying the foundations for performance improvements. Arguably, despite a solid if traditional basis of accountability, the missing element in public expenditure management for the OECD countries prior to the 1970s was top-down budgeting and a medium term orientation. Fiscal belt-tightening in the OECD in the 1970s underscored the conclusion that performance improvements and responsiveness to changing priorities could no longer be achieved by spending more ­ in the short or medium terms. Discipline has more or less been achieved, and variations on the theme of Medium Tem Expenditure Frameworks can now be found throughout the OECD, with the stated objective of linking policy, planning and budgeting.1 In human resource management, the foundations were laid in the late 19th and early 20th century reforms that, broadly, ensured a tradition of oversight and self-discipline within the largely meritocratic group of senior civil servants, reasonably insulated from day-to- day politics. Post-war, OECD civil servants had gained some experience of using performance information to inform management decisions about service delivery prior to the more systematic development of performance metrics, and this led to a reasonably sophisticated view of how performance information can be used (in dialogue and with judgment, avoiding in most circumstance mechanical linkages with resource allocation or staff incentives). Building on these to develop a performance orientation Building on these foundations, over the last three decades, some degree of additional responsiveness has been found in planning and budget preparation. In the 1960s and 1970s some countries tried to sharpen the budget as an instrument for prioritizing expenditures by introducing new routines such as zero-based budgeting or program, planning and budgeting systems. These proved costly to operate and ineffective in shifting priorities. However, more recently pragmatic and less information-intensive approaches to program budgeting combined with more determined program evaluations and expenditure reviews have led to improvements. During the same period, responsiveness in human resource management has been tried through attempts (largely unsuccessful) to reduce the size of the public sector and through a more determined effort to make senior staff more responsive to political control through reforms employment contracts. Managerialism is evident in the attempts to improve efficiency in budget execution emerging in the 1990s. Proponents of reform asserted that organizations and managers should have more delegated authority ­ creating clear objectives, effective incentives and providing the necessary authority over resources and skills to manage for results. There are corresponding HRM trends of: delegation to line ministries or departments, giving them flexibility in tailoring their human resources policies to meet their organizational performance objectives; and individualization of benefits, allowing recruitment 2 arrangements, employment contracts, accountability and pay to be tailored to the specific tasks of the individual. The measurement culture grew in step with the move towards managerialism. At the apex, the presumption that the legislature had a role in scrutinizing (although rarely amending) the executives budget proposal on the basis of performance information grew in strength. In many cases, this required that government produced a set of performance measures for the whole government that is relevant enough for use by various Box 1: Moving towards a performance actors in the budget process. More robust Financial orientation in Latin America Management Information Systems were developed to Many Latin American countries have provide necessary financial information that managers and attempted to move towards program policy makers can link to the non-financial performance budgeting, in order to improve indicators. Measurement is also seen in the increasingly responsiveness to changing policy priorities. Chile has made the most results-focused individual performance appraisal significant progress - although Brazil, arrangements, and, for a few higher positions, these have Colombia and Mexico have also made been associated with performance pay. Performance significant strides. There are significant targets for senior public servants are often linked to shortcomings in some of the program agency targets, and these sometimes feed into the classifications as they are somewhat overly linked to existing administrative individuals performance rating. structures. Responsiveness is less of a Human Resource Management (HRM) In Latin America, public-expenditure management has reform objective in Latin America than in proven far more fragile than in the OECD. Fiscal control many OECD settings ­ as the high degree was a priority in Latin Americas return to macro- of political penetration of the public service has made it distinctly responsive economic stability in the mid 1990s. Cash controls were to political concerns. the emergency measure that often supported the first wave of fiscal reforms. Subsequently, the region has had Managerialism, in the sense of delegation important successes in developing rules to rehabilitate the to operational entities, has made budget as an instrument for fiscal control. Numerical somewhat less headway ­ although there has been a significant increase in the restrictions on spending limits or budget deficits have number of arms-length agencies often been developed through fiscal responsibility laws, created in order to provide some stabilization funds, and limits on sub-national-government delegated financial and human resource borrowing. Procedural rules such as single treasury management authority. The agencies accounts have reinforced the powers of the ministry of have proven to be somewhat problematic, not least because they fall, de jure or de finance vis-à-vis spending ministries and parliaments. facto, outside of hierarchical ministerial More accurate and timely information is resulting from authority, often reporting to the president the introduction in a number of countries of integrated and leading to problems in sustainability financial management information systems (IFMIS). following changes in administrations. But in many countries, the budget is still far from being Measurement has in many ways been the most challenging area as there have been an instrument for planning and implementing priority very significant strides in monitoring and actions: often executed budgets bear a modest relationship evaluation systems, but a continuing lack to the original legislated budget. There are many multi- of alignment between the budgetary year plans, fiscal and economic frameworks on paper, but classifications of activities and the indicators used for M&E. these provide little annual budget certainty for the spending agencies. 3 Following the return to stability, downsizing was a priority in managing human resources and there were some successful cases (though the data are not always clear). Some countries also made progress in improving establishment controls (usually through automated payroll systems). However, the most important "second-generation" reform objective of moving from clientelist, patronage-based systems to merit-based (due- process) systems could be considered Latin Americas greatest failure in public-sector modernization. Chile, then Brazil (and a couple of other more developed countries to a lesser extent) are the exceptions, though quite a few countries have established merit- based "islands" in selected agencies. Box 1 highlights the move toward performance in Latin America along the three dimensions of responsiveness, managerialism, and measurement. Management Foundations A performance orientation in the public sector requires reasonably strong technical management foundations. There is little prospect of relating past performance to future services unless: I. At the time of planning and approving the annual budget and work program there is: i. A good budget classification (allowing funds available to be allocated on the basis of administrative units, economic purpose and functions or programs). ii. A multi-year orientation, in other words a widespread recognition that deferring problems to the next year (or the next administration or management team) is unsustainable ­ noting that the exact form of this multi-year approach can vary significantly and there is a significant risk of ritualism in which the medium term perspective is provided on paper but is not reflected in the mindset of senior staff. iii. A process for preparing the budget that is seen to be reasonable and during which the views of the spending departments are recognized. iv. The wage bill does not crowd out investment or other important recurrent expenditures. II. In implementing the work program or the annual budget, there is: i. Confidence on the part of the spending units that they will get the funds that they were budgeted. ii. Good recording and management of cash balances, debt and guarantees to prevent unwelcome end of year surprises. iii. Effective payroll controls that minimize the usual sins of ghosts and double-dipping, and that salary payments are made on time. iv. Competition, value for money and controls in procurement. III. Reasonably comprehensive internal audit and good scope, nature and follow- up of external audit 4 IV. At all times, institutionalized watchfulness to ensure that: i. Revenues and expenditures are as expected in the budget or spending plan. ii. Expenditure payment arrears are minimized. iii. The public have access to key financial/fiscal information. iv. Semi-autonomous entities are well-regulated, and do not provide expenditure or governance risks. Figure 1 illustrates how these foundations can be used to drive towards a more substantial performance orientation. Figure 1: Building outward from foundations Early steps toward ·Planning and approval performance ·Budget allocations ·Implementation and based on performance ·Hard budget budget execution · Informed decision constraints ·Audit and evaluation making ·Reporting · M&E and audits ·Use of incentives Longer term goals in Management performance Foundations orientation Early steps Incentives for the use of performance information occur at several levels. A systemic contribution to a whole-of-government performance orientation may entail systematically incorporating nonfinancial performance information into budgetary decision-making and allocation processes.2 However, when these foundations can be found at the sector/ministry level, or at any level of government or in any entity or service providing unit, then some early steps towards achieving a performance orientation can be pursued. The fundamental step is the widespread acceptance of the principle of hard budget constraints. That is not to say that managers cease hoping that additional resources can be found ­ the development is that they accept the reasonableness of the proposition that the incentives for using performance information arise when the program or institution that is being funded faces hard budget constraints, recognizing that the information can be used either to make a case for enhanced funding or to improve efficiency within those constraints. When budget constraints are soft, driven primarily by political weight or 5 other ad hoc considerations, then there are few reasons to use performance information for budgetary negotiations. Rather, increasing the aggregate budget is more important than improving efficiency within available funds. However, there are additional steps that mark a move forward. There is no first best way of strategic planning, objective and target setting, but managers (at government, sector, unit level) must be required and prepared to make a solid attempt, and to put thought into output reporting arrangements. Using performance information does not come naturally to many staff accustomed over many years to manage inputs within tight procedural guidelines, and it is important to gain some experience of using performance information to inform management decisions about service delivery. These early steps also require dispelling some of the myths about the use of performance information ­ and there must be a broad recognition at senior staffing levels, that performance information must be Using performance measures can motivate used in dialogue and with judgment public agents toward better outcomes, ­ mechanical linkages with resource enabling governments to properly evaluate, allocation or staff incentives should control, budget, and ultimately to celebrate be very rare. improvements in service delivery. To develop momentum at this stage, some initial incentives can be developed. A defined set of standards for some selected subcomponents of ministry performance might be established, and if these can be met, enhanced delegation of personnel and resource management authority to the ministry might be enabled. These standards might refer to internal processes as much as to delivered results. For example, if a ministry can meet standards for personnel record maintenance, personnel reporting, pay-roll recording and management, then the ministry might be given more flexibility over hiring to reallocation of staff. Other internal ministry processes that would be pre-requisites to enhanced autonomy include internal control, procurement, financial reporting and records management, internal audit, accounting and reporting.3 Finally, like the acceptance of the principle of hard budget constraints, the significance of discipline within the senior staff group must be widely entrenched ­ they must be largely meritocratic and reasonably insulated from day-to-day politics. Without some budgetary and career incentives for the effective use of performance information, then many reports and evaluations will gather dust. Entrenching a performance orientation There is a huge and growing literature on performance management, and this cannot be adequately summarized here. However, the experience of the Public Sector Performance Global Expert Team suggests that the key features that build on the foundations set out above and illustrate that a performance orientation is becoming entrenched, are that: 1. At the time of planning and approving the annual budget and work program there is: a. A budgeting process that rewards departments or programs that have made an effort to develop robust performance reporting arrangements. 6 b. At the time when the budget (or the agency annual business plan) is being approved, there is sufficient performance information to assess its past record ­ but not an overload ­ and that there is sufficient time and capacity to scrutinize the proposal. 2. In implementing the work program or the annual budget, there is: a. Adequate (relevant, timely and reliable) performance information to inform managers' decisions during budget execution, accompanied by an accounting and costing system that assigns costs to program outputs and activities so that the costs of change are evident. b. Those same managers have clear objectives, with the necessary authority over resources and skills to manage for results ­ and that their staff management appraisal is linked to the annual ministry performance assessment. To provide them with that necessary authority, then modest delegation arrangements can allow managers to assume responsibility for some program areas, with some accompanying moves away from detailed line item input controls towards managers being held accountable for both results and the use of inputs. c. There should also be a high level set of performance measures for the organization (government, municipality, department, etc.) that summarizes the key policy objectives, and performance information reporting on progress against those targets. 3. The audit and evaluation functions are adapted, so that: a. In addition to providing management information and risk management, internal audit provides confidence in the accuracy and the validity of performance information that is provided. b. There are regular performance evaluations that provide ex-post information on deliveries to inform budget and managerial decision-making. c. Staff appraisals are developed so that for senior staff they are at least consistent with the high level measures for the organization. Junior staff appraisal mechanisms should generally be used for behavioral incentives and very rarely linked to entity performance. 4. At all times, there will be institutionalized oversight to ensure that: a. The total numbers of performance indicators remain at very modest levels ­ and that there is no over-reliance on single indicators as this will create incentives for gaming. b. Expenditure reviews promote reallocations to consider, ex post, whether the funds could have been better allocated between programs or departments to ensure that policy objectives are met. c. The program structure of the budget remains linked to current policy priorities. The Annex summarizes these seemingly necessary foundations of public management, and the early steps towards performance. It also characterizes the elements that suggest that a performance orientation has been built on those foundations. Recognizing risks In stepping away from the usual mantra that a public sector performance orientation must always start with a hard-to-attain set of managerial arrangements at the whole of 7 government level, the pragmatic approach suggested here must recognize some risks. Specifically, there is the risk of encouraging much managerial effort to be placed in establishing "performance enclaves" that are only lightly institutionalized within the public sector and thus prone to achieving sudden prominence for high performance (as they can be established quickly, particularly with donor assistance) and equally prone to only slightly more gradual erosion in performance as the original government or donor interest diminishes. The risk of improvements being rather short-lived when achieved through "performance enclaves" can arise when specific departments or entities are singled out for performance orientation or when creating new bodies. When established carefully, the specialization of single purpose agencies can lead to improved performance (though, in practice, this is more likely to happen because of the agencys internal culture than because of contractual arrangements). They are very often created to solve the problem highlighted earlier ­ that a key requirement in moving towards a performance orientation is to establish an intrinsic discipline within the senior staff group, ensuring that it is largely meritocratic and reasonably insulated from day-to-day politics. Where this has proved impossible in the larger public sector, then a special purpose agency has had some pragmatic value. The objective has been to create an "island" of performance, sometimes formally merit-based, but, in the case of service delivery, rarely politically independent. Many of these agencies have proven effective, social-assistance agencies notably, but such arms-length service-delivery agencies are often unable to provide a permanent solution to service-delivery problems. First, they tend to be politically unsustainable, because they rely on a transitory political authority (and their very presence, it is argued, undermines attempts to reform permanent administrative structures). Second, standing outside the permanent structure, they can more easily become unaccountable, indeed corrupt. These risks can be mitigated in a couple of ways. First, and most importantly, the gains from any performance enclave must be locked in by ensuring that the number of diverse organizational forms and regulatory arrangements within the public sector is kept to a minimum. Excessive heterogeneity of organizational forms and accountability arrangements means that it is simply harder to evaluate reforms and to determine whether appropriate fiduciary and transparency safeguards are being maintained. Second, internally-motivated reforms can be accompanied by external, demand-side reforms. Many countries have tried to create a more open government to ensure performance. Freedom-of-information laws and the publication of service standards and performance results have created greater transparency. Administrative simplification and one-stop shops have eased access to services and entitlements. Citizens voices are more easily heard through consultative and participatory mechanisms. Civil-society organizations and Ombudsman offices are making it easier to complain about the government. Some governments have given citizens more power by giving them more choices ­ in education (through vouchers) and health (through multiple insurers). In education, some countries have experimented successfully in involving parents in school-management decisions. 8 Senior Management Elements of a Performance Orientation When the conditions exist for a movement towards a performance orientation, including the important acceptance within the administrative culture that hard budget constraints might be unwelcome but are essential, senior ministry staff performance management arrangements are key. Internationally, measures of performance are beginning to enter into many elements of the management of senior public sector staff. Some countries have gone further than others ­ often reflecting the ability to drive through what is generally a politically-challenging set of reforms. The opportunity to introduce performance arrangements for senior staff has most often arisen in the Anglophone administrative tradition, where there are modest requirements for legislative approval of administrative reforms. Most OECD member countries report having a formal performance appraisal system for civil servants based on individual assessment of performance, which in turn rely on objectives established in an employee/management performance agreement (OECD 2005). A number of countries in Africa have also introduced performance assessments for civil servants (Box 2). Measurements of Box 2: Selected examples of performance agreements for civil personal servants in Africa performance can Country Arrangement be used primarily for control, or for Botswana Has a Performance Management System in place. Mandatory Performance Development Plans are prepared dialogue, with and executed by all officers. very different implications. Kenya Rolled out a new performance appraisal system to the service in 2006. One of the policy priorities for the Public Where control is Service Commission of Kenya is to integrate rewards and emphasized, sanctions in the performance appraisal system (Strategic performance Plan 2009-2012) measurement South Africa Uses Performance Assessments and performance related leads to action pay increases. being taken in a Tanzania Introduced the Open Performance Appraisal & Review direct way, with System (OPRAS) in 2004, replacing the Closed Annual decisions about Confidential Report System. individual Uganda As part of the Performance Management System, rewards driven Performance Appraisal Forms are used to identify mainly by the performance gaps and development needs. measurement and with other sources of information playing a negligible role. More effective arrangements emphasize dialogue, and in these performance measurement is just one source of information to be combined with others in determining an individual senior civil servants, agencys or ministrys performance. Other sources of information are used to interpret the measurement data and these are incorporated through formal or informal discussions. 9 The rise of performance-based arrangements for senior civil servants has not been without problems. Practitioners tell stories about ,,gaming, in which outputs are manipulated or the data massaged in order to trigger a reward. These stories do not amount to a proposition that performance measurement or performance management should be abandoned ­ but they do argue for caution and pragmatism. Thus, such arrangements sometimes include devices such as fixed term contracts or performance-based pay. Senior civil servants are likely to be as motivated by promotion and by recognition from their peers and the public as by financial rewards. These tools may provide more practical approaches as the incentives to ,,game are lower in the case of promotion and recognition. In any event, the results of performance measurement of senior civil servants should be included in promotion decisions and in recognition of results. Conclusion There are more opportunities for performance-enhancing reforms than are generally recognized. There are preconditions ­ but there are often organizations or sectors within the public sector where these are met. However, while there are gains in stepping away from the insistence that all reforms must be at the whole of government level, there are risks ­ and the disappointment of managers who have invested significant time in performance-oriented developments only to see the gains erode within years is palpable. These risks can be mitigated somewhat ­ but not entirely. 10 Annex: Public management foundations and steps towards a performance orientation 1. Planning and approval 2. Implementation 3. Audit and 4. Constant a) Strategic b) Budget a) Budget b) Management c) Monitoring evaluation concerns planning and approval execution of information and review preparation Key Good budget An orderly budget Predictability in the Timely and regular Timely and quality Expenditure out- Effective internal foundations of classification and, process that builds availability of funds for financial accounts in-year budget controls for non- turns that reflect ideally, a multi-year credibility and commitment of reconciliation reports and annual salary expenditurethe original public perspective to avoid ensures that no key expenditures for Availability of financial statements Effective internalapproved budget management deferring problems information is spending units information on Oversight audit ­ and sound (in total and in to the next year (or missing from budget Recording and resources received by maintained over scope, nature and composition) the next documentation. management of cash service delivery units aggregate fiscal risk follow-up of Revenue out-turn administration or Transparency in balances, debt and from subsidiary external audit that reflects the management team). making guarantees bodies original approved The wage bill does commitments to Payroll is managed budget not put undue subsidiary bodies (or efficiently and with Monitoring and pressure on the subnational effective controls, and minimization of expenditure profile, governments in the salary payments made expenditure crowding out case of on time payment arrears investment or other central/federal Competition, value for Public access to important recurrent governments) money and controls in key expenditures. procurement financial/fiscal information Semi- autonomous entities are well- regulated Early steps There is an There is a solid Senior staff have some experience of using performance information to A broad recognition There is an towards understanding at the attempt at entity- inform management decisions about service delivery at senior staffing intrinsic senior level about level (government, Consideration of some incentives for improving processes ­ for example if a levels, that discipline within performance the significance of sector, unit) strategic ministry can meet standards for personnel record maintenance, personnel performance the senior staff hard budget planning, objective reporting, pay-roll recording and management, then the ministry might be information must be group, largely constraints and target setting - given more flexibility over hiring to reallocation of staff. Other internal used in dialogue meritocratic and and output reporting ministry processes that would be pre-requisites to enhanced autonomy include and with judgment reasonably arrangements internal control, procurement, financial reporting and records management, ­ mechanical insulated from 4 linkages with day-to-day internal audit, accounting and reporting. resource allocation politics or staff incentives should be very rare 11 1. Planning and approval 2. Implementation 3. Audit and 4. Constant a) Strategic b) Budget a) Budget b) Management c) Monitoring evaluation concerns planning and approval execution of information and review preparation A performance The budgeting Those approving the Performance information There is a high level Entity level Internal audit Total numbers of orientation has process allows and budget (the helps to inform set of performance performance provides performance encourages the legislature in the managers' decisions measures for the information informs management indicators kept at been built on reallocation of case of national or during budget execution, whole entity budget and information, ensure very modest those resources towards subnational (i.e. it is relevant, timely The accounting and managerial sound governance, levels ­ and over- foundations programs that have governments) have and reliable) costing system decision-makers risk management reliance on single when... performance enough but not an Organizations and assigns costs to and the accuracy indicators monitoring overload of managers have clear program outputs and and the validity of avoided to arrangements performance objectives, effective activities the performance minimize information ­ and incentives and have the information incentives for time and capacity to necessary authority over External audits gaming scrutinize the budget resources and skills to enhance the Expenditure proposal manage for results accountability of reviews promote Senior staff management the entity reallocations to appraisal, linked to the Financial and better align annual ministry performance resources with performance assessment evaluations usefully policies Modest delegation inform budget and The program arrangements that allow managerial structure is key managers to assume decision-making. clearly linked to responsibility for some Senior staff policy priorities program areas, with management some accompanying appraisal linked to change in expenditure entity performance controls - away from assessment detailed line item input Junior staff controls towards appraisal used for managers held behavioral accountable for both incentives and very results and the use of rarely linked to inputs. entity performance Sources: (Public Employment and Management Working Party 2006; Ketelaar, Manning et al. 2007; Arizti, Lafuente et al. 2009; PEFA Secretariat 2009) 12 References: Allen, R. and D. Tomassi (2001), Managing Public Expenditure: A Reference Book for Transition Countries, Paris, OECD SIGMA. Arizti, P., M. Lafuente, N. Manning, F. Rojas and T. Thomas (2009). Performance- Informed Budgeting in Latin America: Experiences and Opportunities. World Bank, Washington DC, Available from: http://siteresources.worldbank.org/EXTLACREGTOPPUBSECGOV/Resources/L CSPS_Working_Paper_0309_Performance_informed_budgeting.pdf. Behn, Robert. "Why Measure Performance? Different Purposes Require Different Measures." Public Administration Review, vol. 63, no. 5 (2003): 586 ­ 606. Fiszbein, A. (2005). Citizens, Politicians and Providers: The Latin American Experience with Service Delivery Reform. World Bank, Washington DC. Ketelaar, A., N. Manning and E. Turkisch (2007). Performance Based Arrangements for Senior Civil Servants - OECD Experiences (OECD Governance Working Paper). Paris. Ljungman, G. (2006), 'The Medium-Term Fiscal Framework in Sweden', OECD Journal on Budgeting, 6 (3). OECD (2005), Performance-Related Pay Policies for Government Employees, Paris, OECD. PEFA Secretariat (2009). Public Expenditure and Financial Accountability: Public Financial Management Performance Measurement Framework. PEFA Secretariat, World Bank, http://www.pefa.org/. Public Employment and Management Working Party (2006). OECD Country Peer Reviews of Human Resource Management: Framework for Discussion. OECD, Paris. Schick, A. (2002), 'Does Budgeting Have a Future?', OECD Journal on Budgeting, 2 (2). World Bank (2002). Thailands Hurdle Approach to Budget Reform. World Bank, Washington DC. World Bank (2003), World Development Report 2004: Making Services Work for Poor People, Washington DC. For related materials, visit http://pspget. 1 However , as (Schick 2002) points out, Medium Term Frameworks are more of an exercise in projection than in policy-making ­ they are not self-enforcing and there have been some significant disappointments. See particularly (Allen and Tomassi 2001; Ljungman 2006). 2 See particularly (Arizti, Lafuente et al. 2009) 3 See footnote 4. 4 See (World Bank 2002). Thailand used seven hurdle areas: budget planning, output costing, procurement management, budget and funds control, financial and performance reporting, asset management, and internal audit. These were deemed too many in practice. 13