45091 PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY BERNARD GAUTHIER HEC MONTRÉAL July 2010 This study was commissioned by the World Bank as part of the project “Measuring Progress in Public Services Delivery� (Ghislaine Delaine, Task Team Leader, AFTKL). The author is grateful to Ritva Reinikka and Waly Wane for comments on earlier drafts. Address for correspondence: Bernard Gauthier, Institute of Applied Economics, HEC Montreal, 3000 Cote-Ste-Catherine, Montreal (Quebec) Canada H3T 2A7, bernard.gauthier@hec.ca TABLE OF CONTENTS Executive Summary ............................................................................................. iv 1. Introduction ......................................................................................................1 1.1 Tracking Surveys and the Public Service Provision Framework..............1 1.2 The Various Purposes of PETS and QSDS ..............................................3 1.3 Structure and Contents of the Study .........................................................5 2. PETS and QSDS in Sub-Saharan Africa: Context and Objectives .............6 2.1 Origin and Motivations of the Firsts PETS and QSDS.............................6 2.2 Brief Review of Tracking Surveys’ Motivations, Objectives and Means ...............................................................................................12 2.3 Summary .................................................................................................16 3. Institutional Arrangements: Administrative Structures of Public Expenditure Flows .............................................................................17 3.1 The public Administration Resources Flow Structure ............................17 3.2 Administrative Structures and resources Flows in Practice ....................19 3.3 Information Flows and Processes ...........................................................24 3.4 Summary .................................................................................................25 4. Main Findings.................................................................................................26 4.1 Leakage of Resources .............................................................................26 4.1.1 Patterns of Leakage Levels .........................................................32  Rule-based versus discretionary expenditure........................33  Wage versus non-wage expenditure .....................................34  Levels of government ...........................................................34  In-kind versus cash transfers .................................................35 4.2 Information Asymmetry and Information Flows ....................................35 4.3 Impact of Public Resources ....................................................................37 4.4 Absenteeism and Ghost Workers ............................................................40 4.5 Impact of Decentralization ......................................................................43 4.6 User Fees .................................................................................................44 4.7 Delays .....................................................................................................46 4.8 Efficiency ................................................................................................49 4.9 Other Findings ........................................................................................51 4.10 Summary .................................................................................................55 5. Harmonization Issues: Design and Implementation Decisions ..................58 5.1 The Choice of resources to Track ...........................................................58 5.1.1 The Tracking Choices of Two Successful Tracking Surveys .....59 5.1.2 Other Examples ...........................................................................63 5.1.3 An Alternative Approach to Tracking ........................................67 5.1.4 Principles for Flow Tracking Selection ......................................70 5.2 Sample Selection Issue ...........................................................................71 i 5.2.1 Examples of Sampling Strategies ...............................................71 5.2.2 Alternative Strategy: Mixed Sample-Census Approach .............81 5.2.3 Household Survey Linkages .......................................................82 5.3 Length of Quantitative Data Tracking ....................................................84 5.4 Data Sources: Records versus Recalls ....................................................86 5.5 Survey Timing ........................................................................................86 5.6 Multi-Sector Surveys ..............................................................................87 5.7 Data Management ...................................................................................88 5.8 Questionnaire Design ..............................................................................89 5.8.1 General Considerations ...............................................................89 5.8.2 Instrument Design .......................................................................90  Central Government Unit Module ........................................90  Service Provider Module ......................................................90  Regional Administration Module .........................................92  District Administration Module ............................................92  Staff Module .........................................................................93  Client Module .......................................................................93 6. Good Practice Principles ...............................................................................94 7. Proposal for Future Surveys .......................................................................103 7.1 General Considerations on the Selection Process of Future Tracking Surveys ..................................................................................103 7.2 Potential Countries and Projects ...........................................................104 7.3 Other Potential Projects and Tools .......................................................106 8. Conclusion ....................................................................................................107 References ...........................................................................................................112 ii List of Tables Table 1: PETS-QSDS Surveys in Sub-Saharan Africa: Motivation and Objectives .........................................................................8 Table 2: Leakage of Public Funds .........................................................................27 Table 3: Absenteeism and Ghost Workers.............................................................41 Table 4: Delays ......................................................................................................47 Table 5: Other Findings .........................................................................................53 Table 6: Structure of Resource and Funding Flows in Zambia .............................62 Table 7: Sample and Resources Monitored ...........................................................73 Table 8: Interventions and Reforms .....................................................................108 List of Figures Figure 1: Clients-Government-Service Providers Relationship ..............................2 Figure 2: Funding Flows in Social Sectors - Generic Case ...................................18 Figure 3: Flow of Resources in the Health Sector in Mozambique .......................20 Figure 4: Funding Flows in the Health Sector in Chad .........................................21 Figure 5: Funding Flows across Centralized and Decentralized Provinces in the Education Sector in Zambia .........................................................23 Figure 6: Relationship Between Official and Effective Allocation of Public Resources on Health Services in Chad ....................................39 iii PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY EXECUTIVE SUMMARY To improve governance and reduce corruption in public service delivery, micro- level data is required to understand the incentives and behavior of public officials, service providers and clients in order to identify institutional arrangements conducive to effective service delivery. Two types of micro-level surveys have been devised to measure corruption and assess performance in basic service delivery: Public Expenditure Tracking Survey (PETS) and Quantitative Service Delivery Survey (QSDS). They have been implemented in the last ten years in about three dozen developing countries, the majority in African countries. PETS examine flows of funds and materials from the central government to local service providers, via regional and local governments, in order to identify resource use and leakages. They mainly evaluate the proportion of public resources (financial, human and in-kind) that reaches each level, in particular frontline service providers. QSDS are multi-purpose provider surveys that examine the efficiency of frontline service delivery and the dissipation of resources by collecting information on service providers and on various agents in the system. These two instruments could be applied jointly to obtain a more complete picture of the efficiency and equity of public service delivery. This study examines PETS and QSDS carried out in Africa with the objective of assessing their approaches, main findings and contributions. Section 2 investigates the context, motivations and objectives of PETS and QSDS that have been carried out in Sub-Saharan Africa. Section 3 examines the institutional arrangements for resource allocation and service delivery in social sectors. Section 4 presents some of the main findings of tracking surveys. Section 5 analyzes methodological approaches used in previous tracking surveys in order to identify factors that could explain the difference in past surveys’ success, and identify potential methodological harmonization. Section 6 presents a series of good practice principles that arise from past experience, and discusses how they could be implemented. Section 7 proposes potential future surveys and endeavors. Section 8 concludes the study. Tracking surveys could serve at least three purposes: diagnostic, analytical and impact evaluation. They can identify specific problems in public service delivery and clarify the causes of the problems identified. Alternately, they can be designed to assess the impact of a specific government program or reform. iv The original motivation of the Uganda 1996 PETS survey – i.e. explaining poor performance and identifying leakage – was also generally the main motivation of the following surveys. Most PETS and QSDS implemented in Africa have been motivated by common goals of increasing information on social sector spending to understand the weak link between public expenditures and outcomes. They have been implemented in a context of weak institutional settings, where accounting, monitoring and reporting procedure are deficient. Resource flows to frontline service providers are complex. Social sector institutional arrangements are characterized by several administrative layers involved in the frontline provider supply chain. In each of the various branches or resource flows of the allocation procedure, there are possibilities of leakage. In addition, these flows are generally governed by different allocation rules, administrative processes, recording, accounting and reporting procedures, etc. Tracking surveys have shed light on institutional structures and administrative processes governing resource and information flows in practice, which tend to differ from official rules and procedures. Several major findings emerge from PETS and QSDS carried out in Africa. Notably, PETS and QSDS have identified public resources leakage on a very large scale. The initial Uganda PETS estimated that 87% of non-wage capitation (per student) grants for did not reach its destination. In Tanzania and Ghana, leakage on non-wage education expenditures was 41% and 50% respectively, while in Chad only 1% of non-wage health expenditures to regional heath administrations was estimated to arrive at the health facility level. These high rates of leakage of non-wage expenditures could have dire consequences for public service delivery performance. Certain patterns in leakage levels are typically present: i) Rule-based versus discretionary expenditures: The level of discretion exercised over resource allocation tends to influence resource leakage. In particular, funds allocated using a fixed-rule system tends to suffer less from leakage than those that are at the discretion of public officials or politicians. Furthermore, fund capture has been associated with groups’ bargaining power. In Uganda and Zambia, for instance, schools that receive large amounts of discretionary funds tend to have greater bargaining power with higher administrative levels (Reinikka and Svensson, 2001; Das et al, 2004a). ii) Wage versus non-wage expenditures: Non-wage expenditures suffer more from extensive leakage than do salary expenditures. Local officials and politicians could take advantage of their information advantage to reduce disbursement of non-wage expenditures to health centers or schools, knowing it would attract little attention. In contrast, salaries are often paid directly by the central government to individual workers at the provider level, and failure to pay health workers or v teachers would attract attention as workers know how much they are owed. iii) Levels of government: Leakage is more pronounced at specific tiers of government, depending on the country-specific institutional arrangement. In some cases (e.g. Mozambique, Rwanda, Senegal, Uganda, Tanzania), the most serious resource leakage was observed at the local government (district) level while in others (e.g. Chad, Ghana), leakage was more prevalent at the central government level, before the resources reached local government levels. iv) In-kind versus cash transfers: Leakage is more pronounced with respect to in-kind transfers than financial transfers. This is explained by the fact that the value of in-kind items is typically not known by local governments or frontline providers. This situation was observed for instance in Chad, Ghana, Mozambique and Tanzania. The capture and leakage of resources for purposes not related to the sector objectives are linked to the presence of moral hazard created by information asymmetry and differing objectives between principal and agents. Information asymmetry is a generic problem in most countries examined. PETS and QSDS have observed that there is typically a crucial lack of information at various levels in the public organizational structure regarding resource use and transfers through the supply chain. The information gap and retention of information is linked to dysfunctional accounting, recording and reporting procedures, along with inadequate monitoring systems. Tracking surveys have contributed to demonstrating the positive effect of public expenditures on service delivery and population outcomes. PETS have shown that the potential problem at the origin of the non-significance of public expenditures in most regression analyses was the divergence between official and effective allocations. After accounting for resource leakage, the level of public spending has been shown to have a positive and significant impact on service quality at the local level. Absenteeism among service facility workers, which translates into low quality of services, was found to be prevalent in the health and education sectors in most countries, due to poor accountability relationships and weak incentives. “Ghost� workers, which are the personnel on official payroll who do not really work in facilities, have also been observed in certain countries (e.g. Tanzania, Uganda). Moreover, user fees have been shown to constitute a very important part of service costs in several countries and may have considerable negative effects on accessibility (e.g. Chad, Kenya). Delays and bottlenecks in the allocation of resources through public administrations (e.g. salaries, allowances, financing, materials, equipment, drugs and vaccines) have also been identified in tracking surveys as serious problems affecting quality of services and staff morale. vi Various other contributions could be noted, in particular regarding efficiency and equity of public services, staff incentive systems and analysis of the substitution between public and private education expenditures. The capacity of tracking surveys to measure leakage, absenteeism and achieve overall diagnostic, analysis and evaluation objectives is associated with specific country conditions and differences in methodological choices. The study discusses potential advantages and limitations associated with some of these choices in order to identify lessons and recommendations to favor harmonization. Among the main choices facing tracking surveys is the selection of resources to track. Nonexistent records or accounts, data inconsistencies and other types of problems make certain flows untraceable or data too noisy to be informative. In choosing flows to track, surveys face a trade-off between wide coverage and feasibility. Successful past surveys (e.g. Uganda and Zambia) have restricted the tracking domain and focused on flows for which good quality and consistent data on least at two levels, including the facility level, were available. In addition to a census approach to tracking certain resource flows, surveys could make use of a sampling approach to track specific resources, in particular in-kind items (medication, material, etc.). Sampling strategies vary widely among past surveys. Some have used a random (or systematic) approach to select the sample, while other samples have been selected informally on the basis of their convenience of access or ease of interviewing. In the latter case, valid inferences about the whole population of facilities (as well as clients and staff), cannot be made. While tracking surveys’ sample strategies often involve a stratified sample approach, they could also include a mixed sample-census approach. This latter approach could be more appropriate in some cases to measure leakage levels. Other harmonization issues that have consequences on survey performance and comparability across countries include the length of data collected, problem of seasonality and the survey timing. Various approaches have been followed and, in some cases, data for current and incomplete fiscal years were tracked. Furthermore, there is a trade-off between standardization and flexibility in instrument design. Survey instruments, in particular facility data sheets, should be adapted and customized to the standard accounting and reporting procedure in practice at the facility level in the country in order to facilitate enumerators’ work and improve data quality. Nonetheless, a minimal level of methodological coherence should exist across surveys and a common minimum questionnaire structure (by sector or across sector) should be targeted. As for other harmonization considerations, these questions should be the subject of working group discussions to establish a common approach for future survey work. Beyond the methodological issues that need to be resolved and new applications that could be explored (citizen report cards, vignettes etc.), the biggest challenge for tracking surveys is probably to be able to translate their important findings and vii contributions into policy reforms and institutional changes in order to improve service quality and population outcomes in Sub-Saharan African countries. Indeed, tracking surveys are a means to achieve an end. The information on incentives and deficiencies in organizational structures and rules should ultimately be used to identify policy reforms and help implement a reform agenda in client countries. One of the main achievements in service delivery improvements that have come out of PETS-QSDS has been observed in the education sector in Uganda. The policy reform introduced targeted information flows in order to give clients potential power over service providers. The results have been quite spectacular as leakage rate was reduced very substantially. Several other potential reforms are on the agenda following PETS/QSDS, including: - Improve accounting systems in order to increase transparency and accountability; - Accelerate budget execution at various levels of the delivery chain; - Increase inspection and monitoring at all levels of the service delivery system; - Improve communication and information pass-through; - Provide additional training for decentralized government levels and service delivery units; - Establish mechanisms and incentives in the system to make the service delivery system more client driven at all levels. While there is no doubt that a good number of tracking surveys have been very successful at identifying weak links in the service supply chain, as well as areas were reforms should be put forward, in practice only a few surveys have been able to translate recommendations into policy reforms in client countries. In several countries, lack of political will to put in practice reforms has certainly been a determining factor as few countries have effectively implemented institutional change following PETS/QSDS diagnosis in service delivery. Lack of policy dialogue, insufficient dissemination of results and discussions to ensure the transfer of information about problems identified in the service delivery system should also be noted. More emphasis on clients’ outcome would certainly contribute to promoting institutional reforms in recipient countries. PETS and QSDS seek to improve the efficiency and equity of public service provision in Africa. As such, the success of the exercise should be measured by its capacity to bring about improvements in the quality of services at the population level, that is, its capacity to foster policy dialogue, policy reforms in areas of weaknesses identified and improvement in the outcome at the client/population level. Ultimately, in addition to devising robust methodologies to collect information and detect corruption in public service delivery, the real challenge and the real yardstick on which PETS/QSDS should be measured is their capacity to induce policy reforms to correct the various governance problems identified. viii PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 1 PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 1. INTRODUCTION The delivery and access of basic services to the poor, particularly health, education, water and sanitation, is at the heart of the development agenda. Indeed, most donors, in the framework of the Millennium Development Goals and the Africa Action Plan, have promoted increased government budgetary allocations in health and education sectors to reach the poor. While such approach is certainly necessary, it is however not sufficient to ensure improvement in service delivery performance. Public service provision could be affected by institutional inefficiencies such as leakage of public resources, weak institutional capacity and inadequate incentives. Indeed, even if spending is officially allocated to services that target the poor, funds may not necessarily reach frontline service providers, and effectiveness of services may consequently be affected by poor incentives, absenteeism, and poor quality (Ablo and Reinikka, 1998; World Bank, 2003). Two types of service provider surveys, Public Expenditure Tracking Survey (PETS) and Quantitative Service Delivery Survey (QSDS), have been implemented in the last ten years in about two dozen developing countries to address questions of efficiency and equity of public expenditures and service delivery. This study examines PETS and QSDS carried out in Africa with the objective of assessing their approaches, main findings and contributions. It seeks to identify a common framework in order to increase compatibility among tracking surveys and to propose potential future surveys and follow-up work. 1.1 Tracking Surveys and the Public Service Provision Framework The motivations of PETS-QSDS can be understood in the framework of government provided services, which are characterized by delegation of responsibilities among clients/citizens, government and service providers. There are two main layers of agency problems in the public service organizational arrangement. The role of intermediary agent played by the government in the principal-agent relationship creates a situation where it is difficult for the principal (citizens) to evaluate and control the actions of the decentralized agent (service provider). Information about providers’ activities flows through hierarchical administrative structures (see Figure 1). There could also be an interconnection between citizens and service providers through some form of empowerment (Besley and Ghatak, 2003; World Bank, 2003). In an ideal world, with perfect information shared among parties, citizens would be able to evaluate actions taken on their behalf by governments and service providers, PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 2 and to exercise control over agents’ behavior. However, in practice, citizens have a weak capacity to exercise control through this process as information is imperfect and asymmetric, agents’ objectives do not coincide with those of the principal (citizens), and enforcement is inadequate. One of the standard instruments used to evaluate public services is the budget/benefit incidence study. These studies make use of official information to evaluate the effectiveness and performance of government interventions.1 However, in Sub- Saharan Africa most of the information required to properly assess these activities either does not exist or is not trustworthy. FIGURE 1: CLIENTS-GOVERNMENT-SERVICE PROVIDERS RELATIONSHIP Government/Policymakers Clients/Citizens Service Providers Source: World Bank (2003, p.6); Besley and Ghatak (2003, p.238). Two “new� tools have been developed to provide policy-makers with adequate information on decentralized levels and service providers, which will enable them to evaluate the efficiency or equity of government expenditures: 2 1 See for instance Government of Uganda (2004) “Access to Education and Health Care in Uganda,� which uses the government’s official fiscal accounts and data from the Uganda National Integrated Household survey (HIS) to determine the distribution of public spending in health and education that benefits different income groups. 2 As mentioned by Reinikka and Smith (2004), “the novelty of PETS lies not so much in the development of new methods as in the use of well established methods - micro surveys— to analyze service providers and government structures.� Furthermore, the micro-focus of PETS is in line with the emphasis in economic analysis in the last decade on microeconomic data instead of relying on aggregated macroeconomic data. For a discussion, see Roberts and Tybout (1995). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 3 1. PETS seek to track flows of funds and materials from the central government to basic service providers, via regional and local governments. Using quantitative data collection at the various levels of the supply chain, they mainly examine the proportion of public expenditures that reaches each level, in particular frontline service providers, in order to identify leakages. 2. QSDS seeks to collect information on service providers and on various agents in the system in order to examine the efficiency of service delivery on the frontline as well as dissipation of resources. They examine activities and services at the provider level, the incentives and behavior of various agents, input use, pricing and quality. The focus could also be placed on staff incentives and behavior, relationships with the demand side, etc. Various types of providers could be examined: public, private for-profit and private non-profit. These two instruments could be conducted jointly in order to obtain a more complete picture of the efficiency of a public allocation system, activities at the provider level, as well as various agents involved in the process of service delivery outcomes.3 Furthermore, data from service users (e.g. students or patients) could provide qualitative measures of service quality and performance. Tracking surveys could significantly strengthen the incentives for providers to deliver quality basic services through the provision of information on service delivery systems and provider efficiency and behavior. 1.2 The Various Purposes of PETS and QSDS Tracking surveys have been viewed as potentially pursuing several purposes (Turner et al, 2001; Reinikka and Smith 2004). Reinikka and Smith (2004) argue that tracking surveys could be conducted for three different purposes: diagnostic, analytical and impact evaluation. A tracking survey could serve a diagnostic purpose if it seeks to understand specific situations and identify specific problems without necessarily examining the reason for their occurrence or potential solutions (Reinikka and Smith 2004, p. 35). For instance, by studying the flow of public funds and other resources through various levels of government and administrative strata, PETS could allow policy makers to diagnose how incentives and accountability systems work in practice. PETS could hence be useful for locating and identifying political and bureaucratic capture of resources, corruption and problems of resource deployment. 3 Combined PETS-QSDS are sometimes called Expenditure Tracking and Service Delivery Surveys (ETSDS) or Expenditure and Service Delivery Surveys (ESDS). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 4 Alternatively, as an analytical tool, PETS or QSDS could be designed to pinpoint the causes of the problems identified in the public service provision system and to propose solutions to correct these deficiencies. For instance, the survey could be designed to try to determine factors explaining differences in performance among various types or ownership categories of schools or health facilities.4 Tracking surveys can also be integrated in impact evaluations. They can be designed to examine the impact of a specific government program or reform put in place. For instance, repeated PETS were used in Uganda to evaluate the impact of an information campaign designed to reduce resource leakage in education identified by an initial tracking survey. The impact assessment showed that the information campaign had successfully improved resource flow, and revealed the efficacy of mobilizing civil society against corruption (Reinikka and Svensson, 2004b). Tracking surveys can thus effectively contribute to systematic evaluation of public service delivery and use of public resources, along with the dissemination of information that could lead to reforms of institutional arrangements, in ways that create incentives for more effective use of resources and service delivery. Furthermore, tracking surveys have effectively evaluated the usefulness of public expenditures in social sectors. Contrary to the pessimistic view in the literature concerning the usefulness of public expenditures on population outcomes in health and education,5 PETS have shown that the potential problem at the origin of the non- significance of public expenditures in most regression analyses is the divergence between official and effective allocations. Indeed, as noted by Ablo and Reinikka (1998), health and other social spending may have little impact on population status because expenditures may not translate into improved services in a context where mismanagement and corruption could be key issues. The 1996 Uganda PETS, which tested the hypothesis that weak impact of public education expenditures on outcomes was due to the non-arrival of resources, showed that this hypothesis could not be rejected. Subsequent tracking survey results have supported the hypothesis of Ablo and Reinikka (1998) that after accounting for resource leakage, the level of public spending has a positive and significant impact on service quality at the local level.6 4 In Uganda, for instance, a QSDS examined efficiency questions related to ownership structure of health facilities. In Zambia, a tracking survey was designed to investigate leakage variations among districts whose provinces differ in terms of level of decentralizations. 5 See, for instance, Musgrove, 1996, Filmer and Pritchett, 1999; Filmer et al, 2000. Filmer et al (2000), for instance, did not find that health spending has a significant impact on health status in most of the studies they reviewed. 6 In Chad, for instance, the effect of public health expenditures on service delivery was tested by Gauthier and Wane (2009) through a combined PETS-QSDS. Using regression analysis, the authors showed that effective health expenditure has a positive and significant impact on health output at the regional level. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 5 1.3 Structure and Contents of the Study The objective of this stocktaking exercise is to assess the contributions and methodological approaches of PETS and QSDS carried out in Sub-Saharan Africa. The study seeks to identify potential common framework areas in order to increase compatibility among future tracking surveys. The study is organized as follows. Section 2 examines the context, motivations and objectives of PETS and QSDS that have been carried out so far in Sub-Saharan Africa. Section 3 examines the institutional arrangements for resource allocation and service delivery in social sectors. Section 4 presents some of the main findings of tracking surveys. Section 5 analyzes methodological approaches used in previous tracking surveys in order to identify factors that could explain the difference in past surveys’ success, and identify potential methodological harmonization. Section 6 presents a series of good practice principles that arise from past experience, and discusses how they could be implemented. Section 7 proposes potential future surveys and endeavors. Section 8 concludes the study. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 6 2. PETS-QSDS IN SUB-SAHARAN AFRICA: CONTEXT AND OBJECTIVES This section reviews the main PETS and QSDS implemented to date in Sub-Saharan Africa. First, the motivations and objectives of the first tracking surveys implemented in Uganda are examined. The motivations, objectives and means put forward by some of the following tracking surveys implemented in other African countries are then reviewed. 2.1 Origin and Motivations of the First PETS and QSDS The first PETS was implemented in Uganda in the education and health sectors in 1996. The country was facing stagnant and even declining education and health outcomes despite important increases in expenditures in these sectors in the past decade. In education, despite the tripling of public expenditures, official primary school enrolment figures were stagnant (Ablo and Reinikka, 1998). In the health sector, the country’s budget was higher than in other Sub-Saharan African countries, but evidence indicated that services were of very low quality. Furthermore, a decentralization policy had recently been implemented; district authorities and urban councils were gradually delegating responsibilities of channeling funds to schools and health facilities. The central government had very little information about resource use and reasons for poor outcomes, but it was suspected that local governments might be diverting resources for other purposes. The objective of the first PETS was therefore to track expenditure flows in the hierarchical structure in order to identify factors explaining these poor results, and to measure potential leakages in school and health facility funding. The hypothesis for the poor results was that public resources did not reach the schools and health facilities (Ablo and Reinikka, 1998). Survey instruments were developed to compare official budget allocations with actual spending at various tiers of government, including primary schools and dispensaries. The survey consisted of three instruments. Data were collected at the central level, the district level (local government), and the public primary school and public health care facility level. In the education sector, the tracking exercise focused on a specific fixed rule budget-- non-wage capitation expenditures to schools--which was officially based on enrolment figures at the school level. While data quality problems had hampered data collection in the health sector, the tracking survey in education was able to identify various problems in the sector, in particular large-scale resources leakage. The survey prompted the government to implement policy reforms, including an information campaign. Significant PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 7 improvements in expenditure flows to schools were observed in the following impact evaluation PETS implemented in 1999 and 2001. This success drew considerable attention to the new survey tool. It is also in Uganda that the first QSDS was implemented in the health sector in 2000. Health services in the country were perceived as not meeting the needs of the population. Household survey evidence indicated that for curative care, the population, poor and non-poor alike preferred private non-profit and for-profit health clinics to the less expensive public health facilities (Hutchinson, 2001). Despite an increasing health budget, health indicators (in particular immunization rates) were declining (Lindelow, Reinikka and Svensson, 2003). Information was severely lacking about health service delivery and quality, in particular regarding the scope and nature of the problems in government facilities, along with the performance variance across ownership categories. As improvements in data quality at the facility level were reported (such problems had previously undermined the 1996 PETS in the health sector), a facility level survey was put forward. The survey’s objectives were: a) to assess the types and quality of services provided by different categories of providers b) to identify problems in facility performance, including the extent of drug leakage, as well as staff performance and availability, c) to provide information on user charges and application of user fee policies d) to measure and explain the variations in cost efficiency across health units with a focus on the flow and use of resources, e) to examine the patterns of staff compensation, oversight and monitoring and their effect on performance. Furthermore, the survey was intended to provide baseline data for future evaluation of policy reforms in the sector (Lindelow, Reinikka and Svensson, 2003). The survey collected quantitative and expenditure data among primary health care facilities, about half of which were private for-profit and non-profit facilities. Information was collected from the head nurse or head doctor on a broad range of issues covering infrastructure, inputs (including availability of drugs), output, financing, staffing and remuneration and institutional support. In addition to facility level data, the survey collected information at the district health administration level on various elements (e.g. inputs, staffing, drugs and vaccines supplies and supervision). An exit poll of patients was also conducted to gather, on the demand side, information on individuals’ characteristics, behavior and perceived quality of health services. Following these groundbreaking works in Uganda, more than a dozen PETS and QSDS, as well as combined PETS-QSDS, were launched in a dozen Sub-Saharan countries and about the same number in other developing countries. Most of these surveys were conducted in the health and education sectors, but a few also comprised works in other sectors such as water supply, agriculture and rural roads. Table 1 presents the various PETS and QSDS implemented in Sub-Saharan Africa, along with their purpose, main motivation and objectives. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 8 TABLE 1: PETS-QSDS SURVEYS IN SUB-SAHARAN AFRICA: MOTIVATION AND OBJECTIVES Country Year Type Sectors Purpose Context/Motivation Objectives Uganda 1996 PETS Education and Diagnosis -To identify the reasons -To measure the difference Health why despite substantial between the intended resources increase in public and resources actually received at spending on basic various tiers, including service services, output and providers outcome indicators remained stagnant, in particular official reports of primary enrolment. 1999 PETS Education Follow- -Implementation of an -To collect data on access to and up/impact Information campaign to information and capitation grant 2000* evaluation provide citizens’ voice. reception among the PETS 1996 -Locally implemented school sample in order to evaluate surveys the impact of the information campaign on leakage levels 2000 QSDS Health Analytical -Health services were -To assess services provided by perceived as not different categories of providers meeting the needs of the -To identify problems in facility population. performance, including the extent -People preferred of drug leakage, staff performance private non-profit and and availability for-profit health clinics -To provide information on user for curative care to the charges and application of user fee less expensive public policies health facilities -To measure and explain the -Despite increasing variations in cost efficiency across health budget, health health units with a focus on the indicators were declining flow and use of resources -To examine the patterns of staff compensation, oversight and monitoring and their effect on performance. 2004* QSDS Health Follow-up 2009 PETS Water Analytical For the last over two -Analyze the national budgeting decades, significant process amounts of money into - Establish and track the chain flow the sector to improve of funds and budget support to issues of access and water and sanitation sector from functionality for the the centre to the end-user. resource poor peoples. - Analyze local government accountability mechanisms Tanzania 1999 PETS Education and Diagnosis -Suspicion that serious -To track all non-wage transfers to Health problems existed in the primary schools via local flow of funds from the authorities. central government to frontline providers 2001 PETS Education and Analytical -Monitoring of pro poor -Tracking down government pro Health expenditures in the poor expenditures on priority context of a PER sectors Information on -To assess the efficiency of budget rural water execution supply and roads, judiciary -To make recommendations for and HIV/AIDS improving monitoring pro poor was also expenditure collected. 2003 Education Analytical The study covered the To analyze the flow of resources in Capitation Grant, a stand alone development PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 9 Country Year Type Sectors Purpose Context/Motivation Objectives textbooks and the program in primary education. Development Grant, and also considered data on enrollment and test scores. There are no references to the previous two PETS. 2004 PETS Education Analytical This study was carried To establish more exact out in Tanzania as part knowledge on the actual amount of of a process of resources disbursed from the enhancing accountability central level that reach local and to respond to schools. funding leakage issues. Ghana 2000 PETS Education and Analytical -Major gaps exist in -To measure leakage between Health access to and utilization different points of resources of basic services by the distribution. poor -To identify bottlenecks in resource -Budget shares for flows health and education -To explore ways to ensure that were increased in a funds reach service delivery context of a facilities decentralization policy Rwanda 2000 PETS Education and Analytical -Increase in the -To track expenditures for social Health budgetary allocation to services the social sectors, -To identify delays and leakages of particularly health and budget transfers in order to education improve the effectiveness of -Social indicators remain budget spending. stagnant -To demonstrate surveillance and control of the expenditures to the civil society and external donors. 2004 PETS Education Analytical -Indicators, especially in -To analyze public funds transfer health and education, process, respect of procedures, (Also realized in account keeping and evaluate health, water remained stagnant despite increase leakages of public funds and agriculture -To formulate strategies to improve projects*) budgetary spending in social sectors. the system. Sierra Leone 2000 PETS Agriculture, Diagnosis - In the context of -To provide diagnostic information 2001 education, conflict, the home-grown on key poverty sectors health, security surveys sought to fill an -To provide evidence on leakage in sector, social accountability vacuum the service provider supply chain welfare, rural - Increased budgetary development -To monitor and evaluate allocation to social government services in order to and local sectors without development, increase transparency noticeable improvement water and in outcomes sanitation. Zambia 2001 PETS Education Analysis -In a context of -To examine the structure of QDSD economic crisis, the funding and implications for government and donors service delivery seek to increase quality -To examine the relationship and equity in primary between expenditures and education. educational outcomes -Reforms in the -To determine if the changes administrative and undertaken by the MOE had the institutional structure of desired impacts in terms of the education delivery education outcomes system were introduced 2007 PETS Health -Need to increase per -To map sources and uses of QDSD capita health spending. funds in the health sector -To assess the allocation, release, PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 10 Country Year Type Sectors Purpose Context/Motivation Objectives Health indicators remain actual receipt and use of public weak, signifying possible expenditure down to the facility resource allocation level and/or service delivery -To analyze alternative scenarios problems. for improving allocation of resources in the health sector and options for service delivery. Mozambique 2002 PETS Health Pilot: -Broad public sector -To assess the functioning of the QSDS Diagnosis/ reform agenda designed health service at the primary health to increase efficiency care level with an emphasis on Analysis and transparency in flow of resources and output. public service de livery. -To assess if resources allocated -Health sector reforms, to primary health care reach their including institutional destination. and management -To provide baseline data against reforms. which progress can be assessed Nigeria 2002 PETS Health Diagnosis/ -In the context of -To examine the flow of resources QSDS Analysis decentralization of basic an incentives at the facility level health services at the -To examines the role of local local government level, governments and community to understand how based organizations in the delivery institutional of primary health care services arrangements work in practice and impact service delivery 2006 PETS Education and Diagnosis/ Education: To collect information Health Analysis on capital expenditures for school construction. Senegal 2002 PETS Health Diagnosis/ -Weak correlation -To track flows of public funds (Education*) Analysis between budgetary through the various hierarchical allocation at the national levels to evaluate resources level and outcomes reaching local service providers motivated the tracking of and how long it takes resources. -To assess the impact of leakages and delays on service provision Cameroon 2003 PETS Health Diagnosis -Despite economic -Evaluate delays in budget growth, poverty remains execution of non-wage prevalent expenditures toward local service -In the framework of a providers national governance -To assess leakage levels at plan to fight corruption, different levels in the service reduce poverty and provider supply chain and increase efficiency of procurement process. public expenditures -Evaluate clients’ service -Priority given to access satisfaction to basic services 2004 PETS Education Diagnosis idem idem Madagascar 2003 PETS Education and Analytical -In the aftermath of the Education: -The main objective of QSDS Health and impact 2002 political crisis, the the study was to provide evaluation government used part of information on expenditure the HIPC funds to pay allocations and leakage in the for the tuition fees of all education system, from the two public primary school main funding programs in the children to boost education system (one relatively low enrolment administered by the World Bank, rates. the other by the MOE). -Also expected to shed Three surveys were implemented. light on the effectiveness -First survey: To evaluate the of decentralization impact of the political crisis on PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 11 Country Year Type Sectors Purpose Context/Motivation Objectives policies economic activities and on social -The PETS seek to service delivery. evaluate the -Second survey: to track budget at effectiveness of this the school level. measure on school -Third survey: To collect enrolment information at the district level on budget allocation. Health: -Impact of reinstating user fees -To evaluate health facilities’ efficiency and its determinants and to measure patients’ satisfaction. 2006- PETS Education and Analytical Education: -To improve Education: - To analyze flow of 2007 Health the enrollment and resources through the system and completion rates as well measure absenteeism, leakage as the quality of and bottlenecks education, the Health: -To study if and when Government had budgets, material and salaries substantially increased arrived at the basic facility level investments in the based on primary data collected in education sector in a nation-wide survey. recent years. -To improve their knowledge on Health: Despite large the functioning of the sector and efforts by the thus to address bottlenecks government and donors, towards better service delivery. little updated information was available on the effectiveness of spending in the public health sector. Namibia 2003 PETS Education and Analytical -Despite important -To track public resources to local QSDS Health budgetary allocation in service providers at different levels social sectors, outcomes in order to assess leakage do not meet expectations -Hypothesis is that some funds do not reach destination Chad 2004 PETS Health Analytical -The government and -To compare health resource QSDS donors wanted to ensure access and level of services in all in the context of regional delegations. increasing oil revenues -To analyze resource allocation and public expenditures and their impact on performance of in social programs in the health care facilities framework of the PPTE Initiative, that funding -To provide data on basic would contribute to characteristics of health centers poverty reduction -To provide information on objectives by reaching potential factors explaining their destination. differences in performance between health facilities and types of ownership -To identify means to improve the situation. Kenya 2004 PETS Health and Analytical -Deterioration of health -To provide information for education indicators despite improving the effectiveness of increase in health public expenditures in bursary spending. funds and in primary health care in -In the education sector, order to determine if they benefit important rates of the poor as intended dropout despite bursary PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 12 Country Year Type Sectors Purpose Context/Motivation Objectives programs. Mali 2005 PETS Education Analytical With a consistently Identification and analysis of QSDS increasing budget, the potential causes of problems Ministry of Education encountered in the delivery of (MoE) is also public resources to designated progressively recipients. decentralizing the spending of a higher share of its resources to the regional level. Mali 2006 PETS Health Analytical In the context of public - Provide a quantitative QSDS management reforms assessment of the budget underway to improve execution with a focus on non- health outcomes, the salary expenditures Ministry of Health has - Assess the delays in receiving developed in 2003 the health services and assessment of Medium Term resource losses Expenditure Framework - Analyze the impact of constraints (Cadre de dépenses à and anomalies in the flow moyen terme) of the spending on the efficiency and health sector. performance of health facilities. Niger 2008 PETS Health and Analytical In order to improve - To increase the government’s education development outcomes understanding of the link between in education and health public spending and service and to ensure that delivery at the facility level resources reach their intended beneficiaries, - To contribute to improving the there is a need for effectiveness and accountability in increased resource the use of public funds. allocation to also be accompanied by improvements in the efficiency of public expenditures. Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006), PETS 1996-2004 tables. Note: (*) Reports not available. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 13 2.2 Brief Review of Tracking Surveys’ Motivations, Objectives and Means Below we briefly review the motivations, objectives and means associated with the PETS and QSDS implemented in Sub-Saharan African countries to date. Uganda: Uganda is the country in which the largest number of PETS/QSDS has been implemented. The most recent is a PETS carried out in the water and sanitation sector in 2009. For the last two decades, the government of Uganda and foreign donors have allocated important funding to the sector to improve access and quality of services to the poor. As the funding available is suspected to be used or abused by public officials for private gain, the PETS main objective was to track the flow of funds to the water and sanitation sector and assess potential leakages. Information has been collected at the levels of central government, district, sub-counties and facilities and also comprises interviews with consumers. Tanzania: Tanzania implemented two PETS. The first, in 1999, covered the health sector and primary education, while the second, in 2001, also examined rural water and roads, judiciary, agriculture research and HIV/AIDS. 7 As in Uganda, the motivation for the tracking surveys was linked to the lack of information at the central level on resource use through the various levels of the administrative apparatus. It was strongly suspected that public resources were not used for stated purposes. The first survey focused on non-wage health and education expenditures to hospitals, health centers and primary schools, through district administrations. The second survey, which was part of a Public Expenditure Review (PER), focused on pro-poor expenditures and assessed the efficiency of budget execution in key social sectors.8 The objectives were to review and assess government procedures and channeling of resources in social sectors, as well as administrative responsibilities and reporting mechanisms at each administrative level. Information was collected at three levels of the public hierarchy. At the central government level, information on official budget estimates, disbursement and procedures were collected. Local government administrations (district and urban councils) provided information on resource use and transfers. Finally, information on resources availability for service delivery was collected from schools and health facilities. These two tracking surveys provided information on disbursement procedures and delays for two types of funds (salaries and other charges). Leakage between administrative levels was also estimated. However, data sets are not comparable between the two surveys because of differences in methodology and sampling issues. Ghana: The motivation for the Ghana 2000 PETS arose from Ghana‘s Poverty Reduction Strategy (PRS), which recognized that a significant gap existed in the 7 REPOA/ESRF (2001) 8 While the 2001 PETS in Tanzania officially covered public interventions in rural roads, judiciary, agriculture research and HIV/AIDS, no information on the tracking exercise in these sectors are provided in REPOA/ESRF (2001) PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 14 access and utilization of basic services by the poor. Issues that were identified as crucial were quality, equity, efficiency and financing gaps (Ye and Canagarajah, 2002). With these considerations in mind, the objective of the tracking survey was to improve the efficiency of public spending and improve outcomes in social sectors. Decentralization policies had been put in place to improve the availability and access to services. The PETS was consequently intended to enhance transparency and accountability of public expenditure allocation. As in Uganda and Tanzania, the PETS implemented in Ghana sought to estimate leakage of public funds in the transfer process from the central government to public service facilities through district authorities, in basic education and primary health care. The survey represented a pilot project to measure actual expenditures in the two social sectors. It collected information at the central level, district administrations, health clinics and primary and junior secondary schools. Sample selection was structured to allow linkages with the Ghana Living Standard Survey conducted in 1998, which in turn allows matching between households and facilities data sets. Rwanda: Two PETS were implemented in Rwanda: the first was conducted in 2000 in the health sector, while the second, in 2004, covered education, health, together with water and agricultural projects.9 The objective of the first survey was to identify potential leakage and sources of delays in the flows of funds from the MOF to service providers through the administrative system, and to recommend measures to improve use and flows of resources. The second PETS pursued various objectives, especially a) to assess the system of resource allocation in the social sectors, b) to analyze the degree of compliance with procedures and rules for public expenditures, c) to evaluate delays and leakage of public expenditures funds and their impact on the attainment of planned objectives, d) to analyze account keeping and reporting mechanisms, e) to evaluate the outputs and perception of the beneficiaries In practice, the Rwandan survey was essentially an impact evaluation of three funding programs (the Fund for Genocide Survivors (FARG), the Education Support Fund for Vulnerable and Poor Children, and the Capitation Fund). It also comprised a diagnostic study of salary payments to primary school teachers. Zambia: A combined PETS-QSDS was carried out in Zambia in 2002 in the basic education sector. Education in Zambia was recognized by the government as a key component of development in a PRS paper. In particular, the survey was intended to evaluate the impact of a specific funding program to schools supported by international donors, the “Basic Education Sub-Sector Investment Programme� launched in 1998 (Das et al, 2004a). The survey traces rule-based and discretionary expenditures flowing from the central government to schools via provincial and district administration, in order to identify leakage of funds, inefficiencies and delays. 9 Information on the latter three sectors is, however, not available in the Rwanda PETS survey report, Rwanda (2004). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 15 The report also includes a funding-equity exercise that examines whether educational expenditures could be considered progressive (i.e. if education expenditures translated into more funding for poorer students). The evaluation provided a wide picture of public and private funding to primary and basic education. It introduced two innovations in the tracking survey methodology. The scope of the survey was extended by adding a household survey, which allows exploration of linkages between frontline providers and users of services. Furthermore, this was the first attempt by a PETS-QSDS to measure educational outcomes, which it tries to trace back to educational expenditure. Along with the survey, pupils were tested in two consecutive years, which allowed derivation of learning achievement measures. Mozambique: A combined PETS-QSDS was also implemented in the Mozambican primary health care system that same year. The health system in the country had rapidly expanded in the last three decades. However, there was evidence of problems in service delivery, including low quality, shortage of drugs and equipment, low staff morale and informal charging. (Lindelow et al, 2004). Furthermore, the process whereby resources were allocated between line ministries to districts and facilities was not well known. A further concern, as in most other tracking surveys, was that resource allocated to the health system did not reach facilities that provided services. The objective of the tracking survey was a) to assess the functioning of primary health care services, b) to provide quantitative and factual evidence on how the financing and logistical arrangements for primary health care facilities operated in practice and to determine whether resources reached their destination, and c) to assess the impact of these arrangements on the capacity of facilities to deliver services. The survey further hoped to provide baseline data against which policy reforms and progress could be assessed. The survey collected information from public health care facilities and district administrations. A staff survey and a patient exit poll were also administered. The tracking survey focused on the allocation and execution of non-wage recurrent budgets through the administrative process, facility infrastructure, inputs, outputs, management, user fees and human resources (Lindelow et al, 2004). Madagascar: In Madagascar, in the aftermath of the political crisis of 2002, the government had decided to use part of the HIPC funds to pay for the tuition fees of all public primary school children in order to try to increase enrolment. Indeed, even compared with other Sub-Saharan African countries, Madagascar has a low enrolment rate, as 60% of urban children and only 12% of children in rural areas completed primary school (Francken, 2003). In order to evaluate the effectiveness of this measure on enrolment, the government, with the help of donors, launched a PETS in 2003 to collect information on the impact of the policy. The study was also expected to shed light on the effectiveness of decentralization policies given that the education and health sectors were the most decentralized PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 16 sectors of the country.10 In this context, an important institutional player analyzed in the survey is the district administration. The main objective of the study was to provide information on expenditure allocations and leakage in the education system, specifically a) to quantify the amount of funds that actually arrive at the district education level, b) to quantify the amount of funds and materials that should arrive and that actually arrive at the local facility level from the two main funding programs in the education system (one administered by the World Bank, the other by the MOE). Three distinct surveys were implemented. The first was put forward to evaluate the socio-economic situation following the political crisis. Public primary schools were surveyed on financial data, especially financial contributions to the schools. The second survey was organized to track budgets at the school level. The third survey was organized at the district level to gain insights into budget allocation. Kenya: In Kenya, significant deterioration in health outcomes were observed despite increased health spending. According to a PER, health outcomes and access indicators deteriorated in the 1980s and 1990s as infant mortality, under-five mortality and maternal mortality rates had increased and life expectancy had fallen.11 These results were perceived to be associated with an ineffective health system as services failed to reach targeted people due to diversion of resources, weak incentives, poor accountability mechanisms, and lack of demand attributable to low incomes in a context of increased HIV/AIDS and poverty. Furthermore, in the education sector, despite bursary funds created to support needy children, poor children had continued to drop out of secondary schools due to prohibitive school fees. Poor targeting of programs and fund capture were suspected. The government wanted to continue to shift resources toward core poverty programs such as primary health care and education. However, it was crucial to track the expenditures on some key inputs and services to determine where and how allocations were spent and whether they were benefiting the poor, as intended. A PETS was put forward in 2004. The objective was to provide information on the process of resource allocation to the service providers in the heath and education sectors in order to improve the effectiveness of the MOE bursary program and primary health care services. Specific objectives included: a) to determine the amount and criteria used in allocation of funds and inputs for health centers and dispensaries, b) to assess the quality of services provided to the patients, c) assess the extent to which the 10 The public administrative structure in Madagascar is heavily concentrated. For the budget as a whole in Mozambique, 88% of expenditures are executed at the central level. In contrast, 55% of recurrent expenditures in health and 69% in education are allocated to the provincial and district levels (Francken, 2003). 11 The infant mortality rate had increased from 65 to 74 per 1,000 births, under-five mortality had increased from 100 to 114 and maternal mortality from 480 to 590 per 100,000 births (Republic of Kenya, 2004, p.11). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 17 management of funds in the bursary program complied with MOE procedures; d) to assess the impact of the bursary scheme in terms of its contribution in improving access and completion rates in secondary schools (Republic of Kenya, 2004). Chad: A combined PETS-QSDS was organized in the Chadian health sector in 2004. The Chad PRS paper considered the health sector a central pillar of the government’s strategy to revive growth and alleviate poverty. In a context of increasing oil revenues and public expenditures in social programs as part of the PPTE Initiative, the government of Chad and international donors wanted to ensure that funding would contribute to poverty reduction objectives by reaching its destination and benefiting the population. The Chad survey focused on the health system as a whole (primary, secondary and tertiary health care providers).The objective of the survey was to compare resource access and level of services in health centers of all the country’s regional delegations. Specifically, the objectives were a) to analyze the impact of and bottlenecks and constraints in resource allocation and their impact on efficiency and performance of health care facilities; b) to provide data on basic characteristics of health centers, in particular those that significantly affect public service quality; c) provide information on potential factors explaining differences in performance between health facilities and types of ownership and identify means to improve the situation; d) analyze the impact of service quality on households’ demand for health care service (Gauthier and Wane, 2005). In addition to the facility level data, information was collected at all levels of the public administration (MOF, MOH, regional delegates, district head doctors and regional pharmacy managers). Information was collected on resource use, delivery processes, health output and pricing behavior. The focus was on all health center ownership categories, public, private for-profit and non-profit providers. A staff questionnaire and a patient exit poll were also administered. Niger: In Niger, structural constraints were seen as limiting the effectiveness of public spending in improving development outcomes. Ensuring that expected increased resource allocation reach their intended beneficiaries were seen as essential to Improvements in the efficiency of public expenditures in order to improve development outcomes. A PETS was carried out in 2008 to increase the government’s understanding of the link between public spending and service delivery and to contribute to improving the effectiveness and accountability in the use of public funds. The PETS focused on identifying the discrepancies, inefficiencies and delays in public spending execution for selected expenditures in the education and health sectors. Specific resources were tracked. For the education sector, textbooks and notebooks were selected whereas for the health sector, patient food expenditures, hospital supplies and essential medicines were tracked. Despite poor availability and accuracy of records keeping, several findings emerged from the survey. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 18 Several other surveys were implemented in Africa (e.g. in Cameroon, Senegal, Namibia and Sierra Leone). 2.3 Summary  The original motivation of the Uganda 1996 PETS survey – i.e. to explain poor performance and identify leakage--was also generally the main motivation of the following surveys.  Most PETS and QSDS have been motivated by common goals of increasing information on social sector spending to understand the weak link between public expenditures and outcomes.  The surveys have been implemented in a context of weak institutional settings, marked by deficient accounting, monitoring and reporting procedures.  In some cases, a number of other goals were added, in particular impact evaluation of programs, such as in Ghana, Rwanda and Uganda, as well as more in-depth analysis of provider performance through QSDS. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 19 3. INSTITUTIONAL ARRANGEMENTS: ADMINISTRATIVE STRUCTURE OF PUBLIC EXPENDITURE FLOWS Given that PETS/QSDS are designed to gather information on public expenditure flows, resource uses and service sector agents’ behavior, an adequate understanding of the institutional arrangements through which public resources are allocated to service providers is fundamental. In this section, we discuss the varied nature of these administrative structures and flows in order to apprehend the task facing tracking surveys and the methodological choices that are required to adequately track resource use and flows. 3.1 The Public Administration Resources Flow Structure Resources flows in public administrations social sectors are complex. The various resources required for public service delivery (financial resources, human resources and in-kind transfers) originate from several sources (central government ministries, decentralized administrative levels, bilateral and multilateral donors, NGOs), and take various routes in the organizational system. In addition, these flows are generally governed by different allocation rules, administrative processes, recording and accounting procedures, etc. An essential initial stage in any tracking surveys is thus to identify and analyze the nature and characteristics of these various administrative structures and flows, in order to grasp their role and contribution to service providers’ resources. Figure 2 represents a generic administrative structure and resource flow system in the social sector. A three-tier hierarchy is represented (central, provincial/regional and district/local government) through which resources flow to local service providers. In countries such as Nigeria or Uganda, the administrative structure governing social service providers’ resources flows comprises only two layers (e.g. central and district/local government). In most cases, central government ministries allocate resources (human, financial and/or in-kind) to the various layers of the administrative structure. These resources pass through various channels. Financial and in-kind resources might be the responsibility of a line ministry while human resources could be the responsibility of a civil servant ministry. Other important actors to consider in the service provider supply chain are donors. In most developing countries, international and national donors are pivotal to social sectors’ allocation process. Some donors provide program or sector financial support to the central government, which is then responsible for administering and allocating resources to the various sectors. Donors also provide financial or in-kind support to provincial or district administrations, and sometimes even directly to service providers (e.g. school items, PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 20 health care materials, drugs or equipment). Donors are also involved in capital investments such as construction of schools, clinics and hospitals. FIGURE 2 FUNDING FLOWS IN SOCIAL SECTORS GENERIC CASE Central Donors Government Provincial Administration District Administration Service Providers Funds In-kind Transfers Staff Remuneration In cases of a combined PETS/QSDS, the focus of the inquiry, in addition to public providers, also includes private for-profit and non-profit providers. The relationship with private for-profit and not-for-profit providers is always important to grasp, especially with respect to the environment of the health or school systems, the kind of clinics or schools operating in the country, the mixture of public, private, religious and community facilities, etc. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 21 3.2 Administrative Structures and Resource Flows in Practice Each country is characterized by institutional structures and administrative processes that possess unique specificities and idiosyncrasies. We review a few countries’ administrative structures using survey reports to illustrate the complexity of funding flows and the need to properly understand the institutional structures, in order to develop an adequate methodological approach to implement tracking surveys. The challenge of adequately understanding the institutional arrangements is emphasized by Ye and Canagarajah (2002, p. 3) about the Ghana education and health PETS: The biggest challenge that we encountered in conducting this tracking survey was to understand the systems of public resource distribution in Ghana. On paper, all resources at the level of the central government are allocated by the Ministry of Finance to the line ministries with clearly marked heads and subhead, indicating where the funds should go. In reality, however, the distribution systems of public spending are rather complicated. In Mozambique, as reported by Lindelow et al (2004), the health sector is composed of a three-tier administrative hierarchy responsible for the management of health services (Figure 3). Health providers (hospitals, health centers and health posts) are under the administrative responsibility of District Directorates of Health (DDH). Their responsibilities include planning human resources and budget management, provision of medical supplies and supervision of service providers at the district level. DDH are under the responsibility of Provincial Directorates of Health (PDH), which coordinates the administration of health activities in the province. They enjoy considerable autonomy with respect to allocation of staff, medical and non-medical resources and supervision. Provincial directorates report to the Ministry of Health (MOH), which allocates their resources. Public health care providers in Mozambique, with the exception of hospitals, do not receive any financial resources from the public administrative system. The only sources of financial resources at the provider level are user fees charged for consultation services and drugs. In-kind resources (medical and non-medical materials, drugs and vaccines) are allotted to service providers through a complex administrative and logistical system. The DDH has a central role: distributing in-kind resources to health centers and paying salaries to health workers. Some of these resources are procured directly at the district level through a district logistic unit, but other non-wage recurrent resources (e.g. drugs, vaccines, medical equipment) are procured directly by the provincial or central levels. Further, the investment budget is the responsibility of higher levels. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 22 FIGURE 3: FLOW OF RESOURCES IN THE HEALTH SECTOR IN MOZAMBIQUE Donors Central govt. Central logistics units Financial support Procum. Procurm. drugs, Salary Non-slry vaccines, Med. In-kind budget recur. recurr. Vaccines equip. Drugs support equipment, budget etc. Provincial admin. Provincial logistics units Financial support Procurm. equipment, Salary Non-slry material, Med. budget recurr. Vaccines equip. Drugs etc. budget In-kind support District admin. District logistics units Financial support Procurm. equipment, Salary material, Med. budget Vaccines equip. Drugs etc. Health facility Source: Lindelow (2006) Resources flows in Figure 3 are differentiated by salary, financial flows and in-kind transfers (drugs, vaccines, medical material, etc). The allocation routes taken are also illustrated. In Chad, the health system is composed of a mixture of public, private for-profit and non-profit service providers. The public health system is structured around four levels of responsibility (Figure 4). At the central level, the MOH is in charge of formulating health policies and allocating financial resources. At the intermediary PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 23 level, 14 Regional Health Delegations (RHDs) are in charge of the implementation strategy and managing health personnel. At the decentralized level, 49 sanitary districts are in charge of implementing services within responsibility zones and overseeing health centers (Gauthier and Wane, 2005). FIGURE 4 FUNDING FLOWS IN THE HEALTH SECTOR IN CHAD Donors MOF MOH MHR CPPA Decentralized Credit Regional Health Delegation PPA District Health Delegation Service Providers Funds In-kind Transfers Staff Compensation As in Mozambique, public health care providers (except some hospitals) do not receive any financial resources from the public administrative system, only in-kind resources. The user fee is the only source of financial resources at the local provider level. Public resources arriving at decentralized levels from the Ministry of Health (MOH) are the sum of four components, namely (1) centralized credits, (2) decentralized credits, (3) ad hoc requests, and (4) medication and vaccine delivery. Centralized credits are resources purchased by the MOH (essentially PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 24 materials and medications), intended for regional and district administrations and health centers.12 Delegated credits are budgetary resources given to regional or district administrations. At the decentralized level, regional health delegates or district head doctors are responsible for managing these financial resources, and for redistributing purchased materials and medications to health providers under their jurisdiction. Resources are also transferred through ad hoc requests; these materials are directly requested from the MOH and collected by regional health delegates and district heads in the capital. Moreover, the MOH administers a budget for medications and vaccines destined for health centers and hospitals.13 The budget is administered by the Central Pharmaceutical Procurement Agency (CPPA), which has a monopoly over drugs and medical products sold to the 14 Prefectoral Purchasing Pharmacies (PPP), as well as to the public and non-profit health facilities. All health facilities, including private for-profit and non-profit facilities, must produce monthly reports for district health administrations. In Zambia, in the education sector, public resources also flow through three administrative levels before reaching primary schools (Das et al, 2004a).The administrative structure comprises the Ministry of Education (MOE), province office and district offices (Figure 5). Two distinct types of provinces are observed, decentralized and centralized provinces. District education boards were created in the first type, but not in the second. Resource flows in the administrative system do not follow a simple top-down approach. At each level of the hierarchy, funds may be received directly from the central government or donors. In the case of centralized provinces, all funds are first allocated to provinces, which then determine the allocation among districts, while in the case of decentralized provinces, a portion of the discretionary funds are allocated directly from the MOE to districts, which then redistribute the funds to schools. In both centralized and decentralized provinces, donor funding administered by the MOE is partly allocated directly to districts.14 There is no movement of rule-based funding from the provinces to districts; all of these transfers are made directly from the MOE to districts. Discretionary funding (cash transfers) transit through the province in centralized provinces,15 but is received directly by the districts in decentralized provinces. At the school level, tuitions and other fees are paid by households. 12 The official rationale for centralizing purchases at the MOH level is to benefit from economies of scale through public tendering procedures, combined with a lack of local capacity and suppliers. 13 Medications and vaccines are formally included in the centralized credits, but follow a different path through the CPA and PPAs (Gauthier and Wane, 2005). 14 BESSIP program. 15 Including MOE and case IV donor fund. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 25 FIGURE 5 FUNDING FLOWS ACROSS CENTRALIZED AND DECENTRALIZED PROVINCES IN THE EDUCATION SECTOR IN ZAMBIA Case IV donors Ministry of (primarily the Program Education for Advancement of Girl’s Education) Province Province (Centralized) (Decentralized) District District (Centralized) (Decentralized) Discretionary flows Households Schools Rule-based flows Source: Das et al (2004a) The two most complex administrative processes observed in tracking surveys realized to date in Sub-Saharan Africa are found in Senegal and Cameroon. The PETS reports (République du Cameroun, 2004; World Bank, 2006) describe in some detail the budgetary and administrative processes, which in Senegal involve nine procedures, including a Presidential committee for the approval of the district level budget. At the other end of the spectrum, the administrative structure of the education hierarchy in Uganda examined in the first PETS was relatively simple. The structure of responsibility involved the MOE and district education boards, which are directly responsible to the MOE and supervise school facilities. As in Zambia, private funding to education was considerable at the time of the survey; parents were expected to pay for textbooks, uniforms, school supplies, PTA dues, etc. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 26 3.3 Information Flows and Processes The preceding figures illustrate specific resource supply chains in social sectors. However, in addition to allocating resources (funding, material, personnel), a sector’s institutional structure is characterized by accounting and recording procedures and serves to carry information flows. Certain information about budget and resource allocation flows in a top-down direction, while other information circulates in a bottom-up direction (information regarding the reception of resources, use of input, activities, and needs, etc.) Understanding such information flows and recording procedures is necessary to determine the specific possibilities of a tracking exercise. It is of course important to distinguish between official accounting or recording rules, monitoring procedures and information flows, and what is effectively observed in practice. In Chad, for instance, the information flow system at the decentralized level is characterized by the existence of four main flows (personnel, centralized credits, decentralized credits and ad hoc deliveries). Regional health delegates (as well as regional hospital heads) are informed at the beginning of the year of their annual decentralized financial credit allocation as well as the personnel allocated to their region. In contrast, centralized funds, which are converted at the MOH level into in- kind transfers (medical material, medication, etc.) are discretionary and thus not accompanied by information about their official allocation to regional levels. Note that district health delegates and facility heads do not receive any budgetary information about resource transfers except for the quantities that actually arrive at their administrative unit level. Indeed, regional delegates are thus responsible for allocating financial or in-kind resources to lower levels of the administration, but without any information flows. In-kind transfers sent from the MOH central warehouse arrive at regional warehouses. A “Bon de sortie� is used at the central level and a “Bon de livraison� is used at the regional level to confirm reception of specific quantities of materials. A copy is sent to the center, which is then dated. In addition, some information also flows bottom-up, as regional and districts heads are allowed to transmit specific resources requests to the MOH, through the ad-hoc delivery channel. As mentioned above, ad-hoc deliveries are specific supply pickups made by health officials (regional or district officers) and thus do not involve any other information. Information about services performed at the local facility level is provided monthly by facility heads through a monthly activity report (RMA) to the district official. Information compiled at that level is then sent to regional health administrations and then to the MOH. In all surveys, these supply and information flows and procedures need to be identified and analyzed in order to determine what is amenable to tracking and what should be tracked in order to fulfill the project’s objectives. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 27 3.4 Summary  Social sector institutional arrangements involve complex administrative structures and rules through which supplies and information flow.  Most institutional arrangements are characterized by several administrative layers involved in the supply chain, and leakage could happen at every levels.  Once the administrative procedures and resource flows of the social sectors under study are identified, it is important to determine which resources should be tracked. Indeed, any PETS-QSDS surveys require determination of the flows on which information will be collected and at which levels.  Before addressing these methodological questions in section 5, we examine in the next section some of the main findings resulting from the various PETS and QSDS in Sub-Saharan Africa. We will then try to link these findings with the specific choices of survey methodology in order to identify potential difficulties and challenges of tracking surveys. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 28 4. MAIN FINDINGS In this section, we review some of the main findings of PETS and QSDS. We examine in particular results concerning information problems and leakage of funds and other resources in the administrative and service delivery process. Absenteeism and ghost workers are also analyzed, together with delays and user fee issues. 4.1 Leakage of resources One of the key finding of tracking surveys has been, of course, evidence of public resource leakage. As previously mentioned, the capture and dissipation of resources for purposes not related to the sector objectives is linked to the presence of moral hazard created by information asymmetry and differences in objectives between principal and agents. This phenomenon is also associated with inadequate incentives and improper monitoring and enforcement within the service delivery system. Local officials and politicians, for instance, could take advantage of the gap in information with central government and citizens; they could reduce disbursement or procure few supplies for health centers and schools under their jurisdiction, which they know would attract little attention given the weak supervision in most institutional settings in social sectors in Sub-Saharan Africa. Some surveys have been able to document the problems in detail. Table 2 presents the average leakage level observed in the various countries and the flows of resources on which these observations were based. It should be noted that leakage, generally defined as the share of resources intended but not received by front line providers, has been defined in two ways in tracking survey reports. The original or “strict� definition of leakage was introduced by Ablo and Reinikka (1998) for rule-based expenditures as the share not received with respect to the expected (fixed-rule) entitlement: resources received by facility “Strict� leakage = 1 - resources intended for the facility For non-fixed allocation rule flows (discretionary funding), Lindelow (2006) for instance proposed a “narrow� (or soft) leakage measure as simply the share of resources sent at a certain level and not received at the other, in particular the facility level: ∑ resources received by facility “Narrow� leakage = 1 - Resources disbursed by higher level PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 29 TABLE 2 LEAKAGE OF PUBLIC FUNDS Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause Uganda 1996 PETS Education Education: Education: Education: Education: Education: and Capitation grants -87% (on average) during -Leakage appears -Large variations in -Asymmetric Health 1991-1995 principally at the district leakage across information has Health: -Leakage of salaries much level schools: adverse effects on the smaller -Resources either - Larger schools flow of funds to Non-wage disappeared for private frontline providers and expenditures appear to receive gains or were used by larger share of the service delivery Health: Not defined. district officials for intended funds (per -Schools with greater purpose unrelated to student). capacity to influence education local officials are - Schools with children of better off granted higher shares Health: Most transfers parents experience from government are in- lower degree of kind. A quantitative leakage assessment of the flow of - Schools with higher resources to health share of unqualified centers or service teachers experience delivered could not be more leakage achieved. 1999 and PETS Education -Capitation -Reduction of leakage from -Information campaign is -Shows the value of -Improvements are 2000 grants 87% in 1991-95 to about estimated to account for transparency and associated with better 18% in 1999 and 2000 about ¾ of the efficiency of information about improvement in leakage mobilizing civil society school entitlements against corruption through radio and newspaper campaign 2000 QSDS Health - Financing -Some evidence of drug -Excessive drug use - Drugs, leakage, but average figures in public and private vaccines and not provided. health facilities. This supplies could be associated with over-prescription or leakage. 2009 PETS Water - Water and -Evidence of funds leakage -Procurement sanitation funds but no specific estimates malpractices at allocation District level -Inadequate adherence to accountability procedures and guidelines for PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 30 Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause provision of safe and cleaned water to the community -Incompetent Contractors -Greed and Conflict of Interest Tanzania 1999 PETS Education - Non wage -Leakage was estimated at - Leakage appears at the and Health expenditures 57% in education and 41% district level in health care - Salaries appear less prone to diversion 2001 PETS Education - Non-wage -Average figures for leakage -Decentralized funds -At the district level, - Lack of predictability and Health expenditures not provided (OC) sent to districts are treasury and sector of the disbursement -Difference between essentially all consumed heads tend to promotes leakage disbursement and receipts at the district level. reallocate non-wage especially at the sub- between Treasury and -Only material sent by the expenditures in favor national level. councils is estimated at 18% center to districts is of activities that -Highly aggregated on average for non-wage partially redistributed to benefit the council government records funds for July 1999- facilities. staff at the expense of are reported to June2000 and July- facilities (e.g. undermine -No cash funding below traveling, vehicles, December 2000. the sector/district level, transparency fuel versus school only in-kind material material and medications) 2004 PETS Education The inflow of development On average, councils only The book part of the grant at the school level is acknowledge receiving capitation grant is the 84% of the central level 59% of the development main cause of the disbursement. The inflow of grant sent to the regions. leakage in the transfer overall capitation grant at of capitation grant. the school level is in the range of 54%-64% of the central level disbursement. The inflow of the cash part of the capitation grant at the school level is 76-86% of the central level disbursement. The inflow of books at the school level is 28% of the central level disbursement. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 31 Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause Ghana 2000 PETS Education - Non wage - Leakage estimated at -Large proportion of -Possibilities of -In-kind nature of and Health expenditures about 50% of non-wage leakage seemed to occur leakage were found to transfers increase - Salaries education expenditures between central be much greater information and 80% of the non-wage government and district when the value of asymmetry and lack health expenditures offices (instead of material distributed of accountability in the - Leakage of salaries is between district and was unknown to delivery system and estimated at about 20% facilities), during the recipients reduce feedback from procurement process frontline facilities when public expenditures are translated into in-kind transfers. Rwanda 2000 PETS Education - Recurrent - Some evidence of -Except for staff salaries, -Lack of accountability -District offices and Health expenditures leakage between regions recurrent expenditures in in the use of public accounts are credited and districts, but no firm health and education do funds and other at the discretion of the estimates not reach schools and resources contributed regional offices. health facilities. by users, parents, -The lack of budgets NGOs, donors and and guidelines for the other development use of funds was cited agencies. by health officials as the source of major inefficiencies and causes of delays and potential leakages. 2004 PETS Education -Teachers’ - Some evidence of -There are no controls for -Anomalies are noted salary leakage of capitation the utilization of capitation in the list of - Three funding grant at the school level in grants by schools, as beneficiaries of the programs: particular, but no firm neither the MOE, Education Support estimates. province or district have Fund program at the i) Funds for control mechanisms in central level (the first Genocide - Potential leakage of the Education Support Fund place. names of Survivors beneficiaries are not (FARG) program at the central -District reports are said level, but no estimates are to be unreliable and listed, neither their ii) Education provided. audits of MOE does not exact birth dates or support Funds cover all schools. name of their for Vulnerable parents), which and Poor introduces potential Children, misreporting. iii) Capitation funds. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 32 Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause Zambia 2001 PETS Education - Non-wage -Leakage of 10% for fixed- -Rule-based funding -Rule based funds are -For rule-based funds, QDSD funding for basic rule grants reaches almost every progressive as delays in education (fixed- -76% for discretionary non- school, while greater per pupil disbursement may a school grant, wage expenditures discretionary funds are funding is observed in factor. discretionary mainly used at the district poorer schools -For discretionary non-wage grant and province levels. - Discretionary funds, the few schools program) About 20% of schools disbursement higher that received large receive any funding from to richer schools in amounts have greater discretionary funds rural areas and wealth bargaining power with neutral in urban areas higher administrative -Overall, public levels. funding is regressive: almost 30% higher allocation to richer schools. 2007 PETS Health Various resource A fifth of the health facilities While allocation to the The allocation rules QDSD flows received resources less districts is equity- manifest themselves (government than their intended enhancing, allocation to clearly in the highly budget, basket allocations. hospitals follows inequitable provincial funds, vertical Medical staff experience principles of historical receipt of resources, project funds non-receipt of the full budgeting that tend to where the poorest, and internally amount of salaries (about 15 perpetuate installed most remote and least generated funds) percent of staff); capacity, even if much of urbanized provinces unauthorized salary that existing capacity was receive the lowest per deductions (15.5 percent of heavily influenced by capita MOH releases. staff); and staff payment of mining rather than public “expediter’s fee� to obtain health concerns. salaries (10 percent of staff). Mozambique 2002 PETS Health - No firm estimates of - Unreliable data on QSDS leakage district budget at the - Some evidence of leakage district and provincial of drugs in the transfer from levels make it difficult to provinces to districts assess whether resources reach their intended beneficiaries Nigeria 2002 PETS Health - No firm estimates of - Evidence of capture of -Lack of accountability QSDS leakage rent by local government of local government - But report of high degree officials and important leads to public of leakages in the more rural problems of non payment resources capture state (Kogi) of salaries of health by local officials. workers Senegal 2002 PETS Health Decentralization - No firm estimates of -Decentralized local - Inconsistent data fund leakage governments have on resources sent different priorities than and received PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 33 Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause - Some evidence of leakage health and could use between levels at regional and communal resources earmarked to levels in non-wage health for other purposes. expenditures from central level to providers Cameroon 2003 PETS Health - Non-wage - No firm estimates of recurrent leakage expenditures 2004 PETS Education Various resource - No firm estimates of flows leakage Madagascar 2003 PETS Health - No firm estimates of leakage QSDS Education IPPTE - Leakage of cash funds at - Leakage is associated - Schools are -Lack of information, CRESED the lower echelon of the with remoteness as 56% uninformed about dysfunctional education sector is of schools in the decisions taken accounting system perceived to be low (10% Communes at far higher up in the and absence of and 8% for the two main distance from the capital education sector allocation rules at the sources of cash of the province show system concerning district and school contributions to schools). leakages in comparison their potential level increase the - Leakage of material is with only 21% of the resources and incentives for more common as 28% of schools in the Communes possibilities. leakage. schools report receiving less close to the capital. -Only 35% of schools material than stated by the reported knowing at district. the beginning of the year what they were supposed to receive from the district level (Cisco). 2006- PETS Education Education: Education: Education: Education: Education: 2007 and Health Cash (caisse -The ratio of non-received -Leakage of the “caisse -Smaller schools, -Low financial école), school funds to expected funds for école� is high for some schools in poorer and capacity at the kits, school the school year 2005-2006 schools: in particular less educated areas decentralized levels equipment, from the Ministry of smaller schools and seem to suffer more textbooks and Education to Cisco and from schools in ethnically from leakages. salaries. Cisco to school level heterogenous areas Health: Health: equaled 3% and 4% suffered more from respectively. -A high 40% of leakage in 2005/2006. The likelihood of Current the schools did not receive leakages in the drug expenditures, Health: supply chain all the equipment that they non medical were entitled to during the Leakages of antibiotics increases with drug consumable, last two school years. are especially price drugs problematic: half of the Health: basic health centers that Averaging across survey PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 34 Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause rounds, 73% of the ordered this drugs did not commune pharmacies receive what they were report leakage in the drug supposed to receive. supply chain from district to commune levels. Namibia 2003 PETS Education - No firm estimates of - Incomplete records - Potential leakage of - Poor record keeping QSDS and Health leakage make it difficult to assess medication and other and few records of the whether resources reach material delivery of material to their intended district and school beneficiaries level are available and could increase the incentives for leakage. Chad 2004 PETS Health -Non-wage - Only about 27% of non- -Public resources arrival -It was estimated that -Main factors QSDS recurrent wage budgetary officially rates vary considerably if all public resources explaining low level of expenditures allocated to regions by the among regions. officially budgeted for resources received at MOH reaches regional -The highest rate of regional delegations the local level are:1) health delegations leakage is observed in had reached the the very high rate of -Less than 1% of non-wage the BET region, the most frontline providers in resource budgetary officially allocated remote area of the 2003, the number of centralization at the to regions reach local health country patients seeking MOH level, 2) the lack centers primary health care in of supervision and - A large proportion of Chad would have control of resources, leakage seemed to occur more than doubled and 3) lack of between central during the year planning in the government and regional allocation of delegation during the resources as procurement process via allocations are so called “centralized arbitrary at every credits� levels. Kenya 2004 PETS Education Health: Health: - An audit trail of the Health: and Health -Leakage of total funds Leakage is more bursary funds -Provinces and received at the health center pronounced in health released was not districts are unaware level is estimated at 38% centers than possible because of of budgets and dispensaries. the lack of proper programs. - Leakage of user fees at accounting system the facility level estimated at Education: -Supervisory capacity 25% -The bursary program is of provincial and - Leakage of Community not thoroughly audited district authorities are development funds at facility which increases insufficient. level is estimated at 37% possibilities of leakage -Financial and Education: -Criteria of selection of accounting systems vulnerable and needy are inadequate -More than 80% of schools did not receive their entitled students not followed by Education: amount of bursary funds many schools -Financial PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 35 Survey Resources Country Year Type Sector tracked Leakage Observation Other comment Cause -Evidence that some management of the schools are receiving more school is in the hands allocation than required and of the head teacher that funds are diverted for with minimal influence personal gains of the PTA and BOG. -Total leakage of bursary -Lack of information at funds estimated at 35.8% the school level leads to non accountability of public resources -Poor school records and lack of proper audits Mali 2005 PETS Education Schools received on The leakage rates vary the leakage rate The absence of QSDS average 40% of the total considerably among increases with the transparency on number of textbooks CAPS and can exceed allocation amount, budget information originally allocated to them 90% as is the case for resulting in very high facilitates the capture by the MoE. Baraouéli or leakage volumes for of the education Torokorobougou (in schools which have budget Bamako). been allocated large numbers of textbooks Niger 2008 PETS Education Selected - No firm estimates of Education: Health: and Health government leakage -General lack of Other aspects of the expenditures in systematic information financial management education recording. at DRSPs, however, (textbooks, could be improved. notebooks and -Budget increases did not always lead to spending Record keeping could drawing books be more detailed in for students) and increases. terms of exact dates in health (food - Supplies do not arrive at of financial expenditure and facilities on a regular transactions, amounts hospital basis. and breakdown of supplies, and Health: spending. essential medicines) The quantity of financial resource transfers between MH and the Regional Health offices (DRSPs) are well accounted for. The regional offices are also effective in procuring materials once credit becomes available. Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006), PETS 1996-2004 tables. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 36 Caution has to be exercised with respect to the interpretation of leakage levels (in particular, but not exclusively, the “narrow� measure), as reasons other than corruption might explain low arrival rates or observed differences of resources between levels (Reinikka and Svensson, 2004a; Lindelow, 2006). These include incomplete records or problematic accounting, data collection problems and data entry error. We will discuss these issues in detail in Section 5. Furthermore, the normal use of resources at an administrative level has to be accounted for in the leakage calculation. However, it is often difficult to measure input use at a certain level. For instance, in the Ghana report an 8% administrative overhead cost was used to measure (narrow) leakage.16 As the leakage figures in Table 2 show, several PETS and QSDS in Sub-Saharan Africa have identified leakage on certain public resources on a very large scale. For instance, the initial Uganda PETS estimated that 87% of non-wage capitation (per student) grants for the period 1991-95 did not reach destination, being diverted for a purpose other than education (Ablo and Reinikka, 1998). In Tanzania and Ghana, leakage on non-wage education expenditures was respectively 41% and 50%, while only 1% of non-wage health expenditures to regional heath administrations were estimated to arrive at the health facility level in Chad. Such high rates of leakage of non-wage expenditures could have dire consequences for public service delivery (Reinikka and Svensson, 2005b). When 50% (Ghana), 87% (Uganda) or 99% (Chad) of funds for supplies (medical and non-medical supplies, books and other schooling materials, i.e. non-wage inputs) do not reach health centers or schools, leakage prevention must become a major policy issue in health and education, and possibly in other social sectors. In some tracking surveys, no firm conclusions on leakage could be made due to several factors, related mainly to the fact that the survey instruments chosen could not specifically monitor such problems, or because of implementation problems. These implementation problems could be linked, for instance, to inadequate design of instruments that did not match administrative levels or the facilities’ record practice, to inadequate supervision of survey enumerators and training, among other issues. We will discuss these questions in detail in Section 5 and relate them to methodological issues and particular circumstances facing survey teams that could affect survey’s capacity to efficiently measure diversion of funds and corruption. 4.1.1 Patterns of Leakage Levels Certain patterns in resource leakage levels tend to emerge from the survey findings, in particular in terms of: i) rule-based versus discretionary expenditures; 16 Ye and Canagarajah (2002), however, do not provide justification for their specific choice of overhead costs. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 37 ii) wage versus non-wage expenditures; iii) levels of government; and iv) in-kind versus cash transfers. i) Rule-based versus discretionary expenditures As emphasized by Reinikka and Svensson (2001), Das et al (2004a) and Lindelow (2006), the level of discretion exercised on resource allocation could influence leakage levels. Greater discretionary power granted to particular administrative units, combined with weak supervision and improper incentives, could lead to large fund capture. Indeed, differences in leakage levels have been observed between funds allocated through fixed-rule and those that are at the discretion of public officials or politicians. In Zambia for instance, rule-based funding (per- school grants) presented a level of leakage of only 10%, versus more than 76% for discretionary funding.17 As emphasized by Das et al (2004a), since rule-based funding is clearly defined by a simple allocation rule, capture of funds is more difficult compared with discretionary funds, which are not bound by any specific allocation rule. Similarly, in the health sector in Chad, where no fixed allocation rules apply for most resource allocation, it was estimated that only 1% of non- wage resources allocated to regional health administration in 2003 arrived at the facility level. However, this finding about rule-based versus discretionary expenditure cannot be generalized yet. Indeed, one of the highest leakage levels among surveys was observed on a rule-based funding (capitation grant) in Uganda.18 Of course, this observation does not constitute a contradiction to the hypothesis that rule-based funding is less prone to leakage. Indeed, in the Uganda case, discretionary funding, which could have revealed even higher levels of leakage, was not tracked. Ultimately, in countries surveyed, it was generally observed that the greater the agent’s discretionary power, the higher the leakage observed. Furthermore, funds capture, in countries characterized by relatively important discretionary power and weak monitoring, has been associated with groups’ bargaining power. In Uganda, for instance, large variations were observed across schools in reception of public resources, which were explained by relative bargaining power. Reinikka and Svensson (2001, 2004b) used econometric analysis to explain these differences across schools. They showed that smaller schools and the greater presence of unqualified teachers were associated with less fund reception per student. However, schools in better off communities experienced less leakage of funds. In particular, they estimated that a 10% increase in household income increases the amount of public funding that reaches the school by three percentage points. 17 It should be noted that Das et al (2004a, p. 25) are “agnostic� about the fact that such non- arrival rates at the school level constitute good or bad service delivery, and whether leakage is indeed taking place given that for discretionary funding, no fixed-rule governed the share of resources earmarked for service providers. Indeed, normal administrative use at the provincial and district levels should be accounted for in this figure, as well as delays in arrival, etc. 18 Average leakage level of 87% in Uganda during the 1991-95 period. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 38 Similarly, in Zambia differences in schools’ bargaining power was put forward to explain the fact that only about 20% of schools receive any resources from discretionary funding. Das et al (2004a, p.41) argue that the few schools that received large amounts of discretionary funds have greater bargaining power with higher administrative levels. ii) Wage versus non-wage expenditure: Differences in leakage levels have also been observed between salary and non- salary funds. As observed in Table 2, tracking surveys for which leakage of public funding could be measured (in particular Ghana, Tanzania, Uganda, Zambia), reveal that non-wage expenditures (channeled through intergovernmental allocation mechanisms) suffer more from extensive leakage than do salary expenditures. In the case of non-wage expenditures, local officials and politicians could take advantage of their information advantage to reduce disbursement or provide few non-wage supplies to health centers or schools, knowing it would attract little attention (Reinikka and Svensson, 2004a, p.38). In contrast, failure to pay health workers or teachers would attract attention as workers know how much they are owed. Indeed, salaries and allowances are more easily observed and incentives for the individual recipient are greater, to ensure the arrival of the fund. Salaries are also often paid directly by the central government to individual workers at the service provider level, without going through the administrative apparatus (for instance, in Rwanda, salaries are paid directly into workers’ bank accounts). Alternatively, when salaries transit through the administrative structure, they are generally paid by local authorities directly to workers, thus with the same incentives at the recipient level for ensuring full transfer. Furthermore, salaries and some other forms of staff compensation are generally governed by clearer, more fixed rules than non-wage expenditures, which could also contribute to reducing funds dissipation.19 Hence salaries and allowances seem to suffer from leakage to a much lesser extent than other categories of public expenditures. iii) Levels of government Tracking surveys have also shown, depending on the country’s specific institutional arrangements, that resource dissipation could be more extensive at certain levels of government than others. Indeed, several PETS and QSDS have observed that leakage is more pronounced at specific tiers of government (Reinikka and Svensson, 2003c). For instance, in Mozambique, Rwanda, Senegal, Uganda and Tanzania, the most serious resource leakage was observed at the local government (district) level. In contrast, in Chad and Ghana, for instance, leakage 19 In Zambia, important divergence was observed in terms of rules governing workers’ allowances, with direct impact on delays and potential leakage. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 39 occurred more specifically at the central government level, before the resources reached the district level. In Ghana, it was noted that a large proportion of resource diversion occurred during the procurement of in-kind items between line ministries and district administrations. These varying leakage results are associated with different institutional structures characterized by various information asymmetry problems among parties, coupled with varying discretionary power and weak enforcement capacity. iv) In-kind versus cash transfers Furthermore, leakage has also been shown to be more pronounced in the case of in-kind transfers compared with financial transfers. This situation, observed for instance in Ghana, Mozambique and Tanzania, could be explained by the fact that the value of in-kind items is typically not known by the recipient (at the local government or facility levels). This situation tends to exacerbate the information asymmetry problem, diminishes accountability in the delivery system and reduces potential feedback from service providers. Furthermore, it is generally more complex to monitor numerous small items than less frequent cash transfers (Ye and Canagarajah, 2002; Lindelow, 2006). Such a situation was observed in Tanzania, where district officials had complete discretion over school fund allocation. Although school officials and parents knew they were entitled to some funding from the district level, because resources reaching the schools were predominantly in-kind without any indication of monetary values, school communities seldom knew the value of the in-kind support they received, which greatly reduced accountability. Similarly, in Ghana, education resources are essentially channeled to schools as in-kind resources via district offices. Here again, as schools had little knowledge of the monetary value of what they actually received, information asymmetry was more pronounced, which reduced accountability in the delivery system and increased the possibility of leakage. 4.2 Information Asymmetry and Information Flows As mentioned, information underlies the problem of resource capture and leakage in the social sectors. Indeed, a fundamental and generic problem that PETS and QSDS have noted is information asymmetry through service providers’ supply chain. In most countries examined, there is a crucial lack of information at various levels in the public organizational structure, in particular at the central level, regarding resource use and transfers through the supply chain. This problem of information is of course one of the main issues motivating most tracking surveys. Furthermore, information problems are also acute at lower levels of the hierarchy, as decentralized administrative units are generally not aware of the budgetary resources that they are entitled. Typically, the central administrative level PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 40 organizes procedures governing resource flow in the supply chain, but there is little or no consultation with lower levels and scarce transmission of transparent information. Those who receive the materials at lower levels, as previously mentioned, typically do not have information on what they were supposed to or entitled to receive, in terms of type of material, quantity, value, or when they should receive the materials and from whom. Such situations are observed in most countries, e.g. Chad, Madagascar, Mozambique, Uganda and Tanzania. The information gap and retention of information at the central level in several countries surveyed reinforces moral hazard problems. In Tanzania, for instance, information asymmetry was associated with the use of a cash budgeting system at the central government level. The requirement for the government to avoid running deficits, coupled with the prioritization of salary payments over other social sector expenditures, led to volatile transfers to line ministries and decentralized administration, due to fluctuations in revenue. This in turn gave rise to information asymmetry, given the difficulty for local service providers to know the amount of their monthly resource entitlement (REPOA/ESRF, 2001). A further information problem was observed in Tanzania with respect to rural road expenditures. Instead of being deposited in a specific road fund account at the district (or urban) council level, funds disbursed for road maintenance by the Ministry of Regional Administration and Local Government are deposited in a miscellaneous account. This means that it is almost impossible to trace the funds, because the system of accountability and reporting used is different from that of other funds. Furthermore, the fund is not included in the annual budget estimates and is not disbursed using the same procedure and timing as other grants. As observed by REPOA/ESRF (2001), this system of aggregating expenditure items reduces transparency and increases capture at the local government council. In Madagascar, other significant problems of transparency were observed in budget allocation and basic accounting rules at the decentralized administrative level. The report emphasized that these problems could occur deliberately or due to lack of training. Regardless of the reason, the dysfunctional accounting system increases the incentives for leakage at the district and school levels. In over 50% of the districts and a quarter of schools, it was noted that the accounting system was incomplete or unclear, and 17% of districts could not show any evidence of materials distribution. Furthermore, lack of communication was also observed between central, district and school levels, and information transmitted was often incorrect.20 20 For instance, when the government abolished tuition fees in 2003, 45% of school representative reported in the survey that parents thought that they would receive money if their children attended schools. Evidently, the new policy stated officially that funding was allocated to the school, not to parents! (Francken, 2003). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 41 In Chad, it was reported that the total lack of transparency of budgetary information at the regional and district levels greatly facilitates the capture of the MOH budget at the central level. The regional delegates and heads of the health district lack information on the resources the MOH earmarked for them. The availability of information on the regional allocation of the “centralized purchases� would allow the regional delegates to ensure that the resources they receive indeed correspond to their initial allocation. In the same vein, it was suggested that the availability of budgetary information to the district and health facility heads would help ensure that adequate medical resources reach the population (Gauthier and Wane, 2005). In several African countries, tracking surveys have been able to identify numerous information problems, associated in particular with accounting, recording and reporting systems, supervision and monitoring, etc. Tracking surveys have also been able to indicate areas where reforms in information gathering and supervision systems should be introduced to improve resource allocation. The Uganda case in particular illustrates the potential positive impact of such reforms following a PETS, and the potential benefits of collection and dissemination of data on public services as a tool to mobilize citizens’ “voice.� Indeed, the objective of the information campaign put forward in Uganda was to promote transparency and increase public sector accountability by giving citizens access to information required to understand the working of education grants. Coupled with signals that the government was ready to listen to their voice, the result was an almost complete turnaround, as flows of funds improved greatly following the reform (Reinikka and Svensson, 2004b). 4.3 Impact of Public Resources Associated with the problem of information asymmetry and its symptoms, resource leakage and corruption, is another fundamental contribution of tracking surveys: the demonstration of the significant contribution of public resources on outcomes in social sectors. Contrary to the pessimistic view in the literature concerning the usefulness of public expenditures on population outcomes in health and education, PETS have shown that the potential problem at the origin of the non-significance of public expenditures in most regression analyses was the divergence between official and effective allocations. Indeed, a growing body of research has demonstrated that it cannot be taken for granted that allocating more budgetary resources to social sectors will necessarily deliver better outcomes (Musgrove, 1996, Filmer and Pritchett, 1999; Filmer et al, PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 42 2000). Filmer et al (2000), for instance, did not find that health spending has a significant impact on health status in most of the studies they reviewed.21 However, as noted by Ablo and Reinikka (1998) and Ratzan et al (2003), health and other social spending may have little impact on population status because expenditures may not translate into improved services. Indeed, official public resources may not adequately measure the availability or effectiveness of services in a context where mismanagement and corruption could be principal issues. The 1996 Uganda PETS, which tested the hypothesis that weak impact of public education expenditures on outcomes was due to the non-arrival of resources, showed that this hypothesis could not be rejected. The following tracking survey results have supported the hypothesis of Ablo and Reinikka (1998). In Chad, for instance, the effect of public health expenditures on service delivery was tested by Gauthier and Wane (2009) using regression analysis. At first glance, public resources allocated to regional delegations in the central budget in Chad appeared to have a negative impact on health center output. As shown in Figure 6A, regions that were officially allocated the highest per-capita health expenditures present the lowest ratio of patients having received health services in the region. At first glance, this result would appear to support empirical observations of the weak correlation between official health expenditures and health indicators in several countries (Filmer et al, 2000). However, Gauthier and Wane (2009) showed that this negative conclusion does not hold once leakage of health expenditures is taken into account, and that the reverse is actually true: public health expenditures tend to have a positive and significant impact on health services (Figure 6B). The main difference between Figures 6(A) and (B) in terms of the relationship between public resources and health services is that in the second figure only effective public expenditures (that is, those that reach the regions) create the positive health impact. Public expenditures could therefore contribute to the improvement of the populations’ health, provided they reach the population. The authors further examined the relationship between official and effective health resources and health services through multiple regression analysis. Using the total number of consultations in the health centers of a region as the dependent variable, they showed that official health expenditures do not explain health output at the regional level, as the coefficient of official expenditures is not significant in their regressions.22 21 Filmer et al 2000 (p. 204) noted that: “The cross-national evidence has always been absent or ambivalent on whether health status is improved by greater commitment to or greater spending on primary health care (or both).� 22 Gauthier and Wane (2009) estimated the following equation: Cl  �l  � 2 �l  � l where Cl is the number of patient consultations taking place annually in a region or a district l, Yl is a vector PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 43 FIGURE 6: RELATIONSHIP BETWEEN OFFICIAL AND EFFECTIVE ALLOCATION OF PUBLIC RESOURCES ON HEALTH SERVICES IN CHAD Initial/Effective Allocations and Performance Mayo-Kebbi Mayo-Kebbi 500 500 Moyen Chari Moyen Chari Logone Occidental Logone Occidental Production par 1000 Hbts Production par 1000 Hbts 400 400 Logone Oriental Logone Oriental Kanem Kanem BET BET guera guera 300 300 biltine biltine Tandjile Tandjile Salamat Salamat Chari-Baguirmi Ouaddai Chari-Baguirmi Ouaddai batha batha Lac Lac 200 200 500 1000 1500 2000 100 200 300 400 500 600 Initial per capita Allocation in FCFA Effective per capita Allocation in FCFA Source: Gauthier and Wane (2009) However, when effective health expenditure is introduced as an explanatory variable, its coefficient is always positive and significant. Real public expenditures thus have a positive impact on health output at the regional level. Indeed, they estimated that for a million CFAF (US$2062) of effective public expenditures received in a region in Chad, 693 more patients would receive medical consultations in primary health centers in the region annually. Furthermore, given the estimates23 of leakage and the elasticity of consultation service to health expenditure allocation, they estimated that if all public resources officially budgeted for regional delegations had reached the frontline providers in 2003, the number of patients seeking primary health care in Chad would have more than doubled during the year.24 of region or district characteristics, and εl is an iid error term. In particular, Yl includes either official or effective health expenditures in the region or district, the number of health centers and districts, the population served, the total revenues of health centers and salaries. 23 Gauthier and Wane (2009) estimated that only 1% of the non-recurrent budget allocated to regional health services actually reaches local service providers. 24 The authors estimated that approximately 4 billion CFAF officially budgeted for regional delegations do not reach the regional level. Using the estimated coefficient of effective health expenditures, they estimate that close to 3 million patients do not visit health centers because public resources do not reach service providers. Given that primary health centers in Chad have PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 44 Tracking surveys have also been able to shed light on a possible mechanism by which actual receipt of public resources could allow better access to health services: through the reduction of drug prices. Several empirical studies have shown that drug prices, which typically represent the dominant component of user fees in public health clinics in Sub-Saharan Africa, constitute an important barrier to health service access. The population often does not go to health centers because they believe their resources will be insufficient to cover the total cost of the medical visit and prescribed drugs. In Chad, for instance, the 2004 PETS- QSDS showed that medication costs account for roughly 75% of total medical costs for patients, and as much as 85% in rural areas. The Chad report specifically estimated the effect of public resource leakage on the mark-up charged by primary health centers. Regression results revealed a negative and significant relationship between average medication mark-up in public clinics and effective transfers of public resources to health centers in Chad. Indeed, Gauthier and Wane (2009) showed that local health facilities that receive government transfers were able to charge lower mark-ups on medications than centers that do not receive transfers. Leakage of government resources thus appears to have a significant and negative effect on user fees and thus constitutes a barrier to health service access in that country.25 The impact of increased public resources on population outcome is complex. In Zambia, the education survey examined the inter-connection between public and private decisions and noted that private response to increased public funding in education had to be accounted for. Zambian households devote significant quantities of private spending to their children’s education. Das et al (2004) observed that households responded to increased public spending on education by cutting back their own spending during the period under study. This eviction effect tended to offset the new public funds so that total expenditure on education did not increase, which did not lead to changes in educational test scores. In that specific case, although the rule-based education funds were channeled efficiently to the poor, there was no evidence that funds affected learning outcomes (Das et al, 2004a). 4.4 Absenteeism and Ghost Workers Another question that has been studied in PETS and QSDS, for which interesting results were obtained, is the problem of absenteeism among service facility workers. Table 3 presents some of the findings on absenteeism. treated about 2.5 million patients in 2003, this figure illustrates the impact of public resource leakage on health services in Chad. 25 They also observe that health centers located in rural areas tend to charge significantly lower mark-up than centers located in urban areas and that greater competition among health centers tends to significantly reduce average mark-up on drugs and thus be favorable to users. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 45 TABLE 3 ABSENTEEISM AND GHOST WORKERS Survey Country Year Type Sector Absenteeism Ghost workers Observation Uganda 1996 PETS Education Education: 27% Education: 20% Health Health: 37% 2000 QSDS Health 3.1% overall -109 staff members out - Absenteeism figures 4.4% in public facilities of 465 in district records should be interpreted (23%) do not appear in with care as facility facility records. But staff had prior warning difficult to establish if of visit. Also, head of these are ghost workers facilities may be or due to poorly updated covering for absent district records. staff members Tanzania 1999 PETS Education N.A. -Payrolls suffer from and Health ghost workers 2001 PETS Education N.A. N.A. and Health Ghana 2000 PETS Education N.A. N.A. and Health Rwanda 2000 PETS Health N.A N.A. 2004 PETS Education N.A. -548 teachers (1.8%) -Ghost figures should appear on the MOE list be interpreted with care but not on the facility as teachers transfer lists lists are poorly updated Zambia 2001 PETS Education 17% N.A. QDSD 2007 PETS Health Estimated at 21 % N.A. -self-reported data QDSD - Tardiness reported at 43 % Mozambique 2002 PETS Health Estimated at 19% QSDS Nigeria 2002 PETS Health N.A. N.A. QSDS Senegal 2002 PETS Health N.A. N.A. Cameroon 2003 PETS Health 5.6% overall N.A. 2004 PETS Education 9.4% overall N.A. Madagascar 2003 PETS Education Health: 10.9% overall -Higher rates for high- QSDS and Health skilled medical staff (14.2% for doctors). -Most of absences (more than 50% in rural areas and 32% in urban ones) are approved. 2006- PETS Education Education: 13% overall N.A. Education: Collection 2007 and Health of salary (t 13%), official mission (16%), unauthorized leave (25%) and sick leave, authorized absence or suspension (46%) are the main reasons. . PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 46 Namibia 2003 PETS Education N.A. N.A. QSDS and Health Chad 2004 PETS -Overall, estimated at N.A. -Higher absenteeism in QSDS 10% urban areas could be -Among doctors and explained by grater qualified personnel, alternative absenteeism is about opportunities for the 17% in public clinics medical personnel. and 33% in non-profit clinics - Absenteeism is greater in urban areas Kenya 2004 PETS Education -Estimated at 8.1% N.A. and Health among teachers Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006); PETS 1996-2004 tables. Note: (*) Reports not available. N.A: Not Available. Results in Table 3 support findings of other studies on provider absenteeism (e.g. Rogers et al, 2004; Chaudhury et al, 2006). One of the main conclusions of these studies is that because of poor accountability relationships and weak incentives, service provider absenteeism is prevalent in developing countries, which translates into low quality of services.26 Das et al (2005) in particular have explored the relationship between teacher absenteeism and students’ learning. Absenteeism in Zambia was shown to be associated with shocks, caused for example by illness or death, rather than overall lack of motivation. The authors showed that teachers work harder to compensate for such absences but that children with a frequently absent teacher may fail to improve in their test scores. The findings suggest that programs to allocate substitute teachers could significantly improve education outcomes in such an uncertain environment. Some caution is required in the interpretation of the absenteeism figures in Table 3. Tracking surveys typically report single visits to estimate absenteeism levels. Furthermore, most visits were not random or unannounced. In Chad, the 10% figure reported is certainly low compared with the true problem, as visits by survey enumerators were announced beforehand and organized with district authorities in order to ensure that data could be collected on the premises during team visits. In most countries, absenteeism figures are thus certainly underestimated.27 26 Chaudhury et al (2006) examined provider absenteeism in six countries, including one in Africa (Uganda). The study is based on verification of absenteeism through random unannounced visits. Results show absenteeism rates of between 27 and 40 percent for health care providers and between 11 and 27 percent for teachers. Absenteeism rates were associated with poverty and community characteristics related to effectiveness in monitoring. 27 There is clearly a trade-off in terms of survey strategy objectives when choosing which type of data to collect. In most past surveys, the main objective was to collect multi-purpose data, not simply to obtain less biased information on absenteeism from unannounced visits. The main PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 47 With these caveats in mind, another potential problem in interpreting the information collected is that there could be various reasons for absent personnel the day of the visit, some legitimate (sickness, etc.), which could be incorrectly registered. A further staffing question on which tracking surveys have shed some light is the phenomenon of “ghost� workers. In Uganda for instance, it was estimated that “ghost workers,� that is the number of teachers on official payroll who do not really work in facilities, represented about 20% of the workforce.28 In Mozambique, Lindelow et al (2004) noted important discrepancies between provincial, district and facility level staff records but could not confirm the presence of ghost workers. Data problems were noted, but a further problem arose from some health personnel’s being hired locally through community funds or user fees, which complicated the comparison between the central payroll list and facility personnel. Nonetheless, a number of studies have provided estimates on ghost workers, as reported in Table 3. Again, as for absenteeism, some caution is required in interpreting “ghost� worker data. Several reasons, including speed of updating information, could explain differences between numbers of staff on the central or local government official list and staff found at the facility level. Indeed, staff could have been assigned to a different facility or other district, which the official list does not yet take into account (Lindelow et al, 2006). 4.5 Impact of Decentralization Most countries have put forward an agenda of decentralization, which may include fiscal and/or administrative responsibilities. Donors have largely supported decentralization, which was perceived as encouraging more bottom-up expression of preferences and potential improvement in procurement decisions. A few tracking surveys have been used to examine the impact of decentralization on social sectors’ resource allocation. In Uganda, for instance, this consideration was incorporated in the sample selection process and was studied through the flow of capitation (per student) grants to schools. Findings in this country tend to indicate that decentralization has not produced positive results in terms of resource allocation to service providers. Following decentralization, which was implemented slowly starting in 1993, district authorities and district and urban councils gradually gained control of the funds provided by the central government objective was thus to reduce the probability of being unable to collect the overall survey data the day of the announced visit, but, in doing so, absenteeism levels would certainly be underestimated. 28 As emphasized in Section 4.2.1, most public administrations in Sub-Saharan Africa use a centralized staff payroll system. Generally, teachers and health workers are paid directly by the central ministry. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 48 to primary education. In 1996, it was estimated that the MOE controlled only about ¼ of the total recurrent spending on primary education. Using capitation grant as a proxy to explore the impact of decentralization on the flow of public funds to schools, Ablo and Reinikka (1998, p. 22) note that decentralization appeared to have led to a slight deterioration in the flow of funds to schools. Local governments that possessed decentralized responsibilities for longer periods of time presented greater fund capture (compared with more recently decentralized local government), and lesser transfers to schools. Das et al (2004a) also incorporated the question of decentralization in the sample design in Zambia, in a survey of two centralized and two decentralized provinces.29 They present similar results to Uganda with respect to the negative effect on funding flow to service providers. While the survey indicates that decentralization improved the flow of funds by decreasing spending at the provincial level, it somewhat reduced the allocation of funds to schools. Indeed, decentralized provinces presented greater levels of fund capture than centralized provinces. Overall, while only between 15% and 33% of total funding in the system (rule-based and discretionary funding) reaches schools, the record is somewhat worse in decentralized provinces. Schools in centralized provinces receive around 30% of total funds in the system compared with about 25% for schools in decentralized provinces.30 Khemani (2006) has examined decentralized delivery of primary health care in Nigeria. The study is based on the PETS/QSDS carried out in 2002 in two states (Lagos and Kogi) covering 30 local governments, 252 primary health care facilities and over 700 health care providers. The two states differ on a number of points that influence accountability relationships, in particular level of urbanization, availability of alternatives (such as private providers) and effectiveness in monitoring frontline providers. The study found a high degree of leakages in the more rural Kogi state. Furthermore, the author found evidence of a general problem of accountability at the local level in the use of public resources. Although the study cannot address the question whether decentralization has a beneficial effect on allocation and use of public resources — because the two states examined did not differ significantly in the extent of decentralization of responsibility — the overall policy lesson is that strengthening local government accountability is fundamental to reduce public resources capture. Other tracking surveys have examined decentralization issues, in particular in Madagascar where the school district administrations (Cisco) are responsible for 29 Decentralized provinces were defined as those where a district education board exists and is responsible for school funding. 30 Note that the tracking exercise in Zambia assessed the amount of resources available in the education system within the 4 provinces surveyed, distinguishing between rule-based and discretionary funds, and the originating level of the resources, as well as funds available at the district level. Financial resources available at the school level were tracked per pupil. The researchers analyze enrolment figures collected at the district level. Two districts were dropped from the calculation because of incomplete financial data. (Das et al, 2004a, p.36). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 49 the allocation of the MOE budget to schools, and in Tanzania, where there is a long history of decentralization in the social sectors. 4.6 User fees Another topic for which tracking surveys have produced interesting findings relates to user fees. There has been much debate about the equity and efficiency effects of user fees on the demand and supply sides. For service providers, user fees are sometimes the only source of revenues; in several countries, as previously mentioned, they receive only in-kind items from upper administrative levels. On the demand side, there are clear problems of accessibility and equity for the poor associated with user charges. Tracking surveys have shown that user fees often constitute a very important part of service costs for users and could have considerable negative effects on accessibility. In the Chadian health sector, for instance, contrary to the conclusions of previous studies (Ministère de la Santé publique, 2001; World Bank, 2004), the tracking survey discovered that the most important source of health center financing is user fees. As most public health expenditures in Chad are consumed by the central and regional administrations that do not provide direct services to the population, very little is left for service delivery. Once labor resources are allocated, health centers are left to their own devices to finance their activities through user fees. Indeed, government transfers account for only 2% of health center revenues (excluding salaries) and for one quarter of their revenues, including salaries. The impact in terms of access is significant, since the health centers tend to charge higher user fees to make up for their lack of resources (Gauthier and Wane, 2005). In the education sector in Uganda, although the government was trying to re- establish free education, the tracking survey found that private contributions represented more than 60% of education costs at the school level. Similarly in Zambia, despite an official policy of abolishing Parent Teacher Association (PTA) fees in order to increase enrolment,31 the survey estimated that private education expenditures were prominent in education costs. Das et al (2004a) estimated that education fee and other private non-fee expenditures (textbook, uniforms, etc) represent on average between 54% and 67% of total education costs for rich and poor households respectively.32 As in Uganda and Zambia, despite an official free education system, user fees in Rwanda reportedly reduce enrolment in primary schools. The MOE introduced a policy of “education for all� to improve access to basic education. Under this program, the Ministry pays education fees for all students. Basic education thus became free and compulsory. However, the report notes that the disbursement of 31 PTA fees were abolished for primary and basic schools in April 2002, 3 months before the fielding of the survey (Das et al, 2004a, p. 51). 32 Das et al (2004, p.54) estimated that expenditures for non-fee items were 3 to 9 times higher at the household level than fee expenditures across provinces. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 50 public funds to schools is irregular and insufficient to cover school expenditures. Schools then ask parents to make additional payments, which far exceed public funding.33 School authorities reportedly sent home and excluded children that are unable to pay school fees (Rwanda, 2004). In Kenya, user fees were introduced in 1989 as a way to mobilize resources to finance health care. They represent important discretionary funding for health centers.34 However, user fees purportedly create a barrier to services, especially to the poor and most vulnerable members of the population. Furthermore, important discrepancies and capture are reported in the use of user fee revenues. Finally, the Mozambique QSDS presented other interesting findings related to user fees, namely inconsistencies in the application of official user fee policies by health facilities 4.7 Delays PETS and QSDS have also shed light on the question of delays and bottlenecks in the allocation of resources through public administrations (e.g. salaries, allowances, financing, material, equipment, drugs and vaccines). These issues could have important effects on the quality of services, staff morale and the capacity of providers to deliver services. Table 4 presents estimates on delays in various countries, for certain types of items and inputs. The measurement of delays proved sometimes easier to estimate than quantitative or financial data on the same flows, which would have allowed measurement of leakage levels. In Zambia, for instance (as in most countries), staff compensation flows were not tracked. Instead, information was collected on delays (and arrival time) in the reception of salaries and four types of allowances (compensation and time overdue) at the service provider level.35 For the salary component, the payment system is efficient; over 95% of staff was paid on time and less than 3% reported more than 6 months’ overdue pay. However, allowance disbursements have worse records, depending on the type of allowance. Well defined allowances (hardship and responsibilities) are paid on time, while delays are observed in disbursement of the other types of allowances. In particular, more than 75% of recipients of 33 In the capital, these school fees ranged from 400-2000 Rwf per student per term, while in the province, they ranged between 5 and 500 Rwf (Rwanda, 2004) 34 User fee revenues are reported to generate close to 1 billion Kshs annually (Republic of Kenya, 2004). 35 The four types of allowances studied were i) Double-class allowance, paid to teachers that teach more than their contract stipulation, ii) Hardship allowances, paid as an incentive to teach in rural areas iii) Student-trainee allowance, paid to teachers in their second year of training that are assigned to rural areas iv) other allowances, which include compensation for additional responsibilities at the school (Das et al, 2004a). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 51 “double-class allowances� (additional amount paid for overtime, etc) experience at least 6 month’s overdue pay, possibly because of delays. According to the report, this appears to be partly due to lags in payroll updating. In Rwanda, it was observed that delays were observed in the payment of capitation grant funds to schools. Furthermore, about 13% of teachers do not receive their salaries regularly. More importantly, 82% of teachers report salary arrears in 2003. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 52 TABLE 4 DELAYS Survey Country Year Type Sector Delays Observation Cause Uganda 1996 PETS Education -Anecdotal evidence Health that teacher’s salaries suffer from delays. -However, survey indicates that salary payments reach schools relatively well. 2000 QSDS Health -72% of staff faces -In 20% of government salary delays in public facilities, salary delays facilities (compared to are reported more than 28% in for profit 16 weeks. facilities). -40% of government facilities report stock outs of supplies during the FY Tanzania 1999 PETS Education - Frontline workers and Health suffer delays in pays 2001 PETS Education -Delays in -Delays are reported -Linked to cash and Health disbursement and worse for non-wage budgeting system and significant delays in the expenditures versus the fact that salaries processing of non-wage salaries and in rural are prioritized in the funds, ranging from 6 to areas budget. 42 days at the treasury, -In some district, -This increases while wage transfers were not volatility in transfers disbursements are made by councils to and increase rarely delayed. some sectors for the asymmetry of -Delays are also period 1999-2000 information observed in all districts surveyed. Ghana 2000 PETS Education N.A. N.A. and Health Rwanda 2000 PETS Education -Evidence of delays in Health: -Delays were largely and Health budget execution at the -Very low execution attributed to the central level rate as 80% of non- application of the cash -Considerable delays in wage expenditures are budgeting system in transfers between released at the end of the MOF and cash regions and districts the year constraints of the government 2004 PETS Education -In particular, delays - Only 47% of teachers -Salaries are directly were observed in the knew the amount of transferred to payment of capitation salary arrears. teachers’ bank grant to schools accounts. They don’t -13% of teachers do not receive detailed pay receive their salaries slip. They lack regularly. information about their exact salary and -82% of teachers have deductions at the salary arrears (2003) source. -Irregularities in the payment of the Education Support Fund program reported by 43% of students surveyed PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 53 Survey Country Year Type Sector Delays Observation Cause Sierra Leone 2000 PETS Agriculture, Education: Most Education: Only Bonthe education, schools in districts and Moyamba have low 2001 covered in the survey percentages of health, security reported high teachers who received sector, social percentages of their salaries on time. welfare, rural teachers receiving development salaries on a regular and local basis. development, water and sanitation. Zambia 2001 PETS Education - Salaries: About 5% of - Well defined - Delays in the case of QDSD teachers incur delays allowances (hardship double class -Hardship allowance: and responsibilities) allowances and for almost all the tend to be paid on time student trainees provinces, about 20% -However, less well appear to be due in of teachers incur delays defined allowances part to lag in payroll suffer important delays. updating -“double-class allowances (additional amount paid for overtime, etc): More than 75% of recipients of “experience at least 6 month overdue 2007 PETS Health Delays in the receipt of In contrast to previous Only two-thirds of QDSD salaries are observed years, the timing of health facilities have (22 percent of staff). MOH releases on accurate registers. district grants is predictable. However, more than a third of the DHMTs themselves admitted delays in releasing district grants to facilities, although this is difficult to understand given that most of them received these resources from MOH on time. Out of the 20 DHMTs queried, seven (or 35 percent) reported delays in releasing funds to the health centers and district hospitals under them, 50 percent did not, and 15 percent provided no categorical answer. Mozambique 2002 PETS Health - Delays and QSDS bottlenecks in budget execution and supply management Nigeria 2002 PETS Health - Extensive non- - Regression results - Non payment of QSDS payment of salaries show significant salary was related to observed in one of the differences between problems of two states surveyed, Kogi and Lagos in accountability at the Kogi terms of the extent of local government level - Overall, 42% of staff non-payment of experience salary salaries. delays, reporting not -Non payment of receiving salary for 6 salaries had impact on PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 54 Survey Country Year Type Sector Delays Observation Cause months or more in the provision of services : past year at the time of the greater the extent of the survey. non payment of salaries, the higher the likelihood that facility staff behaved as private providers Senegal 2002 PETS Health - Evidence of delays in - Asymmetry of the decentralization information between fund (non wage) the local government and service providers on funds allocated Cameroon 2003 PETS Health -Delays in the -Fund managers have notification of about 9 months to budgetary envelopes execute their budget to decentralized units 2004 PETS Education -Delays in the notification of budgetary envelopes to decentralized units are noticed. Madagascar 2003 PETS Education -Significant delays to - District is slow in QSDS get IPPTE and budget execution CRESED at the school level 2006- PETS Education Education: Health: Health centers Education: 2007 and Health -Significant delays in where payments are -Frequent teaching the arrival of the “caisse irregular suffer more staff meetings and école�, school kits and from the leakage of inspections lower textbooks at school antibiotics. leakages of the caisse level école in the education Health: supply chain -Extensive delays in the distribution of drugs from the central purchasing unit to the facility-level pharmacies. The distribution takes on average one and a half months. -There are big delays in payment of dispensers’ salaries by communes: 60% of the health centers report irregular payments. Namibia 2003 Education Education: Education and Health - Delays in the supply of -Mismatch between books at the school MOE textbooks level catalogue and Health: available books - Delays in the release of funds at the central level Chad 2004 PETS - 14% of the health - Delays in salary - Poor infrastructure QSDS workers face delays in payment are slightly and absence of receiving salaries. higher 16 % vs 10% in decentralized financial institutions. Workers PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 55 Survey Country Year Type Sector Delays Observation Cause - In public clinics, salary urban areas often have to travel delays affect 20% of the long distances to personnel collect salaries 7 % of health personnel doe not receive their full salaries. Kenya 2004 PETS Health and -Delays in medical - Delays in medical education supplies delivery supplies may be -61% of health centers explained by top-down report stock outs of approach in the drugs during FY 2003- procurement process. 04 Mali 2006 PETS Health and Doctors paid by the QSDS education HIPC fund receive salaries with up to 2 or 3 month delays. Salaries for doctors paid by the City are often delayed by 12 to 20 months. Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006); PETS 1996-2004 tables. Note; (*) Reports not available; N.A.: Not available In Tanzania, the report described the disbursement procedure between the central level and facilities via district authorities. It measured the time of disbursement between these levels and delays for salaries and “other charges,� as well as for certain materials (books and drug kits). Salaries appeared to be less prone to diversion than non-wage expenditures, but delays in pay for frontline staff were also observed. The most important delays are reported for non-wage resources in rural areas.36 Significant delays in the allocation of non-wage funds at the central level were documented. In fact, delays were observed in all districts studied. In some districts, no transfers at all were reported in some sectors for the period under study. In Nigeria, Das Gupta et al (2003) and Khemani (2006) found evidence of long delays in the payment of salaries to health providers. In particular, extensive non- payment of salaries of public health personnel was observed in one of the two states surveyed, Kogi, where 42% of staff respondents reported not receiving a salary for 6 months or more in the past year at the time of the survey. Results of regression analysis indicated significant differences between Kogi and the other state surveyed (Lagos) in terms of the extent of non-payment of salaries. It was further shown that non-payment of salaries could not be explained by lack of resources available to local governments. Evidence suggested that non-payment of salary was related to problems of accountability at the local government level in the use of public resources (Khemani, 2006). Furthermore, it was observed that non-payment of salaries had an impact on provision of services as the greater the 36 As previously mentioned, the cause was linked to a cash budgeting system, which lead to volatile transfers of resources due to fluctuations in revenue. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 56 extent of non payment of salaries, the greater the likelihood that facility staff acted as private providers (more service provided outside the facility, expropriation of drugs from facility stocks, etc.). 4.8 Efficiency Also noteworthy are various analyses of productivity performed using tracking surveys. There is no consensus about which indicators should be used to measure performance of schools or health centers. Indeed, various problems with measures and comparability exist. In the health sector in particular, there is considerable heterogeneity in terms of service output due for instance to variations in case mix across facilities (types of patients, complexity of cases, etc.) and range of services offered. Nonetheless the most frequently used indicators that measure performance of health centers include the number of consultations by specialized staff and the total number of consultation per employee. These indicators do not control for the quality of services offered, an important factor in explaining efficiency, but one which is even harder to measure. Lindelow et al (2004) have examined the question of productivity of health centers using PETS/QSDS data in Mozambique. They analyze seven categories of service output and devise a composite index of output to deal with the problem of multi-output production. They observe significant urban-rural and regional differences in service output per capita. The authors also note important variations in output per health worker across districts (in an 8 to 1 ratio). They find that the low productivity observed in certain districts may be due to low density and consequently, that low productivity may be a cost of extending service to sparsely populated areas in the interest of equity. Lindelow et al (2003) have examined the question of efficiency in Ugandan health clinics using QSDS data. They make use of an output weighted index similar to the one used in Mozambique to measure health workers’ output for different categories of facilities. They note very important differences of outpatient- equivalent service units per workers across facilities in the country (in a 50 to 1 ratio). Important differences in output per worker are also observed across ownership categories, with lower levels observed among non-profit facilities. Gauthier and Wane (2005) have also examined differences in productivity among health facility ownership categories in Chad using PETS/QSDS data. Contrary to Uganda, they observe that religious not-for-profit facilities are the most efficient providers for a variety of performance indicators (including the average number of outpatients and the number of patients per staff member). A typical religious not-for-profit facility in Chad treats 2,300 more patients annually than the average Chadian provider of primary health care. Employees in religious not-for-profit facilities are also the most productive, offering health care to over 270 patients PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 57 more than the average health worker. This difference reached 970 patients annually when the sample is restricted to qualified workers. Finally, Over et al (2006) have examined efficiency of health service delivery in six countries, focusing especially on HIV prevention. They apply a technical efficiency approach based on stochastic production frontier to estimate the determinants of relative productivity. Their objective is to evaluate cost- effectiveness of service delivery, controlling in particular for quality of health care services in improved health output 4.9 Other Findings Among the other interesting findings of past tracking surveys in Africa, it is worth mentioning the question of equity in the allocation of resources and services, by location or between income groups (Table 5). In several countries, variability of health and school spending across geographical areas, regions and districts, as well as within districts, was observed. The considerable difference in resource allocation noted in several countries raises serious issues of equity among socio- economic and demographic groups. In Mozambique, for instance, a nine-fold variation of per capita health spending was observed at the district level ranging from 5000 to 47000 Meticais annually.37 These variations seemed to be mainly driven by staffing and infrastructure patterns across districts. In Chad, non-wage per capita public health spending showed important variations across districts (in a 16 to 1 ratio): at the health center level, the most funded district received 38 CFAF per capita on average versus 2.3 CFAF for the least funded district (Gauthier and Wane, 2005, p. 77). In the schooling system in Zambia, the most funded district received 8 times more average per-student public resources than the least funded district. This variance was due predominantly (90%) to differences within provinces, as opposed to difference across provinces (10%). At the school level, differences in Zambia were even much more pronounced; the most funded school received 3000 times more funding than the least funded one.38 Furthermore, equity considerations in Zambia are examined in details by Das et al (2004a). The authors used a household survey to develop an index of wealth, which measured the progressive nature of two types of school funding (rule-based 37 Excluding drugs and other materials provided to districts in-kind. Large differences in the composition of spending across districts were also noted. Staff expenditure’ share accounted for between 35% and 90% of total spending (excluding drugs and other in-kind resources) (Lindelow et al, 2004) 38 Again, most of the variation was found to be within districts (83%) rather than across districts (Das et al, 2004a). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 58 and discretionary).39 In addition, their equity analysis accounted for the value of staff inputs at the school level and private contributions to education expenditures. The study found that rule-based (per-school) funding had progressive characteristics that led to greater per pupil funding for poorer and rural schools.40 These findings, namely that poorer schools receive higher per-pupil rule-based funding, were obtained using regression analysis. However, staff expenditures per pupil were found to be regressive; they were higher in urban and richer schools.41 Discretionary funding was found to be regressive in rural areas (whereas it was wealth neutral in urban areas). Overall, (accounting for all these sources of public funds) public school funding in Zambia was shown to be regressive, with richer schools receiving almost 30% more public resources per pupil. Furthermore, Das et al (2004a) also showed that household spending further contributes to inequalities in education. Das (2004) further examines the question of the effect of government education expenditure on equity. His model specifically accounts for the difference in the substitution effect between public and private education expenditures between rich and poor. He shows that the success of government education expenditures at reducing educational inequalities depends on the types of expenditures and institutional arrangements used. While cash grants were successful at targeting the poor, they were less successful at modifying overall educational expenditures given the greater crowding out effect on poorer households. Consequently, public transfers did not reduce inequality in overall educational expenditures. Gauthier and Wane (2005) examined the question of equity in the Chad. They focus on access to public health resources at the regional level and user costs among quintiles of revenues. They show educe stark inequities in the Chadian health system. The most affluent individuals have better access, both geographical and financial, to health care and receive a higher quality of care. About 44% of individuals in the wealthiest quintile have another health care provider close to their home, compares with 13% for the poorest quintile. For the most affluent patients, out-of-pocket medical expenses account for 2.1% of their monthly household incomes versus 21% for the poorest, which thus bear a burden that is ten times heavier. Reinikka and Svensson (2003a) have examined how the difference of objective of religious not-for-profit (RNFP) health care providers affect their pricing and service output behavior compared with other categories of health providers in Uganda. They devise a model of service provision accounting for various input use and quality of care. They compare outcomes of RNFP with those expected of private for-profit providers and those of public health centers to test an altruistic motivation hypothesis. They make use of an exogenous change in the financial 39 Details about the construction of the index and its properties are presented in the Appendix of Das et al 2004a, pp. 62-64 40 This was mainly due to the fact that rule-based funding was a fixed amount per school, irrespective of enrolment, and enrolment was lower in poorer rural schools. 41 This was associated with the fact that poor and rural schools tend to have a larger share of lower paid teacher trainees on the payroll. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 59 incentives given to RNFP (an untied government grant allocated to not-for profit primary health care units which did not affect all the sample clinics) as a natural experiment to identify the objectives of religious providers. They show that RNFP pay qualified staff below market wage, that they provide more pro-poor services and services with public good elements and charge lower user-fees. At the same time, they are able care of comparable quality to private for-profit health providers. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 60 TABLE 5 OTHER FINDINGS Survey Country Year Type Sectors Other findings 1 Other findings 2 Other findings 3 Uganda 1996 PETS Education Education: Education: -Two seemingly and Health -Instead of being -Confirmed that public comparable social stagnant, the survey primary education sectors demonstrate showed that primary was mainly funded by quite different enrolment had parents: On average institutional behavior increased by 60% contributed 73% of in record keeping at between 1991 and total school spending frontline level. 1995. Performance in 1991, 60% in 1995. of the education -Parental contribution sector had improved continued to increase much more than the despite higher public official information spending during system reported. 1991-95 Health: -Confirmed that health facilities did not keep systematic financial or patient records in 1991-95 2000 QSDS Health -Service delivery -Considerable user characteristics, in fee differences particular mix of charged by services, quality, government facilities pricing, use of drugs across regions and cost-efficiency, differ among ownership categories 2009 PETS Water -Poor record keeping -The budgeting and -Inadequate - Lack of proper planning process for coordination between physical and financial Water and Sanitation departments at accountability is not consultative district level especially at the enough to capture the District level local district priorities. Tanzania 2001 PETS Education Education Education Roads and Health -Distribution of -Important shortage -Funds disbursed for (roads, rural education material is of exercise books and road maintenance are water, etc) unequal between textbook for students deposited in a councils and between miscellaneous schools in districts account, which reduces transparency and increases capture at the local government council. 2004 PETS Education The disbursement of There are huge There is a complex capitation grant from variations in the system of the central level to the inflow of capitation disbursement from districts has been in grant at the school the central level to the line with the initial level. In the sample, it councils, involving plans for this varies from 1,600 Tsh three ministries program. to 8,700 Tsh per (MOEC, MOF, PO- student per year. RALG) and the regions. Ghana 2000 PETS Education Health and Health - User fees are much higher than they should be because of PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 61 Survey Country Year Type Sectors Other findings 1 Other findings 2 Other findings 3 leakage Rwanda 2000 PETS Education Education: -Sloppy bookkeeping -Lack of monitoring and Health Deficiencies in and poor financial and supervision from education facilities, management at the central authorities including inadequate regional, district and contribute to these numbers of facility levels. poor practices. classrooms in many -No written guidelines districts leading to on how funds should overcrowding and be managed at the denial of access. regional, district or Health: facility level, and no formal requirement - User fees are shown for accountability on to be the main source the utilization of of revenues at the funds. facility level -The majority of population does not use health centers when ill due to high costs and poor quality of medical infrastructure and services. 2004 PETS Education -The commercial -Despite an official bank processing “free education� teachers’ salaries policy, user fees are (UBPR) has not imposed and are returned unpaid much larger than salaries to the capitation grants Treasury since 2001. This represents about ¼ of annual teachers salary. Sierra Leone 2000 PETS Agriculture, Health: Education: 2001 education, About 72 percent of The survey results health, the health facilities show that 78.8 security reported that they did percent of schools did sector, social not generate funds not charge fees with welfare, rural internally during the 15.1 percent admitted development period. About 20 charging fees as the and local percent of the school fees subsidies development, facilities confirmed paid through water and receipt of resources education secretaries sanitation. in kind from the were not transferred District Health Office to them. and 21 percent from Non-Governmental Sources. Zambia 2001 PETS Education -Public funding is -Decentralized QDSD regressive: almost provinces tend to shift 30% higher allocation administrative to richer schools. spending from -Private contribution provinces to districts to school revenues but also to reduce decrease when public resources reaching fund to the school schools. increase PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 62 Survey Country Year Type Sectors Other findings 1 Other findings 2 Other findings 3 Mozambique 2002 PETS Health - Inequity in -Incomplete reporting QSDS resources allocation of user fee revenues across districts and by facilities (67.6% for facilities consultation and - Lack of 79.6% for drugs) transparency about resource flows at various levels with potential efficiency and equity implications. Nigeria 2002 PETS Health -Evidence of a -Capture of resources QSDS general problem of varies by states. accountability at the local government level in the use of public resources. Senegal 2002 PETS Health -Evidence of discretion by local governments in the allocation of resources Madagascar 2003 PETS Education Education: Education: Education: QSDS and Health -Significantly higher -Lack of inspection -Important differences increase in enrolment and monitoring at the in salaries between is observed in more district and school regions for non civil remote (poorer) levels about the way servant teachers (in a schools, indicating funds are used. 1 to 3 ratio), reflecting that the reduction in Health: differences in the schooling costs was relative wealth of an effective pro-poor -From the perspective provinces. policy. of effective staff training, public urban Health: facilities are better - Stock outs of staffed. For all types essential drugs are of centers, there is an relatively frequent. average of 7 More than 6 out of 10 employees including facilities have 3 skilled (a doctor). experienced this problem during the three months preceding the survey. Chad 2004 PETS -Leakage of -The main source of -Important inequity in QSDS resources is health center revenue terms of distributions associated with is user fees. of public health higher drug mark-up Government transfers resources (44% of as regression results account for only 2% individuals in the show a negative and of health center wealthiest quintile significant revenues (excluding have another health relationship between salaries) and for one care provider close to average drug mark- quarter including their home; this up in public clinics salaries. proportion drops to and effective 13% for the poorest transfers of public quintile.) resources to health -For the most affluent centers. patients, the out-of- pocket average medical expenses account for 2.1% of their monthly incomes versus 21% for the poorest quintile. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 63 Survey Country Year Type Sectors Other findings 1 Other findings 2 Other findings 3 Kenya 2004 PETS Health and -Release of money Education Education education though Authority to -Affirmative action is -District school Incur Expenditures needed to target girls auditors are required (AIE) to districts is a and disadvantaged for verification of the bottleneck to child. constituency bursary expenditures at the operations and to district level given Health prepare annual that AIE are issued - Understaffing of national reports. without disbursement health facilities is a of funds at the district major constraint in level. the delivery of services. Mali 2005 PETS Education As regards materials Primary schools are Administrative QSDS other than textbooks completely left out for structures and high and students desks, these materials, school principals primary schools do which account for received cars, not receive much. 12% of the total MoE photocopiers, fax budget and more than machines, or other 20% of the non-wage kinds of material current budget in through these 2004. decisions. Niger 2008 PETS Health and Education Education Health education The inefficiencies The direct distribution Publicly distributed between MEBA of textbooks by essential medicines (Ministry of Basic MEBA to schools, seemed to reach Education) and bypassing regional more the richer rather schools are about 50 and district education than poorer regions. percent. offices, reduced Health inefficiencies to about 25 percent. Evidence suggests that food expenditure discrepancy between Health district health offices Medicine shortages and service providers were cited as a is about 40 percent. problem by 25 percent of clinics, but were rarely cited as such by district hospitals. Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006); PETS 1996-2004 tables Also noteworthy are the different results observed in Zambia and Uganda with respect to the potential substitution effect between private and public education expenditures. Das et al (2004b) formally examined the question of the effect of household response to schooling inputs on educational outcomes. They devised a dynamic model of household cognitive achievement in which households respond optimally to the provision of inputs at the school level. The impact of school inputs in this framework depends on the level of household anticipation about the input and the degree of substitutability between school and household inputs. They make use of school inputs provided by the 2002 PETS/QSDS in Zambia, PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 64 which included test scores on a sample of students one year apart. In addition, they matched the household data to a sub-sample of schools which allowed them to directly relate household and school inputs. They show that household educational expenditures and public cash grants to schools are substitutes and that the elasticity of substitution between the two is high and significant. They find that unanticipated grants would have a significant and substantial impact on learning achievement while anticipated grants would have a small, insignificant effect. The 2004 Rwanda PETS also observed that parents seem to reduce private education spending following the increase in public education expenditures. It was noted that following the introduction of an “Education for All� policy, a number of parents disengaged themselves from the education expenditures of their children. One result was that many students no longer wore uniforms at school (Rwanda, 2004, p. 44). However, it was also noted that the policy has had a positive effect on attendance, as a massive influx of children into primary schools was reported. Interestingly, in contrast, public and private expenditures in Uganda appear to be complementary: parents increased private education expenditures while public expenditures increased during the period studied by Ablo and Reinikka (1998). Further analysis of the factors explaining these differences in private responses across countries to public spending would be required to shed further light on this question. Finally, various other interesting findings include analyses of service quality in schools and health centers, availability of materials and personnel, gender in the workforce and its association with user preferences, the presence of informal payments and corruption at the staff level; staff incentive system (salaries, supervision, performance assessment, sanction or rewards) and staff morale. 4.10 Summary  PETS-QSDS have deepened our understanding of the environment of public service providers in Sub-Saharan Africa. They have helped collect information and document incentives and constraints faced by agents in the service provision process.  The surveys have identified leakage levels and the source of resource dissipation in the administrative process of service delivery. Certain patterns of leakage levels have emerged and are associated with types and categories of funds. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 65  Tracking surveys may have also allowed a better understanding of clients’ rationale and incentives on the demand side, as well as a better understanding of efficiency and equity considerations.  Results and findings vary significantly between surveys, in terms of their capacity to measure leakage, absenteeism, etc. and their ability to better understand specific institutional arrangements.  How can one explain these differences in success levels among surveys in their capacity to track resources and identify inefficiencies and equity problems? In order to answer this question, the next section examines certain elements associated with past surveys’ methodological approaches, idiosyncrasies and contextual situations. We will then try to identify factors that could explain these differences in performance, and from this analysis potentially identify best practice approaches that could guide future endeavors. Specific examples will be discussed to illustrate how surveys’ methodological choices and country specificities have influenced their performance and capacity to attain specific objectives. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 66 5. HARMONIZATION ISSUES: DESIGN AND IMPLEMENTATION DECISIONS Various methodological choices have to be made when designing and implementing tracking surveys. Each has consequences on the survey’s capacity to achieve its monitoring, analysis or evaluation purposes. This section reviews a certain number of key decisions that each survey faces and examines choices made by previous tracking surveys. We discuss potential advantages and limitations associated with some of these choices in order to identify lessons and recommendations to favor harmonization. However, this study does not provide definitive answers or explore these considerations in detail, as a common approach for future survey works would need to ensue from more in-depth analysis and discussions by working groups. We start by reviewing the choice of resources to track. We then examine more specifically sample selection issues, in-kind data collection, length of data tracking, survey timing and questionnaire design. 5.1 The Choice of Resources to Track Any tracking survey requires determination of the specific flows on which financial and quantitative information will be collected and at which administrative levels. Clearly, in each of the various branches or resource flows of the allocation procedure, there are possibilities of leakage: funding, supplies, drugs, equipment or materials could leak or be stolen through the procurement process at various levels in the service provision supply chain. Similarly, salary expenditures could leak through the creation of fictitious (ghost) workers, for instance. However, not all flows are amenable to tracking. Nonexistent records or accounts, data inconsistencies and other types of problems will make certain flows untraceable or the data too noisy to be informative. Furthermore, the complexity and challenges of tracking whole categories of expenditures have led PETS to restrict the tracking exercise to focus on a subset of the whole service provider environment.42 What should determine the choice of flows to track in the supply chain? Several factors are at play. In some cases, the choice set can be determined directly from the research question or survey objectives, which, in certain cases, could call for a specific flow to be tracked. For instance, if the objective of the survey is to identify the prevalence of ghost workers, then the domain of financial flows to track could potentially be restricted to salary flows. Similarly, if the purpose of 42 For instance, the 1996 Uganda education PETS tracked one specific program among non- wage recurrent expenditures: capitation (per student) grants. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 67 the survey is to evaluate the impact of a specific program in the education or health sector (e.g. HIV/AIDS), then the focus of the tracking exercise could naturally be limited to this specific program. Most previous tracking surveys, however, did not have such restricted (or predetermined) sets of flows to track from the outset, and favored more an open mode of trying to track most, if not all, categories of funds. In such cases, more pragmatic reasons for restricting flow tracking, such as data availability or quality, and resources or time constraints then became important considerations. Let’s first examine the choices made by two of the most successful tracking surveys so far, the 1996 Uganda PETS and the 2002 PETS-QSDS in Zambia, both in the education sector. 5.1.1 The Tracking Choices of Two Successful Surveys Uganda: The initial intentions of the PETS team in Uganda in 1996 were to track all public spending in education through the entire delivery system (Ablo and Reinikka, 1998). However, a pilot survey revealed important data availability problems. During the pilot, it was discovered that at both the central government and district levels, official records for both wage and non-wage expenditures were very poor, if not simply non-existent. Moreover, the quality of information at the district level, both on transfers from the MOE and disbursement to schools, was similarly very poor. The team then decided to completely exclude the district administration level from the tracking exercise and to limit data collection to the central government and service provider levels (Ablo and Reinikka, 1998, pp. 8- 9). Furthermore, a pilot survey revealed that at the central government level, expenditure data were not available on salaries paid to primary school teachers, either by districts or schools. The only such data available were aggregate salary payments, which lumped together payments to teachers in primary, secondary and tertiary levels as well as to non-teaching staff which would not have been of any help in a tracking exercise.43 The only systematic information on primary education that was found to be available and of good quality at the central government level was capitation grants44 for non-wage spending. Fortunately, at the school level, financial records were relatively comprehensive. 43 Another factor complicating efforts to track salary spending was that there were more teachers at the school level than on central government payrolls, as additional teachers were hired directly by schools and funded by parents’ associations (Ablo and Reinikka, 1998). 44 Capitation grant is a payment to school per student enrolled and is viewed as a matching (50%) government contribution against the mandated tuition fee, which had to be paid by parents. The MOE official policy was to disburse this per student grant in full to the school (either in cash or in kind), through the district education officer. The grant is intended to be used to partly pay for textbooks and other learning materials, as well as schools’ general running costs. The capitation grant was set in 1991 at the nominal rate of USh 2500 per child enrolled at levels P1-P4, and USh PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 68 It should be noted that regarding donor assistance for primary education, with the exception of one major donor-funded project, the absence of disaggregated data at the MOE level rendered tracking of donor and NGO education expenditures very difficult. Donor assistance was consequently excluded from the tracking exercise. The tracking exercise then ultimately focused on a single flow, capitation grant. The survey objective thus became to determine how much of the capitation grant allocated by the MOE actually reached primary schools. Data (financial and in- kind) were collected at the school level on the reception of this single program. To estimate leakage (between the entitled budgetary allocations and actual reception at the school level), the only other information required was enrolment data at the school level, which was also available. Lesson: A lesson to be learned from the first education PETS in Uganda is that the choice of resources to track has to be guided by the specific country conditions, particularly related to data availability and quality. Furthermore, the availability of records and accounts at least at two levels of the supply chain is required to measure some form of leakage. Data consistency among levels must also be assessed before choosing tracking flows. In Uganda, an effective data assessment and pilot survey revealed the important data problem in the administrative apparatus, particularly at the district level. Through this preliminary data assessment, the team was able to identify a type of flow amenable to tracking and subsequently implement effective data collection. Another potential lesson from the first PETS is the realization that the choice of the flow to track does not have to be a dominant share in the service provider budget to convey important results. Indeed, the flow tracked in Uganda, capitation grants, represented on average only about 6% of total government transfers to schools during the period under study (1991-95) (29% of non-wage government expenditures received by schools during the period).45 Fixed allocation rule: Another factor that drove the success of the Uganda PETS was the existence of a fixed allocation rule for the program tracked. Each school was entitled to a specific amount of cash (or in-kind) transfers based on the number of students enrolled. This fixed rule made it possible to measure leakage in a straightforward fashion, as the ratio of what a school received to what it was entitled to. This fixed allocation rule greatly simplified the leakage measure exercise. Lesson: A lesson could be for analysts to look for such fixed allocation rule programs that could simplify the tracking exercise. We will see, however, in the following cases that other strategies could be conducive to successful funding tracking, even without the fixed allocation rule. 4000 per child enrolled at levels P5-P7. These nominal rates remained the same until 1997 (Ablo and Reinikka, 1998, pp. 9-10). 45 Based on Ablo and Reinikka, 1998, Table 4, p. 10. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 69 The results of the Uganda 1996 PETS are well known. The survey successfully identified important leakage in the non-wage expenditure (capitation) program and effectively proposed reforms, including an information campaign, to reduce the information asymmetry at the source of the program’s weak performance. The follow-up PETS in 1999 and 2000 were also very successful. They were designed to evaluate the impact of the information campaign to reduce resource leakage identified in the first PETS. The surveys included questions about school access to various sources of information, in order to try to establish causality between the information campaign and the reduced leakage observed (Reinikka and Svensson, 2005). These surveys lead to the production of several in-depth and groundbreaking analyses on corruption and rent capture in public services. Several papers have come out of these surveys, including Reinikka (2001); Reinikka and Svensson (2003a,b,c, 2004a,b,c, 2005a,b). Zambia: Another clear success story is the Zambian PETS/QSDS in education in 2002. Through a very thorough preliminary study of the education sector and in- depth analysis of the administrative process, the team was able to grasp the nature and characteristics of resource flows in the public education system. Six main flows were identified and categorized in terms of sources, types and administrative levels’ discretionary power in fund allocation (Table 6).46 Once these flows were identified, the team chose to track non-wage cash flows from the MOE and donors at the provincial, district and facility levels. In-kind transfers, as well as salary transfers, were excluded from the tracking exercise. The specific objective of the tracking exercise then became to determine whether: a) schools received the fixed-rule component of the MOH budget (lump-sum payment per school); b) provinces and districts supported schools further through discretionary expenditures; and c) decentralization had an effect on fund allocation behavior (Das et al, 2004a, p.29). The rationale for excluding staff financial data (salary, allowances etc.) from the tracking exercise was that staff remuneration is disbursed directly from the central government payroll to individual teachers and hence does not pass through the hierarchical administrative system. Still, although it did not track salary flows, the survey analyzed staff compensation in terms of delays and overdue for various types of teacher compensation. Furthermore, financial data on staff compensation were collected at the facility level among a sample of teachers (from grades 5 and 6) in order to measure the value of staff input in each school, which was then used in an equity analysis.47 46 Funds were further classified into four categories, i) Rule-based allocation to schools consisting of a fixed-grant ($600 or $650 depending on the type of school) allocated to schools independent of enrolment; ii) Discretionary allocation to schools by provinces or districts; iiii) Rule-based allocation to teachers: Salaries and monthly allowances paid directly to teachers through a centralized payroll; iv) Discretionary allocation to teachers: Other teachers’ remunerations, such as transfer, leave or funeral benefits, pass through the province or district office, at their discretion. 47 These data were used to calculate the average salary of teachers interviewed in a school and to construct per pupil staff bills. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 70 TABLE 6 STRUCTURE OF RESOURCE AND FUNDING FLOWS IN ZAMBIA Does province Does province Does district Does district Ministry of have have have have What can school Education / Ministry discretion? discretion? discretion? discretion? teachers expect to Flows of Finance (centralized)a (decentralized) (centralized) (decentralized) receive? tracked Flows of Handles YES NO YES YES Depends on district Not tracked educational procurement and materials distributes to (discretionary) provinces/districts Payment of Completely NO NO NO NO Teachers receive Not tracked staff: salaries centralized and salaries and and made directly to allowances allowances teachers according to the (rule-based) salary scale Payment of Disburses funds to YES YES YES YES Payments for one- Not tracked staff: One-time provinces and time benefits such benefits districts for such as leave and (discretionary payments transfers payments) Cash flows from An allocation of NO NO NO NO $600 or $650 Tracked GRZ: fixed either $600 or grant (rule- $650 is made to based funding) every primary- basic school Cash flows from Allocations are YES NO YES YES Depends on Tracked GRZ: made for recurrent district: Schools (discretionary and capital may expect to funding) expenditures, as receive money for well as recurrent / disbursement to infrastructure districts/schools expenditures Cash flows from No discretion. Most YES NO YES YES Depends on district Tracked case IV money from case donors IV donors is for (discretionary PAGE (Program component) for Advancement of Girls’ Education) a. Northern and Eastern province in the PETS/QSDS. Source: From Das et al, 2004a The exclusion of in-kind materials and equipment from the tracking exercise was justified by the fact that the procurement of the school materials was not completed at the time of the survey, which was fielded in July in the middle of the school term (which started in January) (Das et al, 2004a, p.23). This could have introduced some biases in the data collected if provinces or district systematically provided more in-kind resources than cash transfers to certain types of schools or districts. However, as argued by Das et al (2004a, p.50) data on receipts of such materials at the school level tended to be small at the time of the survey, which reduced the potential risk of such problem. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 71 As mentioned, in order to examine differences related to the decentralization axis, schools were surveyed in two centralized and two decentralized provinces. Lesson: Having restricted the tracking exercise to cash flows has facilitated data collection in Zambia, as good quality financial records were available. Furthermore, the distinction introduced between rule-based and discretionary funding allowed the use of a measure of leakage comparable to the one developed by Ablo and Reinikka (1998) in the first Uganda PETS. Indeed, in the original (or “strict�) definition, as previously mentioned in section 4.1, leakage was defined with respect to a fixed rule. In addition, the Zambian survey examined the discretionary components of funds transferred to schools using an equity analysis. The very thorough field work of the survey team, in particular at the pre-survey and instrument design stages (which overall lasted four months for some key researchers) was certainly a key factor in allowing a clear understanding of the institutional ramification of the education sector and the crafting of a very sound survey methodology. The survey was successful in several respects. Notably, it estimated leakage on fixed-rule funds, measured the arrival of non-wage discretionary funding at school level and developed a per pupil equity analysis on total public and private school level funding. In addition to the in-depth survey report (Das et al, 2004a), several papers were produced using the survey data, including Das (2004), Das et al (2004b, 2005), Das and Hammer (2005). 5.1.2 Other Examples Not all tracking surveys, however, were as successful as the Ugandan 1996 education PETS and the Zambian 2002 PETS/QSDS. Uganda: As mentioned in section 2.1, the original Uganda 1996 PETS had the dual objectives of tracking public expenditure flows in education, along with primary health care. However, the second objective was not reached, as consistent quantitative data in the health sector could not be collected. Indeed, the survey found no systematic facility level information on financial information, inputs or outputs. At the central government level, for the period under study, data on staff salaries were not disaggregated either by district or health facilities (as in the education sector). Furthermore, with regard to non-wage public expenditures, resources were mainly allocated in the form of in-kind transfers to health facilities, which rendered tracking more challenging (Ablo and Reinikka, 1998). Contrary to the education sector, where the focus, after a pilot survey, was specifically restricted, as mentioned, to a particular program (capitation grants) and where the district administration level was completely bypassed due to inadequate information at that level, the focus of the tracking exercise in the health sector remained on all public resources and administrative levels. The very PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 72 disappointing data collection and survey results in the health sector are reported by Ablo and Reinikka (1998, p. 26): The lack of almost any financial information at the facility level and the heavy reliance on in-kind measures were not fully anticipated at design and pilot stage of the survey. As the data gathering proceeded, any hope of systematic tracking expenditure on the basis of data from primary health facilities slowly faded. While much of the resources received by health units were in kind, their values were not indicated and hence not easy to compute, except perhaps for drugs. Similarly, health units maintain inventories of equipment with no value or dates of receipts indicated. Ultimately, the health tracking survey did not produce any reliable quantitative measures of expenditure flows in the sector or leakage assessment. Nonetheless, a number of qualitative data and observations on input use, health workers and user fees, etc. were produced (Ablo and Reinikka, 1998). Lesson: Large scale surveys that attempt to track entire sector flows run the risk of not being able to collect consistent and high quality data. A thorough institutional analysis and data assessment should entail the restriction of tracking flows and the identification of flows amenable to produce quality data on at least two levels of the service delivery chain. Mozambique: The PETS/QSDS implemented in Mozambique in the health sector in 2002 also suffered from important data problems. As reported by Lindelow (2006), the complexity of the financing and logistical arrangements in the health system in the country became apparent to the survey team at a relatively early stage. As discussed in section 3.2, primary health care facilities in Mozambique are not allocated funding from higher administrative levels, but only receive in- kind transfers, which complicates tracking. Furthermore, although all transfers go through the district level, procurement of goods could be realized at either the central, provincial or district level depending on the specific input, further complicating the tracking exercise. As Lindelow (2006) emphasized, the survey team faced the choice between focusing: i) on a specific resource flow or program (in line with the capitation grant tracked in the education sector in Uganda), in order to collect very detailed data that would permit reliable assessments of leakage. However, this approach risked having a relatively limited impact because of its narrower focus. Alternatively, ii) the team could choose to take a broader focus and attempt to measure most health expenditures, but incur the risk that the data would not allow firm conclusions on leakage. The second option was chosen, in that the survey attempted to track all non-wage recurrent expenditures, drugs and other supplies, and human resources, at three levels: provinces, districts and facilities. Various data were also collected at the facility level, including information on staffing, user fees, service outputs and infrastructure. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 73 Several other problems complicated the tracking exercise in Mozambique, in particular the absence of fixed allocation rules for most resources. Indeed, other than for salaries, for which payments are determined by the number of workers in the facility and their salary grades, no fixed allocation rule governed the flow of resources. Indeed, while the national health budget established specific allocations by provinces and districts for non-wage recurrent expenditures, district allocations were not binding at the provincial level, and provincial administrations had considerable discretion over these flows. Similarly, provincial and district administrations had discretionary power over the allocation of drugs and other medical supply allocations. As previously discussed in section 4, the absence of fixed allocation rule led Lindelow et al (2004) to measure leakage in a slightly different way, as the difference between the resources that higher levels reported forwarded to lower levels and resources actually received by lower levels (i.e. “narrow� leakage). Furthermore, data quality turned out to be a serious concern at both the provincial and district levels. Large gaps in information were observed in about 75% of the districts between District Health Offices’ financial information records and those provided by the Provincial Health Offices. Furthermore, complete district level financial data could be collected for only about 40% of districts. Similar discrepancies between provincial and district records were also found in the case of medication transfers and health worker data. As mentioned by Lindelow et al (2004), various reasons, such as willful manipulations to hide financial irregularities, could explain these data discrepancies. They could also be due, however, to poor record keeping, poor data collection or data entry errors, etc. Indeed, these discrepancies in information between resource transfers collected from the sender and the receiver do not necessarily mean that resources have been diverted or captured in the process. This pessimistic hypothesis is rejected by Lindelow et al (2004) and Lindelow (2006) in the case of human resources, as provincial records often indicated smaller numbers of staff than there actually were in facilities (the reverse would have raised suspicion of the presence of ghost workers). Similarly, for medications, while in 60% of districts the total value of (individual) drugs reported is lower than that reported by the provinces, a few districts surprisingly reported having received more drugs than officially sent by the provinces! Ultimately, information discrepancies rendered estimation of leakage very difficult in Mozambique. As the report states: “documented discrepancies in information were a clear sign of weaknesses in management and control systems� (Lindelow et al, 2004, p. 6). Even though they could not find firm evidence of leakage, the report still observes that the lack of control “provides few incentives against fraud or malfeasance.� Lesson: A tracking flow strategy similar to that used in the education PETS in Uganda would probably have been productive in Mozambique. This would have required the focus to be on specific funding or flows for which records or PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 74 accounts of good enough quality on at least two levels could be identified. Furthermore, the bypassing of the provincial and/or district levels would probably have been indicated in the tracking exercise given the very limited and inconsistent information available at these levels. Mali: The PETS-QSDS in the education sector in Mali in 2005 faced a relatively similar data inconsistency problem to that encountered in Mozambique. Following data collection, it was observed that district and schools reported very different resources levels, and surprisingly schools often reported receiving more resources then reported to be sent by districts. A post-survey evaluation was realized in a certain number of schools and districts to identify the source of these inconsistencies. It was found that schools often received supplies (books and other material) not just from the district level but also from the MOE (through private suppliers). Furthermore, schools had difficulty separating supplies between the two sources and tended to incorrectly register these supplies as simply arriving from the district. Similarly, district records were inaccurate. Districts sometimes received materials from the MOE which were then sent to schools, but not properly recorded. Also, in some cases district records were based not on what had actually been sent, but on the reports of what schools reported having received. The survey team concluded that the information system was greatly deficient, and that important reforms needed to be implemented (Wane et al, 2006). Lesson: As the Mali and Mozambique experiences reveal, it is important to identify and make sense of the information actually recorded in official accounts and records. Such identification would need to be accomplished during a data and institutional assessment phase or pilot phase, before the launching of the full scale survey. Recommendation A first crucial step, before launching or fielding a survey, is to do a rapid data quality and availability assessment, in order to verify the availability and data and its specific characteristics (variables, length, etc.), and to verify the consistency of data reported across various administrative levels. This preliminary phase could help detect data quality problems and avoid a costly and time consuming exercise of gathering inconsistent data. Instead, in these last two cases, information should probably have been collected only at the facility level (in order to measure resources available for service) and at the central level (MOE) in order to know how much was officially sent, and to bypass all collection of hard financial data at the intermediary levels (province and district). That was what was done in the education sector in Uganda during the first PETS. As reported by Ablo and Reinikka (1998) a preliminary data assessment identified data availability and quality problems at the district level and information was only collected at the school and central government levels. Recommendation: Realizing a rapid data assessment before designing the instruments and piloting a survey should become a norm in all surveys. Reinikka and Smith (2004, pp. 52-53) write on that respect: PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 75 A rapid data assessment may be necessary to determine the availability of records at various layers of government as well as in the private sector, particularly at the school level. Some studies have failed as the availability of records in local government and schools was not adequately assessed beforehand. It is important to verify the availability of records early on, even if it means a delay and some extra up-front costs. Uganda (2000): Before launching the first QSDS in the health sector in Uganda in 2000, this type of rapid data assessment was realized in 1999. The objective was to determine if an in-depth facility survey was feasible, given especially the disappointing experience in the health sector in 1996. The later study found that data were now adequately recorded by facilities in good enough quality to allow a facility survey. PETS and QSDS present a wide variety of resources and funding flows that were chosen to be tracked, as well as various levels where information was collected. Table 7 presents the various levels and resources that were tracked by surveys. 5.1.3 An Alternative Approach to Tracking A slightly different way of tracking resources could provide an interesting approach to track complex flows. The innovation introduced in Chad in the framework of the PETS-QSDS in the health sector in 2004 tracked non-wage expenditures, but involved a sample instead of a census approach for tracking in- kind items. Chad: As in Mozambique, the survey in the Chadian health sector focused on MOH recurrent expenditures.48 As for most other tracking surveys,49 capital expenditures were excluded from the analysis despite their importance (56% of total health budget in 2003) because of the absence of information concerning their execution.50 As mentioned in Section 4.2, MOH non-wage expenditures to regions in Chad take the form of centralized and decentralized credits. The latter consists in a financial transfer to regions, while the former are used to procured in-kind items which are in principle sent to regions.51 48 In 2003, recurrent expenditures on health activities under the responsibility of the 14 regional health delegations (RHD) represent about 60% of the MOH recurrent budget (or two thirds of the MOH non-wage recurrent budget) 49 An exception is the Zambian survey, which tracked capital funding to schools. 50 Capital investments in the health sector in Chad are mainly financed by foreign donors. Information on their implementation is lacking because donors do not have common reporting procedures. 51 Resources are sent to the regions by an official MOH agent. All material exits are registered on exit slips by the central warehouse manager. When the materials reach the regional warehouse, the regional delegate verifies the list of materials and certifies that the materials have been received. He or she notes any missing materials or potential quality problems with the materials. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 76 Given that no administrative records were kept on resources sent to health centers at the district level, the tracking of resources focused on the central government, regional and health center levels. As in Mozambique, health service providers in Chad (health centers and hospitals) are not granted specific budgetary allocations in the national health budget. Their only sources of public resources are in-kind resources provided by higher administrative levels. Contrary to the Uganda capitation grant or the rule-based funding program component analyzed in Zambia, as in Mozambique there were no explicit allocation rules at the MOH level for allocating non-financial resources to regional administrations, districts and health centers.52 Facing the complexity of tracking the numerous in-kind items (drugs, medical and non-medical material), the methodology chosen in Chad was to use a sample strategy instead of a census approach to track in-kind items (medical material, drugs, etc.). For instance, for medical materials, the survey chose to trace a sample of eight items received at the facility level.53 The choice of materials was based on their shipment frequency in MOH invoice list.54 The risk of choosing a rare but high- value material (e.g. a car) would have been not finding that material in the visited health centers simply because not all of them were able to receive it. In contrast, by choosing frequently-shipped materials of small value (e.g. windex), it is likely that a maximum number of health centers would report receiving them. 55 This gave the survey team an upwardly biased percentage of health centers receiving materials from the authorities. The estimation of total material value received was calculated using the share of the eight materials tracked in the value of centralized credit deliveries56 52 All materials purchased with centralized credits to be sent to the regions originate from the central MOH warehouse in N’djamena. The destination points are the MOH warehouses in the regions and districts. Resources are sent to the regions by an official MOH agent. All material exits are registered on exit slips by the central warehouse manager. When the material reaches the regional warehouse, the regional delegate verifies the list of materials and certifies that the materials have been received. He or she notes any missing materials or potential quality problems with the materials (Gauthier and Wane, 2005). 53 Health centers were questioned about the receipt of mattresses, beds, sheets, blankets, blouses for nurses and midwives, soap and detergent. 54 Invoice lists and individual records for 2002 and 2003 were collected at the MOH level, which included items’ destinations and prices. 55 This could be thought off as the “Windex principle,� referred to by Wane (2006). 56 The value of materials delivered is estimated using MOH book prices. The total value of materials was estimated at 1,750,000 CFAF. Given that these eight materials made up 7.4% of the value of centralized credit deliveries (as records indicated at the central warehouse), it was estimated that health centers received approximately 23 million CFAF in centralized credits. Given that the survey covered half the health centers in Chad, the estimate was 50 million CFAF for the entire country (Gauthier and Wane, 2005). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 77 Similarly for medication, a sample of eleven medications and consumables was tracked.57 Value of deliveries to health centers were estimated using the prices charged by the Central Pharmaceutical Procurement Agency (CPPA), which appeared in the invoices. Based on the share of these items in the CPPA total sales, which was also collected, the total value of medications received by the sampled health centers was estimated, as well as for the country as a whole. Ultimately, despite having targeted non-wage recurrent expenditures and facing the additional challenge of collecting in-kind transfers, the tracking methodology of using sampled items and central government prices and shares allowed the measurement of resource leakage. Very substantial leakage was indeed observed: it was estimated that only 1% of resources officially allocated to regional health administrations effectively reached local service providers. An equity analysis of health sector expenditures among regions was also performed. Mali: A relatively similar approach for tracking in-kind items was adopted in Mali in the education sector where a selected sample of school supplies was tracked. However, instead of basing sample selection on shipment frequency, a more ad-hoc approach to item selection was used given that the surveyors did not have access to the list of materials sent to facilities. Instead, seven school items were chosen based on reception records of some schools. The valuation of in-kind items was estimated using the official prices quoted by the MOE for the purchase of in-kind items. The Mali survey went further by checking these official prices relative to market prices for the same items.58 This allowed verification of signs of malfeasance, but also determined whether the MOE or local authorities made correct use of economies of scale in purchases. To do so, the survey administrators collected price information from private suppliers about certain items (e.g. specific books), per unit and for large quantities of these items. They observed major differences between official and market prices and concluded that the MOE does not take advantage of economies of scale. Lesson and Recommendation: To track in-kind items such as school supplies, drugs and medical and non-medical materials, a strategy based on a sample approach could be used instead of a census approach. High frequency items should be targeted instead of high value ones. Valuation of in-kind items should be done using a standardized price list (such as the line ministry list if available).59 Resources received should be estimated relative to the share of the sample items in the global budgetary allocation. 57 The total value of deliveries was estimated at about 5 million CFAF. Based on the fact that these 11items accounted for 55% of CPPA sales in 2003, it was estimated that the total value of medications and consumable received by the health centers to be 9 million CFAF, and for the country as a whole, 20 million CFAF (Gauthier and Wane, 2005). 58 The Madagascar survey also compared official prices of in-kind items relative to market prices. 59 In certain countries such as in Ghana, however, the official value of in-kind transfers was not known by school officials. The Ghana survey used rough estimates provided by school respondents. However, such an approach introduces important biases in the data. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 78 5.1.4 Principles for Flow Tracking Selection In an ideal world, tracking surveys would probably want to track an entire sector’s budget. Some surveys have come close to that objective, such as the Zambian education survey, which has tracked non-wage financial transfers and capital expenditures. In most cases, however, this exercise has not been feasible and the challenge proved insurmountable to those who cast the net too wide, because of data quality problems, inconsistencies, recording procedures, etc. Recommendation: If the entire spectrum of expenditures does not lend itself to tracking, either because of poor quality data, recording procedures, disaggregated line items, large number of programs or sources of flows, or even survey budget constraints, survey teams have at least three potential strategies for tracking selection: 1. Pick one or a few specific programs that lend themselves to tracking. Once the program is identified, all the items associated with the program have to be tracked on the sending and receiving end on at least two levels including service provider, in order to estimate leakage. This was the approach chosen successfully by the Uganda 1996 PETS in primary education, focusing on the capitation (per student) grant program at the central government and facility levels. 2. Track a large budget line, if transfers are essentially all in cash. Good quality financial data must be available on at least two levels of the administrative structure for the tracking period. All financial flows during a certain time period have to be tracked. This is the approach successfully chosen in Zambia in the primary and basic education sector for non-wage education financial transfers. 3. Track a large budget line, such as non-wage recurrent expenditure, which could consist of cash and in-kind transfers. For in-kind transfers, there is a need to adopt a sampling strategy to track the flow. The strategy entails selection of a few small items with a high frequency in the flow. This is the approach successfully chosen in Chad. If this approach is followed, information on the share of these items, for instance medical materials, in the total budget has to be obtained in order to estimate total leakage in the budget, for instance. The same strategy could be followed with drugs, non- medical materials, etc.60 60 The first step is to obtain the flows of items from the sending end, and the share in total budget of these items. For instance in Chad, the survey team collected information from the CPPA, the public monopoly that manages medication purchases to all public hospitals and facilities. Records on the annual quantities and value of medication sent to facilities were collected and ranked in annual delivery values. The top ten medications in terms of delivery values were selected to be PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 79 Especially if records are in a much disaggregated state, as is typically the case at the facility level, this approach of sampling a few items instead of targeting wide (census type) coverage is probably indicated. Enumerators’ resources could then be allocated to collecting good quality data sheets. This trade-off between coverage and feasibility is somewhat equivalent to the trade-off between sampling and non-sampling errors in the survey sample choice phase (that we examine in the next section). Here, however, instead of the share of facilities relative to the entire facility population, the choice pertains to the share of expenditures to track with respect to the total budget. While a census would involve tracking all budgets, a sample choice requires selection of a share of the (in-kind) budget to track, sufficiently large to be representative but manageable enough to be realistically surveyed The choice of a sampling methodology is another fundamental step in a tracking survey, which we will now examine 5.2 Sample Selection Issues Without exploring sampling methodologies in detail, we will examine some issues related to sampling design in order to identify patterns in past surveys’ sampling approaches that had implications for survey results and performance.61 5.2.1 Examples of Sampling Strategies Various strategies have been adopted in past tracking surveys. Some sought to design a sample methodology based on valid samples for the results to represent the situation of the whole sector. Others sampling choices have been based more on convenience than on statistics. The objective for the sample to be representative of the universe it examines is rendered more complex in PETS-QSDS by the fact that there are several units to be analyzed (central government ministries, provincial and district administrations, local service providers, staff, service users, households, etc), and for each of these units, an adequate sample choice has to be made. With respect to service providers, the development of a representative sample requires information on the population under study. Most Sub-Saharan African countries do not have a reliable census of service providers. Still, such information regarding the presence and characteristics of local service providers, though imperfect, is generally available through the country’s education or health tracked from facilities. At the facility level, data were collected on these 10 items’ annual reception from the CPPA, as well as their buying and selling prices. 61 For detailed procedures to design sampling strategies for obtaining unbiased estimates, see for instance, Turner et al (2001). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 80 Management Information Systems (MIS) (i.e. list of schools or health centers). If in certain cases the coverage of these MIS could include private non-profit and for-profit facilities, generally not all types of facilities participate in the reporting procedure. Indeed, schools or health centers lists are often incomplete or outdated. Still most tracking surveys have used this information to constitute an initial sample frame of the facility population which was then, generally, verified and updated. Once the sample frame is determined, a sample stratification is often introduced given that the sample frame is generally large (sometimes containing several hundred schools or health facilities), and that different types of facilities may be targeted (public-private, rural-urban, etc.) as well as administrative levels.62 As emphasized by Reinikka and Smith (2004, pp.55-56), at least four issues have to be taken into account in the choice of a sample size. First, the sample should be sufficiently large and diverse to represent the various types of service providers. Second, some sub-categories may require more extensive sampling. Third, the adequate sample size is a trade-off between minimizing sampling and non- sampling errors.63 Non-sampling errors, which increase with sample size, are generally more a concern than sampling errors in tracking surveys as data are often in a highly disaggregated form and hence difficult to collect. Enumerator training and field testing are therefore critical in obtaining high quality data. Finally, of course, budget constraints must also be taken into account in determining sample size. Furthermore, sampling design is complicated when PETS-QSDS are jointly conducted. Indeed, to adequately measure leakage in a PETS, it is better to sample a relatively large number of local government (districts) which implies, with a strict budget constraint, reducing the number of service providers sampled in each district. However, in a QSDS, it could be preferable to interview a greater number of facilities in a smaller number of districts in order to assess difference of behavior and performance among types of facilities within districts (Reinikka and Smith, 2004). These various objectives and constraints have been resolved differently in past tracking surveys. Table 7 summarizes the samples and sampling procedures in various countries. 62 Stratification consists in dividing the survey population into sub-categories which are then sampled independently as if they were independent populations. This allows reducing sample variance and ensuring a sufficient number of observations for separate analysis of different sub- categories. 63 Sampling error, which decreases with the sample size, occurs because of the sample approach, which does not collect information on the whole population. Non-sampling errors, which increase with the sample size, are all other errors, mainly caused by poor survey implementation. Turner et al (2001, p. 106) define non-sampling error as all survey errors other than sampling errors, including response error, non-response, interviewer error, data entry and coding errors, errors of concept, and questionnaire design and wording errors. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 81 TABLE 7 SAMPLE AND RESOURCES MONITORED Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked Uganda 1996 PETS Education Education: Education: Education: 2 levels: and Health -Two criteria for sample -18 districts (out of 39) -Annual capitation (per student) - Central government (enrolment) selection: a) broad regional grant from the central coverage -250 public primary - Facilities schools government (financial and in- b) Representative of the kind transfers) school population in the Health: -Panel data 5 years: 1991-1995 district. -19 districts -Stratified random sample: -100 health facilities For each of the 5 regions of Health: non-wage expenditure the country, 2 or 3 districts were drawn with probability proportional to the number of primary schools in the district. The selection of schools in the district was based on school performance in the primary leaving examination results Health: In each district selected, 5 primarily government facilities were visited. 1999 PETS Education N.A. -16 districts -Data for 1999 and 2001 2 levels: and -218 public primary - Central government (enrolment) 2000 schools - Facilities 2000 QSDS Health -Two-stage stratified sample -10 District Data for 1999-2000: 3 units: Three principles: a) focus on administrations (out of 45) -Medical consumable -Districts dispensaries; b) all regions -155 (public, private for included; c) all categories -Contraceptives -Health facilities profit and non profit) (public, private for-profit and health facilities -non medical consumable -Patients non-profit) should be -capital inputs surveyed. -1617 patients -First stage: 10 districts were randomly selected. -Vaccines: 6 months data: -From the selected districts, -Drugs (6) : 1 month data a sample of public and non profit facilities was randomly drawn. Private for profit facilities were identified based on information given by public facilities. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 82 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked 2009 PETS Water -Selection of 8 districts from -At least 8 facilities per - Water and sanitation funds 5 levels: each of the 4 regions. district allocation from Finance -Central level Planning and Economic -Districts, - From the selected districts, -983 water users Development to districts -Sub-counties random selection2 sub – -Communities water points counties. -Data from FY 2004-2005 to FY 2008-2009 facilities -Water users Tanzania 1999 PETS Education N.A. -3 districts (out of 115) -Non-wage education and 2 levels: and Health -45 primary schools health expenditures - District -36 health facilities -Data for 1998 -Facilities 2001 PETS Education -Choice of districts based on -5 districts - Non-wage expenditures 3 levels: and Health geographical balance (rural- -16 primary schools urban) and whether or not a - Data for FY 1999-2000 and -Central government financial management -15 health clinics first half 2000 - Districts system (FMS) was in place. - Health facilities and schools -At the council level, 2 schools and health facilities chosen from the ward that houses the council headquarter and 2 others from a ward considered remote by the council 2004 PETS Education -Three-stage s stratified -7 regions -Capitation grant (cash and 3 levels: sample: First stage: book), development grant and Selection of 7 regions - 3 councils capacity building grant for -Central government according to a stratification - 210 schools school communities - Councils based on the Human -Data for 2002-2003 - Schools Development. Second stage: Selection of 3 councils according to their proximity to the regional headquarters Third stage: within each council 10 schools were sampled. Ghana 2000 PETS Education -Designed with the aim of -10 regions -Total recurrent expenditure 3 units: and Health matching data with the -40 districts (wage and non-wage) - District offices (health and Ghana Living Standard -119 primary schools -Data for FY 1997-1998 and education) Survey Round 4 (GLSS4) of -79 junior secondary 1998-1999 1998. schools - Health facilities and schools -Two-stage stratified -172 primary health -User perception survey was sample: First stage: From clinics carried out each of the 10 regions, 4 districts were chosen: 2 depressed, 1 average and 1 PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 83 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked better off (based on criteria of natural resources endowment, infrastructure and level of developments). Second stage: In each selected district, 3 primary and 2 junior secondary schools were selected, as well as 3 health posts, 1 clinic and 1health center. Service facilities located in the EA of the GLSS4 were automatically part of the sample as well as facilities reported used by GLSS4 household. Rwanda 2000 PETS Education Education: All administrative Education: - Recurrent expenditures 5 units: and Health units (provincial and district levels offices) were visited. -12 Provincial Education (cash, in kind contribution and -Central government 400 primary schools were Offices (out of 12) equipments) -Provincial selected using a two-stage -151 District Education -Districts stratified random sampling Offices (DEOs) (out of -Data for 1998 and 1999 method. 43 and 357 schools 154) -Facilities were sampled in urban and -390 primary schools (out -Users rural areas respectively, with of 2100) probability proportional to the number of schools in the Health: area. For the urban area, 9 -11 Regional Health of 43 schools were sampled Offices (out of 11) from Kigali. - 37 District Health Health: Nation-wide survey Offices (out of 40) of facilities and -250 health centers (out administrative units of 351) 2004 PETS Education Education: Two-stage: In a -36 districts (out of 106) - Teachers’ salary 4 units: and first stage, for each 12 -107 primary schools - Three funding programs: -Central government Health* provinces, 3 districts were selected. Second stage: (out of 2203) i) Funds for Genocide Survivors -schools primary schools were -48 secondary schools (FARG) -beneficiary students chosen in selected districts (out of 339) ii) Education support Funds for -banks as well as 1 secondary -36 Banks Vulnerable and Poor Children school per district. In iii) Capitation funds. addition, 1 secondary school -48 students per province which was determined to be the main -Data for FY 2003 beneficiary of the FARG PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 84 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked program was selected Sierra 2000 PETS Agriculture Two-stage: In a first stage, The sampling frame for -Financial and material , selection of the enumeration the Enumeration areas resources Leone 2001 areas for the 8 districts and education, (EAs) was therefore -Data for the second half of health, 12 constituencies in the made of 102 chiefdoms Western Areas were 2001 security -176 health facilities sector, selected. social -477 primary schools welfare, rural developme nt and local developme nt, water and sanitation. Zambia 2001 PETS Education -Stratified random sample -33 districts - Non-wage funding for basic 6 units: QDSD from urban/rural location. -182 primary schools education (fixed-school grant, -Central level -Schools were chosen from (grades 1-9) discretionary non-wage grant program) -Provinces 4 provinces (2 richest and 2 poor with enrolment rates -Data for June 2001-June 2002 -Districts just marginally better than -Schools the worst performer) -Households -Student achievement 2007 PETS Health 18 hospitals, 90 rural Various resource flows 4 units : QDSD health centers, 40 urban (government budget, basket funds, vertical project funds and -central government health centers. internally generated funds) -district FY05-06 financial data -health facilities -patients Mozambique 2002 PETS Health -Sample was selected in 2 -11 Provinces (out of 11) - Allocation: recurrent budget 5 units: QSDS stages: random selection of -35 Districts panel data 2000 to 2002 -Provincial directorate of health districts followed by random - Execution of district recurrent selection of facilities within -90 Public primary health - District directorate of health centers budget 2000 and 2001 districts - Health facilities -167 workers - Drugs and other supplies -Sampling weights were - Staff used to provide for -679 patients - District and facility data on user fees: - Patient exit polls nationally representatives estimates - Service output: - Users and staff were also selected randomly PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 85 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked Nigeria 2002 PETS Health -30 local government 3 units: QSDS -252 health facilities - Local government -700 staff - Facilities - Staff Senegal 2002 PETS Health -Stratified sampling -10 districts - Panel data 6 years: 1997- methodology was used to be -37 local governments 2002 representative of urban/rural -100 facilities - Decentralization Fund level. (recurrent non-wage -5 regions were first expenditures) randomly selected. - Equipment Fund -Then 2 departments from - Investment program each region, and 10 health posts and 1 district within each department were selected. Cameroon 2003 PETS Health -Covers all 10 provincial -36 district (out of 36) -Non-wage recurrent - Provinces health delegations and the -143 health facilities expenditures - Districts two main urban areas, (including 36 hospitals - Data for 2001-2002 and 2002- Douala and Yaoundé. - Health facilities and 34 private facilities) 2003 (9 months) -Stratified multistage. - Patients -2952 patients (in-patients -In a first stage, 3 and out-patients) departments (UP) in each province are selected. 12 health facilities are selected in each department including the district hospital and 1 private clinic. -User survey: 6 users are selected per “zone de dénombrement� in each district for the main urban areas and 3 in others. 2004 PETS Education -Covers all 10 provincial 720 primary schools and Various resource flows education delegations and 432 secondary schools the two main urban areas, (of which 324 general Douala and Yaoundé. secondary schools and -Stratified multistage. 108 technical secondary -In a first stage, 3 schools) departments (UP) in each province are selected. -12,000 households -User survey: 8 households are selected per “zone de PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 86 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked dénombrement� in each district for the 20main urban areas and 14 in others. Madagascar 2003 PETS Education Education: Education: Education: Education: and health First survey: stratified First survey: 36 districts -Two main funding programs to 2 levels: random sample covering and 326 primary schools schools : CRESED and IPPTE -Districts school authorities 144 communes in 36 Second survey: 185 -District allocation data collected (Cisco) districts (out of 111) primary schools were for two years, 2001-02 and -Schools Second survey: track budget surveyed, about half of 2002-03 at the school level. The the schools included in purpose was to provide the first survey. nationally representative Third survey: 24 district figures on budget and administrations (out of leakages from Cisco to the 111) school level. Health: Third survey: budget First survey: 53 FS allocation at the district level (“formations sanitaires�) Health: in urban areas and 100 in First survey: selection of the rural ones (in total, 84 public health facility the public health centers and most visited in each “zone 69 private ones (of which de dénombrement� 19 faith-based ones)). Second survey: household Second survey: 12 survey to track their living households in each UPS conditions and their in urban areas and 14 in attendance to health rural ones facilities. Third survey: 1350 Third survey: individual individuals (10 individuals survey (user perception per FS) survey carried out) 2006- PETS Education Stratified random sample: 229 schools Education: 2 units: 2007 and Health -For each the 6 biggest 113 health centers regions (out of 22), 2 Cash, caisse école, school kits, -district districts were randomly school equipment, textbooks -facility level selected. and salaries. -From selected districts, 3 Health: communes were randomly Current expenditures, non selected giving greater medical consumable, drugs weight to the communes with more schools. -Within each commune, 3 public primary schools were randomly selected. In the selected communes, all public health centers of PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 87 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked Type II were visited. If public health centers of type I were present in the commune, one was visited based on random selection. Namibia 2003 PETS Education - Convenient sample of Education: Education: Education: 5 units: and Health regions (7 out of 13) 113 public and private Non-wage recurrent -Central government - Random sample of schools primary and secondary expenditures -Regions and health facilities within schools School Development Fund regions -Schools Health: 2002-03 -School board 9 district hospitals Health: -Students 10 health centers Non-wage recurrent Health: 5 units: 22 clinics expenditures -Central government 192 patients 2002-03 -Regions -Districts -Facilities and Patients Chad 2004 PETS/Q Primary -Mixed stratified sample- -14 Regional heath -Non-wage recurrent 7 units: SDS health care census strategy delegations expenditures -Central government In the fist step, either 1 or 2 -13 Regional pharmacies -Data for 2003 -Regional heath delegation districts depending on the -21 District health -8 medical material were traced number of districts in the -Regional pharamacies delegations -10 drugs were traced region were selected at -District health delegations random in each of the 14 -281 Health Facilities (public, private for-profit -Health Facilities delegations. In the second step, all health centers were and private non-profit) -Staff identified and visited on a -1274 workers -Patients census basis. -1801 Patients Kenya 2004 PETS Education Facilities Education: Education 4 units: and Health Stratified sample: In each of -26 districts and 3 Bursary funds -Central government the 8 provinces, 2 poor and divisions in Nairobi 2001-02, 2002-03, 2003-04 -Districts 1 rich districts were selected province using poverty index. Health -Facilities -330 public secondary Facilities were selected in schools Non-wage expenditures (funds -Users each districts using and in-kind) systematic random sampling - students 2003-04 Users: random exit -598 parents interviews of non serious Health: cases in health centers. -27 districts and 3 Random selection of 2 divisions in Nairobi students who had received a province bursary in each class. 2 of PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 88 Survey Resources monitored and Country Year Type Sectors Sample strategy Sample years Levels/Units tracked the nearest parents in each -214 public health sampled school were facilities interviewed. -772 patients Mali 2005 PETS Education A 3-stage sample strategy: - -3 regions Selected government 4 units: QSDS selection of 3 regions -15 CAP expenditures (Bamako and 2 others -525 schools -Direction d’Académie randomly selected) -1255 teachers d’Enseignement -In each academy, two CPC -Centre d’Apprentissage were randomly when the Pédagogique academy has four or less, -Schools and three CPC were -Teachers selected throughout academy with at least five. -Finally, in each CAP, thirty- five schools have been investigated. 2006 PETS Health and A one-stage strategy for 7 rural community Selected government Health QSDS education urban health facilities.. facilities, 3 urban ones expenditures in health A two-stage strategy for 3 units: and 2 benchmark facilities (mosquito nets, medicines, rural health facilities: out of a total of 504 medical supplies, salaries in -Central government selection of 7 out of 35 institutions community health facilities, - Regional districts and then selection operating costs) of a facility within this - Facilities district. Niger 2008 PETS Education Three regions, Dosso, Education: Education: Education: 6 units: and Health Tillabéri and CU Niamey were selected based on 3 regional education Selected government -Central government their close proximity to the offices out of 8, 21 district expenditures in education -Book suppliers capital. Within each region, education offices out of (textbooks, notebooks and 23, 299 primary schools drawing books for students) -Regional districts and service providers were randomly out of 2,736 in the 3 -Districts selected to ensure data regions surveyed -Schools were regionally Health: -Students representative. 3 regional health offices, Health : 5 units: 14 district health offices out of 14, 11 district -Central government hospitals out of 14, 60 -Private drug suppliers central clinics out of 233 and regional -Regional -Districts -Facilities Sources: Survey reports; Uganda, Tanzania: Reinikka and Svensson (2004a); Lindelow et al (2006). Note : N.A.: Non Available. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 89 PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 90 Tanzania: Table 7 reveals the large variation in samples and sampling units among surveys. In Tanzania for instance, in both the 1999 and 2000 PETS, only a very limited number of schools and health facilities were surveyed and results thus have limited applications. The rationale was that the surveys were more tentative in their approaches, being realized on a pilot basis and hence did not aim to produce nationally representative results. Uganda (1996): In contrast, in the 1996 Uganda education PETS, a relatively large sample was used. The sample selection of schools was governed by two criteria a) broad regional coverage, and b) representativeness of the population of schools in the district (Ablo and Reinikka, 1998). A stratified random sample was utilized. For each of the five regions of the country, 2 or 3 districts were drawn with probability proportional to the number of primary schools in the district. In the 18 districts selected (out of 39), the number of schools visited was based on the total number of schools in the districts. The total sample comprised 250 public primary schools. Uganda (2000): The sample strategy for the QSDS in the health sector in Uganda in 2000 also entailed a two stage stratified sample. The design was governed by three principles: a) for homogeneity purpose, the focus was to be placed only on dispensaries; b) to capture regional differences and subject to security constraints, all regions should be included; c) to capture ownership categories differences, all categories (public, private for-profit and non-profit) should be surveyed (Lindelow et al, 2003). For public and private non-profit dispensaries, the initial sample frame was based on the 1999 MOH facility registry. For private for-profit facilities, given that no census existed, the strategy retained was to ask sampled public facilities to identify the closest private for-profit facility.64 Data were collected at the district administration and health facility levels. In addition, a patient exit poll was carried out. Mozambique: The overall sampling strategy in the 2002 PETS-QSDS in Mozambique was quite sophisticated. Five units of observation were considered in the survey: provincial and district administrations, public primary level facilities (health posts and health centers), staff and patients. A two-stage stratified sample was used for the facility sample selection. The sample was stratified in terms of urban and rural facilities, in order to contain sufficient urban facilities to allow adequate analysis. Given that no reliable sample frame existed, an initial list of primary facilities provided by the MOH was updated by enumerators during field work (Lindelow et al, 2004, pp. 8-9). A random selection of districts was first drawn. All 15 urban districts were automatically 64 Ablo and Reinikka (1998) do not provide further information on how selection was made of private for-profit facilities based on this ad-hoc list. Furthermore, they don’t discuss potential biases that could be introduced through this sampling procedure. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 91 selected, along with a sample of 20 rural districts (selected with a probability proportional to the number of first level facilities in the initial sample frame).65 In the second stage, two considerations were taken into account in selecting facilities, a) in each district, several facilities needed to be selected in order to assess the extent to which facility performance could be explained by local management instead of factors at higher levels; b) to reduce implementation costs, facilities needed to be clustered in some areas. In this second stage, three facilities were selected (with equal probability) in rural districts and two in urban districts. The choice of facilities was made by enumerators in the field using “preprinted forms with a random numbers series and clear criteria for listing facilities�66 to minimize the risk of manipulation (Lindelow et al, 2004, pp. 8, 11). According to the report, the sample in the rural area is approximately self- weighted, that is, all rural facilities have the same chance of being selected. Overall, sampling weights were used67 to ensure that estimates are nationally representative. A staff questionnaire was administered. The sample design was as follows. In each facility, up to three workers were interviewed. The head of the health center was always interviewed, while the other two workers (or less for facilities with less than 3 employees) were selected randomly among staff members with technical responsibilities (excluding helpers, etc). Furthermore, an exit poll of patients was carried out. The selection of patients was done randomly to reduce selection biases (which could be introduced if the interviews are restricted to certain hours of the day, if discretion is left to the interviewers, facility staff or patients themselves). For patients, only those attending “normal� outpatient services were surveyed and all other types were excluded (maternal and child health consultations, family planning and similar services).68 To select patients randomly, numbered tokens were handed out to all outpatients who were asked to see the enumerators after the consultation. Patients were then selected randomly with intervals between the users, which were determined by estimates of the total number of users expected on a particular day (Lindelow et al, 2004). The final patient sample presents variations in terms of number of patients interviewed in each facility because of errors in predicting the number of patients and idiosyncrasies in ways the sampling process was implemented. Still, Lindelow et al (2004) notes that the resulting sample of patients is more likely to be representative than ad-hoc sampling alternatives. 65 The rural sample was “implicitly stratified� to ensure a representation based on the number of facilities in the region. 66 Criteria not reported. 67 Weights are the inverse of the selection probability of each unit (Lindelow et al, 2004, p. 9). 68 Lindelow et al (2004) notes that many patients’ interviews had to be carried out in the local language, which involved an informal translation of the questionnaire by the enumerators. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 92 Lesson: In previous surveys, samples have been chosen using various methodological approaches, often involving stratifications by levels or categories. Some surveys have correctly used a random (or systematic) approach to select the sample. However, some facility samples have been chosen informally on the basis of their convenience of access or ease of interviewing, instead of through the use of random selection techniques.69 The key problem, of course, with such informal methods is that valid inference about the whole population of facilities (as well as clients or staff), cannot be made from the survey results. Indeed, for a sampling strategy to be valid, the sample has to be representative of the universe it aims to apprehend. Recommendation: Convenience sampling should not be an option for tracking surveys. It is clearly crucial for samples to be chosen through a valid statistical strategy, as valid generalization about the whole service provider population or other units cannot be made if the sample is biased. Sophisticated sampling procedures, such as the one adopted for patients in Mozambique, have to be weighted in terms of their costs and benefits compared to more simpler approaches in terms of potentially less biased samples, but also in terms of variability in number of patients by facilities and in differences in strategy implementation among enumerators. 5.2.2 Alternative Strategy: Mixed Sample-Census Approach There are sampling design alternatives to a “pure� stratified sample approach (where samples are drawn at each strata level) in particular, the mixed sample- census approach. In such a strategy, areas (for instance, districts) could be selected in a first stage of sample selection. In a second stage, a census approach could be used in which all facilities in the district are inventoried and then surveyed on a census basis. This strategy could sometimes be better suited to measure leakage than the pure sample approach (in which a small number of facilities are visited in each district). In particular, such a strategy could be more appropriate in the case where no fixed-allocation rules are in practice and only “narrow� leakage could be measured. The problem with the first (sample strategy) approach is that one cannot estimate reliable leakage figures in such a case as one needs complete data on the flows tracked. Indeed, when measuring “narrow� leakage (as the difference between what was reportedly sent and what arrived), all resources in an area have to be assessed. With only a few facilities visited per district in the sampling strategy, it is not possible to say anything about resource use in that specific district, in terms of reception of materials, financing, drugs, user fees, etc relative to other districts (or aggregated at the provincial level). Chad: The alternative strategy of combining a stratified sample in the first stage (to choose districts, for instance) and a census in the second stage (within 69 The same problem of bias arises through the use, for instance, of the quota approach in the case of exit poll surveys, such as selecting the first 5 or 10 clients at a facility (Turner et al, 2001). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 93 districts), allows evaluation of the use of all resources within districts, and hence measurement of leakage in the case of fixed allocation rules. It should be noted that this strategy presents the further advantage of allowing obtaining representative results without the needs for weights (Turner et al, 2001, p. 49).70 This strategy was used in Chad, where a two-stage sample-census strategy was utilized. First, in each of the country’s 14 delegations, either one or two districts (depending on the number of districts in the region) were selected at random. Second, in each of the selected districts, all the health centers were identified and visited.71 Also, given the importance of the capital, N’Djamena, all its health centers were included in the sample. The original health center list was provided by the MOH Division of Sanitary Information and Statistics. In addition, surveyors were instructed to identify and visit all health centers not on the initial list in a selected district, especially the private clinics. The final sample included 281 health centers, of which approximately two thirds were public, 14% private for-profit and 19% private non-profit.72 Recommendation: Choice between pure sampling strategies and mixed sampling-census strategies should be considered. The second strategy could be more practical in the absence of fixed allocation rules and where “narrow� leakage is measured, for which all resources in an area have to be assessed. 5.2.3 Household Survey Linkages Tracking surveys’ sampling methodologies also differs in terms of linkages with population surveys. Indeed, facility survey samples could be chosen independently of any household survey, or instead linked to population survey sample areas.73 In the second type, the design of the sample requires adoption of the same sample areas used to generate the household survey. While the link with household survey constrains the facility sample in several ways and could affect its representativeness such linkages could possess important value added. Indeed, in a linked survey, the facility survey provides information on the service supply environment to which the population included in the household survey is exposed (Turner et al, 2001). It then allows investigation of how population behavior and outcomes are affected by the presence of service providers.74 For the purpose of using PETS/QSDS for specific 70 Indeed, there is a need to determine the appropriate sample size or various stratification between types (private, public, sizes, etc) when the facility sample is not a census. (Turner et al, 2001). 71 One regional delegation (B.E.T.) was not included in the final sample because of security problems in the region at the time of the survey (Gauthier and Wane, 2005). 72 About two thirds of the health centers were located in rural areas, less than one-quarter in the capital and 14% in other urban areas. 73 Turner et al (2001, p. 25) refer to these two types as “stand-alone� facility survey and “linked survey,� respectively. 74 Non-linked surveys could be used to conduct multivariate analysis but in such case, the household data needs to be aggregated at some administrative or geographical level to match the PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 94 program impact assessment, the linkage of facility and household surveys is thus fundamental Ghana: Few tracking surveys in Africa have chosen the route of linkages. The Ghana 2000 PETS is an exception. Sample selection was structured to allow linkages with the Ghana Living Standard Survey (GLSS 4) conducted in 1998, and thus the matching between households and facilities data sets. The PETS sample covered 143 of the 300 GLSS 4 enumeration areas. 75 Schools and health facilities located in the enumeration areas of the household survey were automatically part of the sample, together with facilities used by GLSS 4 households. There is a trade-off with such an approach as a choice has to be made for the sample to be representative of the population of individuals or of the facility population. In the Ghana case, the final facility sample is probably representative of the household population and not of the facility population. In such situations, care should be taken to assign the proper weights to the facility sample to avoid biases. In the Ghana case, although the report does not provide details on the question, it could be inferred that the weights used were most probably based on individual population instead of facilities, which in such case would introduce a bias. Chad: In Chad, efforts were made to ensure linkages between the tracking survey and a household survey (ECOSIT) that was about to be launched in 2004 at the time of the design of the PETS-QSDS. A health module was devised by the tracking survey team and included in the household survey. Furthermore, facility survey codes were used in the household survey in order to be able to identify the facilities used by households.76 However, the linkage is not perfect; given that the household survey is much broader in scope, it includes enumeration areas (EA) that were not included in the facility survey. Zambia: The Zambian PETS/QSDS took a different path to link supply and demand. Das et al (2004) innovated in devising a household survey which was then linked with their sample schools’ students in order to assess schooling achievement. The household survey was conducted in parallel with the school survey in a sub-sample of schools in two consecutive years in four provinces.77 The sub-sample of schools was chosen among schools sufficiently far apart (as measured using a Geographical Information System that was used to map the location of every school). This technique allowed matching household and school outcome of interest with program data. The unit of analysis will then no longer be the individual. (Turner et al, 2001, p25) 75 The GLSS 4 sample was selected from a sampling frame of 13,000 enumeration areas developed from the 1984 population census (Ye and Canagarajah, 2002). 76 In general, in such household surveys, it is not possible to identify the health centers that households utilize closer to their house. 77 By comparing test scores from these two years, they were able to derive learning achievement results. These were then tested against two types of funding to assess the impact of education expenditures. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 95 data. However, their results are limited to the sub-sample of schools matched with rural households (sample size not provided), given that the method chosen disqualifies schools in urban areas which are not widely enough separated. Furthermore, it should be noted that the household stratification chosen was not based on a preexisting survey. In that respect, the household sample may not be representative of the country’s population. However, the objective in Zambia was to develop direct linkages between schools and households. Indeed, by collecting these household panel data, Das et al (2004a) were able to incorporate household assets and spending in a detailed equity analysis, which revealed important elements on public and private spending and learning achievements. Lessons and Recommendation: The choice of linking facility and household surveys depends on the objective pursued. The choice depends ultimately on what population one wants the survey sample to be representative of (service providers, households, etc.). Still, there is potentially huge value added of linking supply and demand results in order to be able to examine the impact of service provision in terms of population outcomes. It could thus be recommended that tracking surveys add a household survey component whenever possible. When a household survey is not directly incorporated in the QSDS, teams should try to establish contacts with LSMS teams or other household surveys in the country. If a household survey is almost completed, it could be worth establishing links with it. Whatever the route taken (stand alone or linked surveys), again the sampling strategy must be scientifically chosen, in order for the sample estimates to provide valid inference about the sector under study. 5.3 Length of Quantitative Data Tracking Another methodological choice that has consequences for survey results relates to the length of the data to collect. There has been much variance in this respect in past tracking surveys. This introduces problems of comparability and consistency among data collected within and across countries. As observed in Table 7, which presents the length of data collected by countries in terms of types of data collected and levels, data collected vary substantially. In some cases, data collection covered period of five years and sometimes only one month for specific data. Such variations greatly depend, of course, on the difficulty of collection of the specific data targeted and the recording procedures. Still, in designing survey instruments and determining length and periods for data collection, the trade-off between data accuracy and completeness must be considered. Completeness of data collected (or targeted) generally detracts from accuracy, as typically more missing data will be found by enumerators on a longer time period. Given that data at the facility level generally are in quite disaggregated forms, the work of the enumerator is made very tedious, if not unfeasible, when too much data and too long a period are targeted. During the rapid data assessment phase, the length of data collected should be determined PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 96 and relatively standardized among types of data, with the objective of balancing the goal of completeness with data accuracy. It should be noted that short collection periods are generally associated with problems of seasonality, which could bias the data. If data are collected on a monthly basis, for instance, there is of course a need to aggregate the data on an annual basis. While in the education sector, there are few problems of seasonality (except maybe for dropout levels), in the health sector, seasonality is generally an important issue. If data are compiled on a monthly basis and only certain months are available (or collected), aggregation on an annual basis is problematic, as frequentation data and types of illnesses are potentially quite different from month to month and across regions. In general, it is better to collect annual data if they exist instead of monthly data, except if the issue of seasonality of services is specifically targeted as a management or performance issue. If only monthly data for a short time span are available, a strategy has to be devised to reduce biases introduced by seasonality. Various approaches to deal with seasonality issues have been used in past surveys. Details of these approaches were, for the most part, not clearly presented in survey reports, which precludes further analysis of this issue here. However, it is certainly a problem that needs further attention and discussion before further field work is carried out. Leaving aside the question of seasonality, a strategy that could greatly simplify enumerators’ work and increase data quality is to adapt survey instruments with the standard accounting and reporting procedure prevailing in the country. Chad: In Chad, for instance, health facility data sheets using the same structure as the official monthly activity reports (RMA) sent by facilities to the MOH were utilized to facilitate data collection at the facility level. Furthermore, the RMA electronic files compiled by the MOH (through district administrations reporting) were also collected at the MOH level. It was this possible to verify (“triangularize�) the validity of information provided at the facility level. In several other countries data sheets were customized to facilities’ recording practices. Recommendation: Length of data collected should be determined during the rapid data assessment phase (and verified during the pilot phase), in order to balance completeness objective and data accuracy. Seasonality issues have to be considered and a proper strategy devised and clearly explained to reduce potential biases. Survey instruments, in particular facility data sheets, should be adapted and customized to the standard accounting and reporting procedure in practice at the facility level in the country in order to facilitate enumerators’ work and increase data quality. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 97 5.4 Data Sources: Records versus Recalls Another methodological issue which has implications for survey results and performance relates to source of quantitative data collected. There has been some variance among surveys in terms of data sources because in certain countries, quantitative data were based on recalls from the respondent instead of being based on accounts or records. To minimize measurement errors, it is of course recommended to use records as much as possible and, as mentioned earlier, to adapt survey instruments to the specific recording procedures in practice in the administrative units and facilities under study. Still, in some past surveys, respondent recalls were used given either because no records were available at the facility level, or for other unspecified reasons. Data quality is certainly in doubt in such situations. In Ghana, for instance, the information collected from schools was based on recall data rather than data obtained from school records or accounts, making the data significantly less reliable (Reinikka and Smith, 2004). In Madagascar also, data quality problems were reported. Records of funds arrivals (accounts, receipts, bank statements, etc) were difficult to obtain at the district and school levels and enumerators sometimes had to rely on amounts reported by respondents (Francken, 2003) It should be noted that for certain inputs, for instance resources allocated directly by donors, information based on records could be very difficult to gather. Indeed, resources received from NGOs or directly from other donors at the facility or local government levels, are generally not recorded using standard procedures. Enumerators have little alternatives then to collect such information from the recall of the respondent in-charge. Recommendation: In order to minimize measurement errors, it is recommended to use records or accounts to collect quantitative or financial data. In exceptional cases, when no other sources of data are available and data based on recalls are collected, clear indications in that respect should be reported. Furthermore, as previously mentioned, survey instruments should be adapted to the specific recording procedures in practice in the administrative units and facilities under study. 5.5 Survey Timing In the pre-design phase, an essential decision is to determine when to field the survey. One important element to consider is of course the fiscal year period in use in the country. If the target is to obtain data on flows of funds over a one-year period, these clearly have to correspond to the fiscal year. The tracking should in general always be done on at least the last completed fiscal year. Ideally, the survey should be carried out two or three months after the end of the fiscal year in PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 98 order for accounting books to be closed. In any respect, the tracking should always be done on the preceding fiscal year, never on the current one. This problem was encountered in countries such as Cameroon and Madagascar, where data for incomplete fiscal years were tracked. In Cameroon, data were collected for 2 fiscal years (201-2002 and 2002-03). However, the survey was carried out in October-November 2003 at the beginning of the last quarter, which precludes the calculation of budget execution and arrival rates of material during the last surveyed year because the year was not completed.78 The surveyors were able to measure imperfectly what had arrived during the first eight (to ten) months of the year, but could not relate the differences between arrivals and official annual budget lines, as delays in procurement and deliveries could simply be the reason for the observed differences. Similarly, the Madagascar survey tracked two main funding programs to schools79 and examined delays in salary payments to teachers. However, the timing of the field research (April-May 2003) did not capture clear information about leakage given that the 2002-2003 school year was in progress, and some of the data were collected for the incomplete school year. Indeed, the study could not distinguish between direct leakage and delays in budget execution of the main schooling programs studied due to the data collected (Francken, 2003, p.22). In Zambia, the same problem was faced for in-kind items which could not be tracked in the PETS-QSDS, given that the survey was realized during the school year.80 Recommendation: The fielding of a survey should ideally be done two or three months after the end of a fiscal year, in order for accounting books to be closed. If this is not possible, and the survey is fielded in the course of a fiscal year, the data tracking should cover the preceding year, never the current year. 5.6 Multi-Sector Surveys Multi-sectors surveys have been realized in various countries (e.g. Cameroon, Uganda, Rwanda), generally encompassing health and education simultaneously in a single project. In a number of cases, more than two sectors were included: in Tanzania, for example, water supply, rural roads, agriculture, etc. were also surveyed. 78 République du Cameroun (2004, p. 35). 79 The third survey tracked two education programs, CRESED and IPPTE (Francken, 2003). 80 As mentioned earlier, in the Zambian case, it was argued that not tracking in-kind transfers should not have affected tracking results given that relatively few in-kind items are procured for schools in Zambia, as most transfers take the form of cash and personnel. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 99 Economies of scale, however, do not seem to translate into tracking survey quality. Indeed, various problems are associated with multi-sector surveys. First, there have been frequent problems of sample design: as often the same enumeration areas (village, wards, etc.) were chosen in order to reduce costs. However, such an approach necessarily means that some of the facility samples are biased, as the sample reflects one of the facility populations. Second, multi-sectors surveyed are generally associated with much less in-depth data collection in certain sectors. Typically, one or some sectors will be sacrificed due to budget or other resource constraints if, for instance, the same teams of enumerators have to visit both schools and heath centers in a given area, within a limited time frame. Finally, with respect to reporting, some sectors have been prioritized and were better reported in multi-sector surveys carried out so far, while the other sectors tend to be much less thoroughly analysed. For instance, reports for the Tanzanian water supply, rural roads are not publicly available. Recommendation: One cannot categorically claim that multi-sectors surveys should never be carried out. However, such surveys should not be the norm, and if they are put forward, extra precautions have to be taken to ensure project quality. Separate sampling designs must be chosen to ensure representative samples in every sector, interviews should probably be conducted by different teams or, if by the same team, at different times, and adequate time and resources must be allotted. 5.7 Data Management An issue which is directly related to the survey instruments that we examine in the next section is data management. There is clearly a need to harmonize data management in survey works in order to allow comparability of data sets. This would be favored by the standardization of the data management programs. Thus, CSPRO or CSPRO X (which allows data to be entered directly during the collection phase)81 could probably satisfy the needs and requirements of the various survey endeavors. Recommendation: Economies of scale in training and building of data entry programs would certainly be observed, as well as improvements in data quality and comparability, if a standard state of the art data management program, such as CSPRO, was consistently used in future survey endeavors. 81 CSPRO X requires the use of a special portable data entry device for each enumerator. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 100 5.8 Questionnaires Design There has been much variance in terms of instruments used to collect quantitative and qualitative data. We now examine instrument design, in particular the basic structure of questionnaires, that is the minimal set of elements that should probably be included and information that should be collected in tracking surveys. We start by discussing some general issues related to questionnaire harmonization before examining some basic elements of the main survey modules. 5.8.1 General Considerations There is clearly a trade-off between standardization of survey instruments -- a common general structure to allow maximum comparability-- and flexibility in instrument design -- to allow accounting for the various specificities of each country. Nonetheless, there is potentially a minimal level of methodological coherence among surveys that should be targeted. For cross-country comparability purposes, one would probably want to agree on a relatively standard general approach and a common minimum structure of questionnaires, including a sampling methodology, etc.82 The current weak harmonization among tracking survey instruments used has reduced possibilities of cross-country comparisons or regression analysis. The fact that surveys were financed and supported by different groups, with their own agenda and priorities, certainly made it more difficult to develop a common methodology. The tradeoff was that, potentially, important analyses were left aside in the process. It may be possible, however, that the potential uses of tracking surveys for cross- country comparisons could be somewhat limited. Lindelow (2006) argues that given the complex institutional factors at play and the little variation in their measures, tracking surveys may not be well suited to make inferences about the determinants of the outcomes of interest. Accordingly, they may be more useful as diagnostic tools than analytical tools. Tracking surveys could also be well suited to evaluate the impact of institutional reforms through repeated surveys (Reinikka and Svensson, 2004a). Among other questions that will have to be further investigated is whether instruments should be devised by social sectors or with the same basic structure across sectors. The institutional arrangements and market environment vary substantially among social sectors. For instance in the water sector, contrary to the health or education sectors (where a multitude of service providers are present), there could be a unique or a small number of large providers sometimes combined, in countries where decentralization is observed, with a certain number of small water providers and private resellers. Such contrasting industrial 82 Other surveys, such as DHS, in this respect are very rigid in terms of facility module components across countries. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 101 structures complicate standardization, but the question of standardization of questionnaire design across sectors would need to be examined in greater detail. As for other harmonization considerations, as previously mentioned, these questions should be the subject of working group discussions to reach a more in- depth understanding of the implication of these questions, and consensus on the way to achieve harmonization. Nonetheless, we can tentatively offer some general considerations regarding certain harmonization issues. 5.8.2 Instrument Design There has been much variance across surveys in terms of instruments used to collect quantitative and qualitative data. We examine here some elements that should probably be included in questionnaire modules. We mainly present sections headlines and some main items. For specific examples of instruments, we would refer the reader to the almost state-of-the-art questionnaires devised in the education sector by the Zambia 2002 PETS-QSDS team (see: Das, Dercon, Habyarimana and Krishnan, 2004). Their specific instruments include modules for central government units, provincial administrations, districts, schools, staff, students and households. In the health sector, we would refer the reader to the Chad PETS-QSDS 2004 instruments (see: Gauthier, Gersovitz, Philipson and Wane, 2004), which are, in part, based on the Mozambique 2002 PETS-QSDS instruments (see: Lindelow et al, 2002). Central Government Unit Module In PETS in particular, it is essential to collect information about the roles played by each ministry, administrative unit and level in the frontline provider supply chain and monitoring process. Furthermore, it is imperative that administrative data be collected at the central level. These annual data, both for the entire sector under study and specific programs tracked, should be disaggregated at the provincial, district, and potentially service provider level if available. Such data would allow measurement of resources officially allocated to decentralized levels. It may also make it possible to “triangularize� information provided at lower levels. Service Provider Module At the service provider level, the Facility Questionnaire seeks to collect a large spectrum of data and information on facilities’ characteristics and activities. The questionnaire is generally administered to the person in charge of the facility. In schools, this person is generally the head teacher or school director. In health centers, the person tends to be the head doctor or head nurse. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 102 The general structure of the service provider questionnaire module should include data collection on, at least, the following five areas: 1) characteristics of the facility; 2) financing; 3) inputs; 4) outputs; 5) institutional support: 1) Facility characteristics/infrastructures: i.Location, type, level, ownership, catchment area, organization and services offered, competition in provision of services, etc. 2) Financing i. User charges, cost of services by categories, expenditures, and financial and in-kind support, etc. 3) Inputs: i. Staff, material and other inputs: books, supplies, drugs, vaccines, medical and non-medical consumables, infrastructure (capital inputs), etc. 4) Outputs: i. Sector specific: number of students /teacher, enrolment rate, passage rate, facility utilization and referrals, etc. 5) Institutional and organizational issues: i. Supervision, reporting, performance assessment, auditing, rewards systems, procurement, citizens’ participation, information about services and pricing, etc. Among the key indicators that need to be collected are measures of access to services. These include the physical distance of the targeted population (e.g. walking distance to the health clinic or school), the variety of facilities available in the area and services offered, as well as opening and closing time of facilities. Measures of affordability of services are also important elements to capture, which include direct costs (such as user fees, transport costs, bribes and other informal payments) and indirect costs (opportunity costs, such as waiting time, etc.). Measures of quality of services are also essential, which could be proxied by physical and human capital, consultation duration, availability of material, etc. Furthermore, measures of the accountability relationship should be targeted. These could include various proxies such as supervision, reporting, auditing, performance assessment, rewards systems and feedback mechanisms. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 103 In addition, the Facility Questionnaire is generally supplemented by a Facility Data Sheet which is used to collect detailed quantitative information from the school or health unit records or accounts on various themes, in particular: 1) Staffing and salary structure; 2) Student enrolment or patient records for the last completed fiscal year; 3) Types of students or patients using the facility; 4) Service offered (class levels, health services such as vaccinations.); 5) Supplies available and usage (books, chalk, medication, vaccines, etc.). As previously mentioned, survey instruments, in particular facility data sheets, should be adapted and customized to the standard accounting and reporting procedure in practice at the facility level in the country in order to facilitate enumerators’ work and increase data quality. Regional Administration Module For countries with a health or education sector comprising a provincial or regional level of administration, a Regional Administration Questionnaire is generally administered to the Provincial Directorate of Health or Education. At such levels, information should be collected about roles played in the frontline provider supply chain and the monitoring process. Furthermore, quantitative annual data should be collected for the sector under study and specific programs tracked. The data collected at that level should cover areas such as: 1) allocation and execution of budgets; 2) distribution and management of drugs and other supplies; 3) human resources; 4) infrastructure and equipment; 5) monitoring and governance; 6) service outputs District Administration Module At the district or local government level, a District Administration Questionnaire is generally administered to the district director (district head doctor, etc). Again at that level, information should be collected about roles played in the frontline provider supply chain and monitoring process. Furthermore, quantitative annual data should be collected for the sector under study and specific programs tracked. The questionnaire should cover at least four areas: a) health infrastructure b) staff training c) support and supervision arrangements d) sources of financing A District Data Sheet is generally used to collect detailed information for the last completed fiscal year on: 1) Staffing and salary structures; 2) Basic and supplementary supplies (material, medication, vaccines, etc.); 3) Services and activities (students, outpatients, inpatients, immunization, deliveries, etc); 4) Financing (government budget, donor support, user fees) PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 104 Staff Module At the facility level, a Staff Questionnaire is generally administered to obtain information on the characteristics of the personnel as well as their incentive structure. The questionnaire is most often applied to a sample of teachers or health workers. It includes information on the following areas: 1) Characteristics: status, formation; 2) Incentive system: Salaries, allowances, supervision, performance assessment, performance-based promotions, sanction or rewards; 3) Perception of work environment. Client Module The perspective of users (students, patients, etc) sheds valuable light on problems of quality or efficiency in service delivery, given that users are the ultimate beneficiaries of services. Most surveys to date have included a patient exit poll or student survey. In general, at each of the school or health facilities surveyed, a number of users were interviewed. In the case of health facilities, outpatients were surveyed after having completed a visit. If the patient was a child, the caregiver would be interviewed. The number of users interviewed per facility sampled varied somewhat from survey to survey. In Mozambique for instance, approximately eight outpatients were interviewed (for a total of 679 individuals). The selection procedure was sophisticated (see section 5.2). In Uganda, an exit poll was used to interview about ten patients per facility (for a total of 1617 individuals). The Client Questionnaire generally covers the following main areas: 1) Client’s characteristics; 2) Reason for visiting the facility; 3)Access to the facility and other health care providers; 4) Services received, medication prescribed; 5) Costs of services (payments made the day of the visit and on previous occasions); 6) Exemptions; 7) Perception of quality: reception in clinics, consultation duration; 8) Informal payments/Corruption; 9) Alternative sources of services (schools, health services). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 105 6. GOOD PRACTICE PRINCIPLES There is a minimal level of methodological coherence among surveys that should be targeted in order to increase the quality and comparability of survey results. Through the various steps that characterize tracking surveys -- from team composition to findings dissemination-- this section proposes a series of good practice principles that arise from past experience, and discusses how they could be implemented. 1. Preliminary steps: Survey team qualifications, time frame and financing The preliminary steps for tracking survey work include proper core team qualification, adequate time frame and financial resources. A necessary element for core survey team composition is prior experience in such kinds of surveys, as well as adequate country- and sector-specific knowledge. Particular skills in sampling strategies and instrument design are necessary. Sufficient time and resources are necessary to plan, design and implement a survey. A full survey, from the planning stage to data analysis, involves a minimum of one year, and could take up to two. Adequate time should be further allotted for findings dissemination and policy reform discussions with recipient country government. 2. Planning/Consultation phase: Survey objective identification, government cooperation, involvement and ownership PETS-QSDS are information gathering tools for policy-makers. A first step in the survey is for the core team to establish broad-based consultation with country stakeholders. These include government ministries (finance, sector ministries, etc.), donors and civil society organizations. These consultations with stakeholders should lead to the identification of important issues relating to equity and efficiency problems in public service delivery that need to be addressed and to an agreement on the aim and objectives of the study. It should also be determined whether these country-specific issues are amenable to survey work. Among the objectives of PETS, the goal of providing evidence of leakage should generally be regarded as one of the main objectives. A necessary condition for success is to ensure that the government will be involved and committed to the project and reform. Country ownership and clear commitment to reforms are certainly key ingredients in successful survey work and service delivery reforms. Also, cooperation of public officials can be ensured only with government involvement. Furthermore, the sense of ownership in the study will make it more likely that recommendations will be translated into policy reforms. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 106 In that respect, a clear willingness on the part of the government to improve the situation is needed to ascertain the possibility of putting forward a useful and effective tracking survey. Indeed, the greater the rent-seeking benefits of entrenched power groups with close connections to the government, the weaker the willingness to implement reforms. On the contrary, the players will strive to secure the status quo that benefits the entrenched power base. 3. Analysis of institutional arrangements and policy environment: determination of specific survey objectives As the general objectives of tracking surveys is to assess resources used and leaked at the different levels of the service provider chain, an essential step in a successful tracking exercise is to analyze the institutional arrangements used for resource allocation in the social sector under study. This involves determining how the public hierarchy is structured, the roles and responsibilities of various administrative units, the allocation rules used by different resources at the various levels, and the nature of information flows including accounting, reporting and monitoring procedures. This institutional assessment phase also entails understanding the environment of public providers, the kind of schools or heath facilities operating in the country, and the mixture of public, private, religious and community facilities. In this phase, that the core team must spend sufficient time in the country to understand the sector and institutional arrangements under study. In most Sub- Saharan African countries, schools and health facilities do not receive a single monthly or annual financial allocation, but instead generally receive a mixture of financial and in-kind resources through several channels. Also, depending on the specific resources, procurement could be realized at various levels (central, provincial or district). Each of these mechanisms is typically governed by different administrative and recording procedures. Furthermore, staff salaries are often paid directly to the facility personnel by the central government or district levels. The institutional review should clarify these characteristics of resource flow, along with the role and responsibilities of each agent in the service delivery supply chain and accountability framework. 4. Research questions and choice of tracking flows Once the institutional arrangements are understood, the general survey objectives should be translated into specific objectives and research questions that will drive the data collection strategy. In addition, hypotheses about the research questions should be formulated in order to determine what kind of data should be collected. Furthermore, resources that will be specifically tracked in order to assess leakage and measure organizational performance have to be determined. PETS could PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 107 focus on tracking the whole range of government expenditures by sector or could focus on certain types of expenditures (for instance non-wage recurrent expenditures, salaries or capital expenditures). However, successful surveys have greatly restricted the domain of flows tracked in order to improve data collection quality. For instance, the Uganda 1996 education PETS collected financial information on only one expenditure program, capitation (per student) grants to schools. In the Zambian 2002 education PETS/QSDS, the tracking exercise was confined to non-wage cash transfers to schools. Adequate and systematic measurement of leakage should be developed and adapted to country circumstances. If tracking is done on an expenditure program for which a fixed allocation rule is in place, then leakage could be readily measured as the ratio of entitled funds that did not reach facility (assuming that reliable data could be collected about the funds reaching facilities). However, in the absence of a fixed allocation rule, where discretion is left to officials to determine the allocation to facilities according to needs or other considerations, the measurement of “narrow� leakage should be targeted. This consists in measuring the ratio of actual resources disbursed at a higher level and the resources received at the lower or facility level. In both cases, it is fundamental to identify the relevant data required to verify the hypothesis and to systematically measure leakage levels. Finally, facility data should be “triangulated� by collecting comparable data at the higher administrative levels, as well as from users and private providers’ associations, if applicable. 5-Rapid data assessment Following the identification of the data required to analyze service delivery performance, a rapid data assessment should be performed. The objective is to verify that the data required to test the hypothesis are available, and if not, to adapt the empirical strategy to the available data. At each level of the public hierarchy, as well as facility types and various locations (e.g. capital versus regions), data availability, quality and consistency should be verified. This small scale study should determine the survey’s feasibility and usefulness. A simple questionnaire administered at various administrative levels is usually sufficient for such a purpose. Data problems are frequent. Records could be badly kept, incomplete, unreliable and hard to understand. It is important to know what information is recorded in official records, the quality and availability of data, the aggregation level, periods available, consistency between levels, etc. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 108 The rapid data assessment seeks to determine the availability of records at various levels as well as among private providers, and avoid a costly and time consuming exercise of gathering inconsistent data. As Reinikka and Smith (2004, p. 53) emphasize, “Some studies have failed as the availability of records in local governments and schools were not adequately assessed beforehand�. For instance, if information is of poor quality at the local government level (region or district), this level could be bypassed and information could be collected only at the facility level (in order to measure resources available for service) and at the central ministry level (in order to know how much was officially sent). This rapid assessment of data availability should lead to redefinition of research questions and to the final choice of tracking flows in light of the available data. 6. Sampling strategy The sampling strategy is fundamental for tracking survey quality. While the sample strategy should be chosen to fit the study’s objectives, it should always be representative of the population under study. Most Sub-Saharan African countries do not have a reliable census of service providers. Consequently, an alternative is to create a sample frame from administrative records based on a list of schools or health facilities available from the central government or donor agencies. Based on this sample frame, the sample has to be randomly selected at every stage (regions, districts, facilities). If results are required for different categories of facilities (types, location, etc.), then the sample needs to be explicitly stratified. The sample size of each category must be sufficiently large to yield reliable results. The cluster approach (e.g. enumeration areas, wards) tends to reduce costs, but is generally not an efficient sampling method as the resulting sample is less well distributed than a random sampling strategy. If some of the flows tracked have no fixed allocation rules and the “narrow� leakage approach is followed (which requires measuring resources sent from a level vs. resources received), then a mixed sample-census strategy must be adopted. In such a situation, in each administrative level (for instance, district) chosen randomly, where public resources could be assessed, all facilities within the administrative level’s jurisdiction should be surveyed on a census basis, in order to collect complete information on the reception of resources. If private for-profit and non-profit providers are sufficiently active in the country under study and if the survey focuses on the sector as a whole (or on differences across facility types), then all types of facilities should be included in the sample strategy. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 109 In the health sector, hospitals should be surveyed if the analysis focuses on tracking the whole health budget, for instance, but hospitals should not be included if the focus is on the efficiency of local health centers. If more than one sector is part of the tracking exercise, separate samples have to be drawn for each sector in order for the samples to be representative of the facility population of each sector. As much as possible, linkage between facility survey data and household data should be targeted in order to allow for the measurement of final outcome at the population level and to link service and population outcomes. PETS/QSDS could either comprise a household survey module, or alternatively the team could establish linkages with a self-standing household survey near implementation in the country, if applicable. However, although a choice must be made between the facility sample’s being representative of the population of individuals or of facilities, the sample strategy should be as representative as possible of the facility population. Finally, care should be taken to use a common coding for facilities in both surveys and to assign the proper weights to the sample. 7- Instrument design The design of survey instruments depends on survey objectives and the choice of tracking flows. In order to allow for comparability across countries, it is important to harmonize the structure and content of questionnaires. Exit polls and staff questionnaires could be relatively standardized. However, output activities and inputs are harder to standardize due to variability in accounting systems. A list of service delivery indicators should be compiled in each sector, which would allow statistical measurement. These indicators should measure the evolution and monitoring of progress in time series analysis. Based on this common structure and indicators, questionnaires have to be customized to the country’s context. In particular, quantitative data questions (e.g. financial, input and output data), should be adapted to the country’s administrative system, that is the categories used in the administrative reporting structure. This strategy would maximize the probability of having data reported and collected by enumerators. As previously mentioned, it is important to collect administrative data at the central level. These annual data, both for the entire sector under study and specific programs tracked, should be disaggregated at the provincial, district, and potentially service provider level if available. Such data would allow measurement of resources officially allocated to decentralized levels. It may also make it possible to triangularize information provided at lower levels. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 110 In the past, surveys have collected information that is not used. Parsimony of data collected is recommended to reduce costs, but also to increase the quality of data collected. The questionnaire should be focused, and contain a reduced number of questions. Data collection should ideally involve annual data and cover a period of a maximum of two financial years (one financial year is probably even more adequate) to maximize data collection quality. If monthly data are collected, seasonality issues need to be examined beforehand. With respect to in-kind items (e.g. school supplies, drugs and medical and non- medical materials), a sample approach should ideally be used instead of a census approach when the number of different items is too large. A limited number of items, for instance the top 10 most frequently used medications, should be targeted. High frequency items should be tracked instead of high value items. The list of items needs to be country specific. The choice of materials (or medications) should be based on their shipment frequency in the Ministry invoice list (if available). With such an approach, the risk inherent in choosing a rare but high- value material (e.g. a car) is not finding that material in the visited service provider simply because not all of them were able to receive it. In contrast, by choosing frequently shipped materials of small value (e.g. Windex), it is likely that a maximum number of facilities would report receiving them. This would give an upwardly biased percentage of facilities receiving materials from the authorities. Valuation of in-kind items should be done using a standardized price list (such as the line ministry list, if available). Resources received should be estimated relative to the share of the sample items in the global budget allocation. Survey instruments should be circulated among stakeholders for comments. 8- Preliminary pilot phase: All survey instruments (including regional or district offices, facilities, staff, users, households, etc) should be tested through a small scale pilot phase on a specific number of units and ownership types or regions. The choice of tracking flows should be assessed, as well as the quality and consistency of data. In particular, quantitative data questions, financial data, inputs and outputs, which have been customized to the country’s administrative system, should be carefully examined. To minimize measurement errors, government and facility records should be used as much as possible. Instruments should be revised to account for such considerations (recall versus records). Furthermore, questions’ wording, ambiguous responses, answer codes, etc. should be revised at this stage. This exercise should also be accompanied by the verification and revision of the sampling procedure. Following the preliminary pilot, revised survey questionnaires should be circulated anew among stakeholders for comments. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 111 9- Choice of consultant team Procurement issues associated with survey implementation, in particular the choice of local consultants, are important considerations. Local consultants are likely to be more cost effective to conduct the PETS-QSDS and contribute in local capacity building. However, the choice of local consultants could be challenging and time consuming. Bank procurement rules could require relatively long procedures, involving, for instance, public tendering rules. It is crucial that the selected local consultant team implementing the survey be independent, absent of conflict of interest and of high quality. This has tremendous impact on data quality, as difficulties of implementing the survey are generally very important, and potential conflict of interest could be present. PETS should be carried out by civil society organizations. In some countries, including Cameroon and Sierra Leone, PETS have been carried out by government officers in various ministries. This approach does not guarantee the independence of survey teams. 10- Training A minimum of one week and possibly two are required for enumerator and supervisors training. The training by the survey core team involves acquainting enumerators and supervisors with the instruments and techniques used in data collection. Training should include the testing of instruments by agents in the filed. Enumerators or supervisors should have access to a survey manual detailing questions’ objectives and interpretation. Following the training, questionnaires may be revised. 11- Full pilot phase Following (or during) the training, all questionnaires should be tested. Field testing of instruments is crucial for increasing the likelihood of obtaining good quality survey information. A test on about 5% of the sample including all types of respondents should be adequate. Following the pilot, a final revision of instruments should be carried out. This includes revising wording of questions, ambiguous responses, format of the questionnaires and answer codes. Common approaches to data collection have to be determined in cases of ambiguities. 12- Survey implementation: Monitoring by the core team Once all the previous conditions are met, the full scale survey could be conducted. The survey should be fielded ideally two to three months after the end of the fiscal year (for books to be closed) and tracking should cover the last completed PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 112 fiscal year. Some members of the core survey team have to be in the country for the whole survey and data entry period. Fieldwork must be closely monitored. It is crucial that the core survey team closely supervise survey implementing teams in the field. Random visits to enumerators in the field are essential to ensure quality control and coherence in the interpretation of questionnaires. In particular, random checks of questionnaires and data quality should be done throughout the survey implementation. 13- Data entry Data entry programs should be written following the completion of the questionnaires and should be tested during the survey pilot phase. Data entry should start during survey implementation and should be completed promptly following data collection. A standard state of the art data management program, such as CSPRO (or CSPRO X), should be used. A common program would allow economies of scale in training and building of data entry programs and should lead to an improvement in data quality and comparability across country data sets. Inconsistencies and potential errors in data detected should be verified while the survey is still being fielded. Data entry programs should include a unique identification code for each questionnaire and unit interviewed in order to match responses within districts and regions. Various controls should be introduced in the data entry program in order to reduce data entry errors, as well as validation mechanisms to detect data inconsistencies. 14-Data analysis and report writing Data cleaning and analysis should be done shortly after the end of data collection. Final versions of reports should be produced promptly. At minimum, reports’ contents should include background information on the state of service delivery in the sector, descriptive and analytical information on various units analyzed in the survey work (local governments, facilities, staff, clients, etc.), evaluation of efficiency and equity in service delivery, funding disbursed at the central level and received at the decentralized level, etc. Reports should clearly identify and communicate the specific findings, the measures of leakage at each level and recommendations to be implemented. The reports should be discussed with stakeholders and adequately edited. Furthermore, the reports should be distributed among civil society. 15- Result dissemination and policy reforms PETS results should be disseminated promptly among government ministries and units, as well as NGOs and civil society organizations, following the report PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 113 completion, to increase impacts in terms service delivery quality and efficiency and population outcomes. Information collected should be analyzed and used for planning and reform processes. Among the service delivery reforms that should be considered, the display of expenditure information at the facility level should be targeted following the successful use of such approach in Uganda. Various initiatives that could enhance clients’ power should be proposed and implemented. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 114 7. PROPOSAL FOR FUTURE SURVEYS This section discusses potential surveys and other endeavors. Among the possibilities for future works are baseline surveys, follow-ups, impact evaluations, new experiments, report cards and vignettes. 7.1 General Considerations for Future Tracking Surveys83 Two main objectives could be targeted in future survey work on service delivery: a) to cover sectors that few past tracking surveys have analyzed, such as water supply, sanitation and agricultural services. b) to do baseline surveys, follow-up surveys, impact evaluations of various interventions, such as citizen report cards, and vignettes. Among the research questions that could be explored are: i) Benchmarking leakage of resources and better explaining capture of funds (including delays etc.). Robust baseline surveys of the sector studied should be devised to diagnose and analyze the process by which public resources are translated into services and identify shortcomings in the supply chain and among frontline providers, in particular leakage of resources. Factors explaining leakage that could be examined notably concern users’ bargaining power, service provider remoteness, size of facilities, access to infrastructure, incentives, etc. Developing a core set of questions for PETS to be applied across countries should be a priority. ii) Benchmarking absence rates and better explaining provider behavior. Service provider absenteeism has been found to be a widespread problem in a number of countries. It is also an effective way to get policymakers’ attention to performance in service delivery. It would be worthwhile to document the extent of the problem in SSA countries and identify factors that are associated with worker absence. Baseline surveys could seek to measure absenteeism in various facility types, ownership categories, locations, etc.; to measure time on task for different types of workers and to try to better explain reasons for absence rates.. For instance, in Zambia Das et al. (2004) found that teacher absenteeism had more to do with health problems than with shirking. Again, developing a set of core questions to be applied across countries is a priority. iii) Evaluating interventions to combat leakage, absenteeism and other service delivery failures with a focus on human development and other sectors. 83 This section borrows in part from Gautier and Reinikka (2007) PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 115 Impact evaluation of pilots or reform programs such as information campaigns, citizen report cards, monitoring mechanisms and incentive schemes for service providers need to be carried out. iv) Link demand and supply sides of service delivery to develop a better understanding of development outcomes. Future research should target linkages between facility survey data and household data in order to allow measurement of final outcomes at the household level and to link service and human development and other outcomes. PETS/QSDS, for instance, can comprise a household survey module. Alternatively, linkages with another self-standing household survey can be established.84 Various other interventions related to weaknesses in the delivery chain could also be studied, including impact evaluation. Among topics themes that could be examined in these future projects are: efficiency and productivity analysis; user fees, tuition and non-tuition fees; drug use and over-prescription; black market for medication; linkages with traditional/informal sector; household demand for health care and education; substitution effects between private and public expenditure. In the past, the selection process of tracking surveys has predominantly taken two routes: a) it has involved the Bank and other donors (e.g. Chad or Nigeria); and b) it has been (at least partly) initiated by the country, which has taken ownership of the project (e.g. Uganda or Sierra Leone). For future tracking surveys and other endeavors on service delivery, the identification and selection process should probably comprise the following steps. 1. In the various countries, the initial challenge would be to identify important issues relating to equity and efficiency problems in public service delivery. Discussions with Bank teams and country governments should lead to the identification of these issues and determine whether these country-specific issues are amenable to survey work. At this stage, a necessary condition for success is, as mentioned, to ensure that the government will be involved and committed to the project and reform. Country ownership and clear commitment to reforms are certainly key ingredients in successful service delivery survey work and reforms. 2. Once the intervention is identified, a small scale study should be conducted to examine the survey’s usefulness and feasibility. 3. Once these conditions are met, the full scale project could go ahead. 84 Although a choice must be made between the facility sample’s being representative of the population of individuals or of facilities, the sample strategy should be as representative as possible of the facility population. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 116 It should also be noted that tracking surveys should be perceived as part of a larger set of tools that could be used to improve service delivery and public expenditure performance. Others instruments include internal and external audits assessment of financial management procedures, Citizen report cards, health vignettes, etc. 7.2 Potential Countries and Projects With these considerations about the identification and selection of future surveys in mind, we review some issues that could be involved in considering future interventions in some countries where surveys have been implemented in the past and that would benefit from follow up interventions. Nigeria: One of the key service delivery problem identified in the previous 2002 QSDS was the non-arrival of salaries in one of the two state surveyed (Kogi). Given the existence of a baseline, a randomized evaluation could be carried on such a question. This would involve identifying a randomized group on which reform in the salary supply chain would be put forward (for instance some districts in one state), as well as a control group without such reform (other districts). In order to avoid selection problems, and following the Heckman principle, “difference of difference� would be taken to avoid biases.85 Cameroun: A good baseline exists in Cameroon in the health and education sectors. Data are of good quality and exist for a large sample of facilities. Household appreciation of health centers and primary and secondary schools was also collected. Currently, service delivery in these social sectors face problems associated with what could be called a situation of “false decentralization�. Accordingly, budgets for the decentralized administrations are managed at the central level (as in Chad for instance, with so called “centralized credits�). In such contexts, public funding could easily be diverted and captured. An interesting future initiative could involve devising a decentralization reform in some regions and conducting a randomized impact evaluation on such reform. Chad: A good baseline exists in the health sector. However, despite the potential for reforms to be implemented in the country, the important current political problems are such that it would be very difficult to think of a follow up in the health sector, or of establishing a baseline in education Ghana: A baseline exists, but only as a PETS. A larger baseline could be targeted as a PETS/QSDS, with follow-ups on certain key delivery failures. There may be issues with respect to the representative nature of the baseline sample that would need to be examined. Madagascar: The data from the different PETS /QSDS conducted in the country have not been fully exploited yet. Among questions that could be examined in 85 Indeed, without such difference between groups from the baseline, maybe other things would be captured if only a comparison between a randomized group versus a control group is done (as other non-observable factors influence the before-after difference). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 117 future survey work are issues of leakage related to remoteness, size of schools and access to infrastructure, as well as issues of user fees or decentralization. Mozambique: A baseline exists. Interesting future work could involve random impact evaluation on, for instance, information system reform at various levels in the administrative system. Rwanda: The focus of the previous education PETS in 2000 on the tracking of specific funding programs was pertinent. However, the sample design used presented some limitations. Data on the programs’ client satisfaction were collected by asking opinions only from people who benefited from the program (bursary, etc). The sample was thus clearly biased. Furthermore, no QSDS was implemented. Hence, no adequate baseline currently exists. This could be the objective of a future endeavor. Senegal: The sample selection in the 2005 PETS-QSDS in Senegal presented some problems. Enumerators visited schools and health centers in the same enumeration areas, which is clearly a problem as the sample is necessarily not representative of one of the two populations. (It could be observed that this joint enumeration areas strategy could have been adequate if a census approach had been realized within, for instance, a sample of districts). In a future PETS-QSDS in one of the two sectors, a new baseline could then be established. Furthermore, one of the main problems observed in the health and education sector in Senegal was delays in the reception of resources at the local government and provider levels. A randomized impact evaluation could, for instance, be realized to assess a reform in the decision making process in some regions. Tanzania: In both past PETS in Tanzania, samples were too small to be representative of the school and health center population and, as such, could not serve as a baseline. Furthermore, no QSDS was realized. It may be worth doing a combined PETS/QSDS to establish a baseline which would seek to increase transparency and resource allocation at the local provider level. Also, the water sector which has been examined in the 2001 PETS and for which basic information has been collected could be surveyed anew, in order to develop a baseline. Zambia: Zambia possesses a very good baseline in the education sector. There is clear potential for further quality work in this country on various issues, using different instruments. In particular, policy interventions that could enhance teacher presence in the school should be considered. Two options that could be evaluated are, for instance, to allow the possibility of substitute teachers, or various incentives to reduce absenteeism. It is crucial to increase the actual time teaching (which in many respects in Zambia has more to do with the health of teachers rather than with shirking). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 118 7.3 Other Potential Projects and Tools Decentralization is another important policy to evaluate. In several countries, decentralization has been implemented and needs to be evaluated. In some countries (e.g. Cameroon, Chad), service delivery in social sectors encounters problems associated with what could be called a situation of “false decentralization.� Accordingly, budgets for the decentralized administrations are still managed at the central level (e.g. “centralized credits� in Chad). In such contexts, public funding can be diverted and captured more easily. An interesting experiment could involve devising a decentralization reform in some regions and conducting an impact evaluation on this reform. Future research could also focus on leakage of drugs. In a number of African countries, the poor availability of drugs has been directly attributed to problems of corruption in the distribution of medicines (see for instance, Van der Geest, 1987, on Cameroon). Nevertheless, there is almost no systematic evidence on the extent of leakage or the implications that such leakage has for health outcomes in the African context. The importance is magnified in the context of the AIDS crisis. Given the high value of AIDS medicines, existing problems of distribution will be exacerbated, underscoring the need to understand the link between leakage and welfare (Das, Gauthier and Philipson, 2005). Future research could focus on developing a theoretical and empirical framework for examining the economic and health impact of such leakage of resources on outcomes. Future theoretical and empirical research could also examine various organizational and institutional arrangements in order to develop a more realistic explanation of relative organizational efficiency. A better understanding of the effects of specific institutional arrangements on service provider behavior and outcomes could help determine who stands to gain or lose from specific institutional arrangements, who exerts influence, how this influence is exercised, and how efficiency and equity are affected. Vignettes: Future empirical research could also include qualitative tools such as vignettes. These tools, through unblind standardized case studies involving an actor, test the level of knowledge and competence of service providers and have clear potential for service delivery improvements. Leonard and Masatu (2005) have examined quality in outpatient services in Tanzania using direct clinician observation as well as vignettes, and recommend a combined approach to evaluate health service quality. Das and Hammer (2004) observed that there is a disconnect between practitioner knowledge and services rendered by the doctor or clinician. The practitioner’s incentives for behavior may come from elsewhere (market, case load, specific patient facing the doctor, etc). These issues could be further explored in new stand-alone vignette projects, or combined with QSDS, to analyze service quality issues. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 119 Citizen report cards: Finally, dissemination tools like Citizen Report Cards, have proven to be powerful tools for identifying bottlenecks, inefficiencies and waste in service delivery, especially problems of resource leakage, delays, absenteeism, and inequities in the actual resource allocation. Report cards entail collecting periodic information from users about their perception of service quality. The cards are then used to generate score cards which are used to identify weaknesses in service delivery. Scores are subsequently disseminated among the population, including to service providers. The impact on services is then assessed. Interesting experiments with this new tool have been developed in Uganda (Reinikka and Svensson, 2004c). In the Ugandan education sector, for example, Reinikka and Svensson (2005, 2009) use a PETS approach to study the effects of a public information campaign aimed at empowering parents. They find a large reduction in resource leakage toward primary schools in response to the campaign. The introduction of a Citizen Report Card for Bangalore’s public agencies led to major improvements in public satisfaction and the agencies’ ratings. Similarly, corruption in the transactions between users and public officials declined markedly (Samuel, 2002). A randomized evaluation of a citizen report card intervention in the health sector in Uganda in 2004-2005 concluded that the CRC led to significant improvements in the quality and quantity of primary health care provision and resulted in improved health outcomes in the communities (Björkman and Svensson, 2009). The use of this new tool could be tested in other countries and other random impact evaluation realized. Furthermore, It should be noted that an element not fully examined at this stage with respect to this new tool is the relationship between public and private information. As emphasized by Hellwig (2002) and Morris and Shin (2002), individuals tend to form beliefs and rank services and providers. Public information will affect individual beliefs, but in a complex manner (e.g., through Bayesian updating) that would be worth investigating. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 120 8. CONCLUSION PETS and QSDS have been powerful instruments for gathering information on the service delivery chain and local service providers in order to evaluate the efficiency and equity of government expenditures. In the last ten years, tracking surveys have shed light on the process whereby public expenditures translate into service provision at the local level. By identifying resources available at various levels of the supply chain and examining behavior and incentives of various agents within institutional arrangements, they have identified problems of governance, capture of funds and corruption. In several countries’ social sectors, the surveys suggested areas where reforms could be effectively implemented. Furthermore, they have been instrumental in bringing about effective changes in resource allocation, notably through an information campaign in the education sector in Uganda. This study has summarized some of the main results of surveys implemented in Sub-Saharan Africa and examined methodological elements that could affect survey results and performance. As discussed, important harmonization issues remain to be resolved. A sound methodological basis is required for tracking survey implementation, in particular with respect to the use of representative samples, proper preliminary institutional and data assessment leading to an adequate choice of tracking flows and the use of efficient and relatively common instruments. There are new approaches (e.g. impact evaluation, report cards, vignettes.) and important policy questions to examine and reforms to evaluate in various countries where tracking surveys could be useful. However, beyond these methodological issues and new applications, the biggest challenge for tracking surveys is probably to translate their important findings and contributions into policy reforms and institutional changes in order to improve service quality and population outcomes in Sub-Saharan African countries. Indeed, tracking surveys are a means to an end. The information on incentives and deficiencies in organizational structures and rules should ultimately be used to identify policy reforms and help implement a reform agenda in client countries. Table 8 presents some of the policy reforms brought about so far by PETS/QSDS. To date, one of the main achievements in service delivery improvements that have come out of PETS-QSDS has been observed in the education sector in Uganda. The policy reform introduced targeted information flows in order to give clients potential power over service providers. Through an information campaign in newspapers and on radio that targeted school associations, parents and the community, clients were better informed and given the ability to voice their demands. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 121 TABLE 8 INTERVENTIONS AND REFORMS Survey Country Year Type Sectors Interventions Objective Reported effects Implications Uganda 1996 PETS Education -Central government attempted - To make information - Flow of funds improved and Health to remedy the situation by available to citizens and dramatically from 13% of publishing monthly PTAs intended capitation intergovernmental transfers of - To signal to local grants reaching schools public funds in the main governments that the central in 1991-95 to over 80% newspapers and on the radio, government had resumed its in 1999 and 2000 and required schools to post oversight role - Delays in transfers are information on inflow of funds. still considerable. 1999 PETS Education -Results suggest that and efforts to increase 2000 citizens’ ability to monitor the system, and to inform them about their rights and entitlements, are important aspect in controlling corruption 2000 QSDS Health Tanzania 1999 PETS Education and Health 2001 PETS Education -Treasury initiated dissemination -Early indications are and Health of itemized local government that the campaign has budget to members of reduced leakage Parliament and regular considerably publication of budget allocation to various levels of government in newspapers Ghana 2000 PETS Education -PETS opened an avenue for -Survey results have not and Health inter-ministerial collaboration produced a strong and provided a practical response to reduce approach to assess service leakage and rent delivery capture, either through innovations in transparencies or increased oversights at various government levels. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 122 Survey Country Year Type Sectors Interventions Objective Reported effects Implications Rwanda 2000 PETS Health 2004 PETS Education (Also realized in health, water and agriculture projects) Sierra Leone 2000 PETS Education, -PETS has led to modifications and Health and in the MOF procedures, 2001 various including tracking of other expenditures to regional sectors administrative levels Zambia 2001 PETS Education QDSD Mozambique 2002 PETS Health QSDS Nigeria 2002 PETS Health QSDS Senegal 2002 PETS Health Cameroon 2003 PETS Education and Health Madagascar 2003 PETS Education QDSD 2005 PETS Health QDSD Namibia 2003 PETS Education QSDS and Health Chad 2004 PETS Health QSDS Kenya 2004 Health and education Sources: Survey reports. Uganda, Tanzania: Reinikka and Svensson (2004a). PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 123 The results have been quite spectacular, in that the leakage rate was reduced dramatically. As noted by Reinikka and Svensson (2004, p.23) similar information problems exist in other countries, making the information campaign approach adopted in Uganda potentially widely applicable. Such an information campaign approach to reducing corruption has also been used in Tanzania. Following the identification in two PETS of fund diversion at the local government level, the government started publishing information in newspapers covering allocations for ministries, regions and local authorities (councils) of budget allocations for the selected pro-poor spending programs. Positive impacts of the campaign have been reported. While a formal impact evaluation has not been conducted, reports indicate that the information campaign has reduced leakage (World Bank, 2003, p.14). In addition to information campaigns, several other potential reforms are on the agenda following PETS/QSDS. The areas identified for reforms in various survey reports include:  Improving internal controls, in particular accounting and reporting systems in order to enhance transparency and accountability (basic accounting systems are often not in place and rules not followed);  Speeding up budget execution at various levels of the delivery chain;  Increasing inspection and monitoring at all level of the service delivery system;  Improving communication and information pass-through (dissemination of information is a general problem between the administrative levels, including with the population);  Providing additional training for decentralized government levels and service delivery units (problems often result from a poorly trained personnel);  Establishing mechanisms and incentives in the system to make the service delivery system more client driven at all levels. While it cannot be denied that a good number of tracking surveys have been very successful at identifying weak links in the service supply chain, as well as areas where reforms should be put forward, in practice, however, few countries have effectively followed up on the diagnosis made by PETS/QSDS and implemented decisive reforms in service delivery. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 124 In several countries, lack of political will to put reforms in practice has certainly been a factor behind weak institutional change. Lack of policy dialogue, insufficient dissemination of results and discussions to ensure the transfer of information about problems identified in the service delivery system are also noteworthy. More emphasis on client outcome would certainly contribute to promoting institutional reforms in recipient countries. This would probably entail reform of incentives on the project supply side. Success in project evaluation should therefore be contingent not only on the survey implementation itself, but more importantly on the capacity to translate and bring about policy and institutional reforms in the client country. PETS and QSDS seek to improve the efficiency and equity of public service provision in Africa. As such, the success of the exercise should be measured by its capacity to bring about improvements in the quality of services at the population level. That is, in its capacity to lead to policy dialogue, policy reforms in areas of weaknesses identified and improvement in the outcome at the client/population level. Ultimately, in addition to devising robust methodologies to collect information and detect corruption in public service delivery, the real challenge and the real yardstick on which PETS/QSDS should be measured is the capacity to induce policy reforms to correct the various governance problems identified. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 125 References Ablo, Emmanuel and Ritva Reinikka (1998) “Do Budgets Really Matter? Evidence from Public Spending on Education and Health in Uganda� Policy Research Working Paper 1926, The World Bank, Washington D.C. Anti Corruption Coalition Uganda (2009) “Public Expenditure Tracking Survey in the Water Sector�, June. Anderson, James, Daniel Kaufmann and Francesca Recanatini (2003) “Service Delivery, Poverty and Corruption: Common Treads from Diagnostic Surveys�, Background paper for the World Development Report 2004, The World Bank, Washington D.C. Besley, Timothy and R. Burgess (2003) “The Political Economy of Government Responsiveness: Theory and Evidence from India�, Quarterly Journal of Economics, 117(4): 1415-51 Besley, Timothy and Maitreesh Ghatak (2003) “Incentives, Choice and Accountability in the Provision of Public services� Oxford Review of Economic Policy, 19 (2): 235-249 Bjorkman, Martina and Madestam, Andreas (2003) "Public Expenditure Tracking Survey: Primary School Pilot", report to the World Bank, Stockholm. Chaudhury, N. and J.S. Hammer (2004) “Ghost Doctors: Absenteeism in Rural Bengladeshi Health Facilities�, The World Bank Economic Review, 18 (3): 423-441. Chaudhury, N., J. S. Hammer, M. Kremer, K. Muralidharan and F. Halsey Rogers (2006) “Missing in Action: Teacher and Health Worker Absence in Developing Countries�, Journal of Economic Perspectives, 20 (1): 91-116 Das, Jishnu (2004) “Equity in Educational Expenditures: Can Government Subsidies Help?� Policy Research Working Paper 3249, The World Bank, Washington, DC: World Bank Das, Jishnu, Stefan Dercon, James Habyarimana and Pramila Krishnan (2004a) « Public and Private Funding Basic Education in Zambia: Implications of Budgetary Allocations for Service Delivery� Africa Region Human Development Working Paper Series No. 62, The World Bank, Washington, DC. Das, Jishnu, Stefan Dercon, James Habyarimana and Pramila Krishnan (2004b) “When Can School Inputs Improve Test Scores�, Policy Research Working Paper 3217, The World Bank, Washington, DC. Das, Jishnu, Stefan Dercon, James Habyarimana and Pramila Krishnan (2005) “Teacher Shocks and Student Learning: Evidence from Zambia� Policy PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 126 Research Working Paper 3602, The World Bank, Washington, DC: World Bank Das, Jishnu and Jeffrey Hammer (2005) “Which Doctor? Combining vignettes and item response to measure clinical competence�, Journal of Development Economics, 78: 348-383. Das Gupta, M, V. Gauri and S. Khemani (2003) “Primary Health Care in Nigeria: Decentralized Service Delivery in the States of Lagos and Kogi.� Africa Region Human Development Working Paper Series No. 70, The World Bank, Washington, DC. September. Dehn, Jan, Ritva Reinikka and Jakob Svensson (2003) “Survey Tools for Assessing Performance in Service Delivery.� In Francois Bourguignon and Luiz Pereira da Silva (eds.). The Impact of Economic Policies on Poverty and Income Distribution: Evaluation Techniques and Tools. A copublication of the World Bank and Oxford University Press. Engberg-Pedersen, P., K. Kaiser, G. Kisunko, I. Kushnarova and M. Lindelow (2005) Public Expenditure Tracking Surveys: A Stocktaking, Mimeo, Washington, DC: The World Bank.. Filmer, Deon and Lant H. Pritchett (1999) “The impact of Public Spending on Health: Does Money Matter?� Social Science and Medicine, 58: 247-258 Filmer, Deon (2003) « The Incidence of Public Expenditures on Health and Education » Background Note for the World Development Report 2004:, Washington D.C., May Filmer, Deon; Jeffrey S. Hammer and Lant H. Pritchett (2000) “Weak Links in the Chain: A Diagnosis of Health Policy in Poor Countries�, World Bank Research Observer, 15 (2): 199-224. --------- (2002) “Weak Links in the Chain II: A Prescription for Health Policy in Poor Countries�, World Bank Research Observer, 17 (1): 47-66. Francken, Nathalie (2003) Service Delivery in Public Primary Schools in Madagascar: Results of a Budget Tracking Survey, The World Bank, Madagascar Country Office, Antananarivo, September. Gauthier, Bernard, Mark Gersovitz, Tomas Philpson and Waly Wane (2004) « Survey Instruments for the PETS/QSDS in the Health Sector in Chad�, The World bank, Washington DC., May, Mimeo. Gauthier, Bernard and Waly Wane (2005) “Suivi des dépenses publiques à destination dans le secteur santé au Tchad: Analyse des résultats d’enquête," Development Research Group, The World Bank, Mimeo. ------------------ (2009) “Leakage of Public Resources in the Health Sector: An Empirical Investigation of Chad�, Journal of African Economies, 18 (1): 52. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 127 Gauthier, Bernard and Ritva Reinikka (2007) “Methodological Approaches to the Study of Institutions and Service Delivery: A Review of PETS, QSDS and CRCS�, AERC Core Paper, The World Bank, September, processed. Government of Uganda (2004) “Access to Education and Health Care in Uganda, processed. Hellwig, Christian (2002) “Public Information, Private Information and the Multiciplicity of Equilibria in Coordination Games�, Journal of Economic Theory, 107 (2): 191-222. Hutchinson, Paul (2001) “Cambatting illness.� In Ritva Reinikka and Paul Collier (Eds.) Uganda Recovery: The Role of Farms, Firms and Government. Regional and Sectoral Studies. Washington DC.: World Bank Keefer, Philip and Stuti Khemani (2005) “Democracy, Public Expenditures and the Poor: Understanding Political Incentives for Providing Public Services", World Bank Research Observer, 20 (1): 1-27 Khemani, Stuti (2006) “Local Government Accountability for Health Service Delivery in Nigeria�, Journal of African Economies, 15: 285-312. Leonard, Kenneth L. and Melkiory C. Masatu (2005) “The use of direct clinician observation and vignettes for health service quality evaluation in developing countries� Social Science & Medicine, 61: 1944-1951 Lewis, Maureen (2006) “Governance and Corruption in Public Health Systems� Working Paper 78, Center for Global Development, Washington D.C., January Lindelow, Magnus and Adam Wagstaff (2003) “Health Facility Surveys: An Introduction� Policy Research Working Paper 2953, The World Bank, Washington DC. Lindelow, Magnus (2004), “Health Care Decisions as a Family Matter: Intrahousehold Education Externalities and the Utilization of Health Services.� Policy Research Working Paper 3324. Development Research Group, The World Bank,Washington, D.C. Lindelow, Magnus (2006) “Tracking Public Money in the Health Sector in Mozambique: Conceptual and Practical Challenges�, East Asia Human Development Unit, Africa Region Human Development Working Paper 29097, The World Bank, April. Lindelow, Magnus; Inna Kushnarova and Kai Kaiser (2006) “Measuring corruption in the health sector: what we can learn from public expenditure tracking and service delivery surveys in developing countries�, In J. Kotalik and D. Rodriguez (Eds.) Global Corruption Report 2006: Corruption and Health, London: Pluto Press. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 128 Lindelow, Magnus, Ritva Reinikka and Jakob Svensson (2003) “Health Care on the Frontlines: Survey Evidence on Public and Private Providers in Uganda� Africa Region Human Development Working Paper Series, The World Bank, Washington DC. Lindelow, Magnus and Adam Wagstaff (2003) Health Facility Surveys: An Introduction, Policy Research Working Paper 2953, The World Bank, Washington DC. Lindelow, Magnus, Patrick Ward and Nathalie Zorzi (2004), “Primary Health Care in Mozambique: Service Delivery in a Complex Hierarchy .� Africa Region Human Development Working Paper 69, Development Research Group, The World Bank, Washington, D.C. Manshande, Jean-Pierre and Iona Zamfir (2005), L’Efficience et l’Equité des Formations Sanitaires Malgaches: Résultats d’une enquête� Africa Region Human Development Working Paper No. 76. The World Bank, Washington DC. Ministère de la Santé publique (2001) « Revue des dépenses publique du secteur santé », Direction générale, Division des ressources financières, République du Tchad, processed. Minten, Bart, Mead Over, Mamisoa Razakamanantsoa and Waly Wane (2004). “The potential of community based insurance towards health finance in Madagascar�, June, processed. Morris, Stephen and H. S. Shin (2002)�The Social Value of Public Information�, American Economic Review, 92 (5): 1521-1534. Msuya, Joyce (2004) “Horizontal and Vertical Delivery of Health Services: What Are the Trade Offs?� Background paper for the World Development Report 2004, The World Bank, Washington D.C., processed. Mukwena, R.M.; Z.O Bolo and J.E. Odada (2003) A Public Expenditure Tracking Survey : Ministries of Health and Social Services and Basic Education, Sport and Culture, The Namibian Economic Policy Research Unit and Office of the Auditor General, May, processed. Musgrove, Philip (1996) « Public and Private Roles in Health: Theory and financing patterns » World Bank Discussion Paper 339, Washington D.C. World Bank, 2008 “Niger Public Expenditure Tracking Survey� Education and Health (In Two Volumes) Volume I: Main Report, December 21. Over, Mead, Mattias Lundberg, Waly Wane and Sergio Bautista (2006) Estimating and explaining the efficiency of health service delivery in developing countries: with applications to HIV prevention in five countries and primary care in Chad, Presentation at the workshop on measuring service delivery, World Bank, April 28, processed. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 129 Picazo, Oscar F. and Zhao, Feng (2009) “Zambia Health Sector Public Expenditure Review: Accounting for Resources to Improve Effective Service Coverage�, World Bank Country Study N. 47110 Rajkumar, Andrew Sunil and Vinaya Swaroop (2002) Public Spending and Outcomes: Does Governance Matter? Policy Research Working Paper 2840, The World Bank, Washington, DC. REPOA, (2004) “Tanzania Public Expenditure Tracking Study of the Financial and Non-financial PEDP flows from Central government to schools in 2002 and 2003� Reinikka, Ritva (2001) “Recovery in service delivery: evidence from schools and health centers� In R. Reinikka and P. Collier (Eds), Uganda’s recovery: The role of farms, firms and government. World Bank Regional and Sectoral Studies, Washington DC: World Bank. Reinikka, Ritva and Nathanael Smith (2004) Public Expenditure Tracking Surveys in Education, International Institute for Educational Planning, UNESCO, Paris. Reinikka, Ritva and Jakob Svensson (2001) “Explaining Leakage of Public Funds� Policy Research Working Paper 2709. The World Bank, Washington, D.C. ------------ (2003a) “Working for God? Evaluating Service Delivery of Religious Not-for-Profit Health Care Providers in Uganda.� Policy Research Working Paper 3058. World Bank, Development Research Group, Washington, D.C. ------------ (2003b) “Survey Techniques to Measure and Explain Corruption.� Policy Research Working Paper 3071. The World Bank, Washington, D.C., ------------ (2003c) “Measuring and Understanding Corruption at the Micro Level.� In Donatella Della Porta and Susan Rose-Ackerman (eds.), Corrupt Exchanges: Empirical Themes in the Politics and the Political Economy of Corruption. Baden-Baden: Nomos Verlasgesellschaft., ------------ (2004a) Efficiency of Public Spending: New Microeconomic Tools to Assess Service Delivery.� In Tony Addison and Alan Roe (eds.). Fiscal Policy for Development: Poverty, Reconstruction and Growth. Palgrave Macmillan. May ------------ (2004b) “Local Capture: Evidence from a Central Government Transfer Program in Uganda�, Quarterly Journal of Economics, 119 (2): 1-28. ------------ (2004c) “Citizen report card at the community level: an impact evaluation� DECRG, The World Bank, processed PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 130 ----------- (2005a), “Fighting Corruption to Improve Schooling: Evidence from a Newspaper Campaign in Uganda.� Journal of the European Economic Association, 3 (2-3): 259-267. ----------- (2005b) “Using Micro Surveys to Measure and Explain Corruption.� World Development, 33 (12) (December). ----------- (2005c) The Power of Information: Evidence from a Newspaper Campaign to Reduce Capture of Public Funds, Policy Research Working Paper Series 3239, The World Bank, Washington, DC. REPOA/ESRF (2001) “Pro Poor Expenditure Tracking� Draft report submitted to the PER working group, Research on Poverty Alleviation and Economic and Social Research Foundation, Tanzania, processed. Republic of Kenya (2004) “Public Expenditure Tracking Survey (PETS) 2004�, Preliminary Report, Ministry of Planning and National Development, Ministry of Finance, Ministry of Health, Ministry of Education Science and Technology, Nairobi, Kenya, Mimeo. République du Cameroun (2004) “Enquête sur le suivi des dépenses publiques et la satisfaction des bénéficiaires dans les secteurs de l’éducation et de la santé, Phase I : Volet Santé », Rapport Principal des Résultats, Institut National de la Statistique, Avril. République du Cameroun (2005) “Enquête sur le suivi des dépenses publiques et la satisfaction des bénéficiaires dans les secteurs de l’éducation et de la santé, Phase II : Volet Education », Rapport Principal des Résultats, Institut National de la Statistique, Juillet. Roberts, Mark and James Tybout (1995) “Producer Turnover and Productivity Growth in Developing Countries� The World Bank Research Observer, 12 (1): 1-18 Rogers, F. Halsey, J.R. Lopez-Calix, N. Cordoba, N. Chaudhury, J. Hammer, M. Kremer and K. Muralidharan (2004) “Teacher Absence and incentives in Primary Education: Results from a New National Teacher tracking Survey in Ecuador,� in Ecuador: Creating Fiscal Space for Poverty Reduction, Washington DC.: World Bank, chapter 6. Rwanda (2004) “Rwanda: Public Expenditure Tracking Survey (PETS) Education�, September, Mimeo. Schade, Klaus and John Ashipala (2004) Public Expenditure Tracking Survey (PETS) and Quantitative Delivery Survey (QSDS): The Health Sector, The Namibian Economic Policy Research Unit, August, Mimeo. Schade, Klaus and Erwin Naimhwaka (2004) Public Expenditure Tracking Survey (PETS) and Quantitative Delivery Survey (QSDS): The Education Sector, The Namibian Economic Policy Research Unit, August, Mimeo. PETS-QSDS IN SUB-SAHARAN AFRICA: A STOCKTAKING STUDY 131 Turner, Anthony G., Gustavo Angeles, Amy O. Tsui, Marilyn Wilkinson and Robert Magnani (2001) Sampling Manual for Facility Surveys: For Population, Maternal Health, Child health and STD Programs in Developing Countries, MEASURE Evaluation Manual Series, No. 3, MEASURE Evaluation, Carolina Population Center, University of North Carolina at Chapel Hill, July. Wallace, Lindsay; Jim Edgerton, Richard Harris and Erisa Ocheing (2003) Sierra Leone: Public Expenditure Tracking Surveys (PETS) Review, Final Report, DFID and World Bank, May. Wane, Waly (2006) “Linking Leaking Public Funds and Outcomes: The Performance of Health Districts in Chad�, Presentation at the Workshop on Measuring Service Delivery, World Bank, April 27-28. World Bank (2003) World Development Report 2004: Making Services Work for Poor People, The World Bank and Oxford University Press, Washington, D.C. --------- (2004) “Le secteur de la santé au Tchad: Analyse et perspectives dans le cadre de la stratégie de réduction de la pauvreté� Groupe Thématique santé et pauvreté, February. ---------- (2006) Expenditure Tracking Survey in Senegal: The Health Sector, April, processed. World Bank (2005) « Enquête sur le système de suivi des dépenses dans le secteur de l‘éducation au Mali », Décembre. World Bank (2007) « Enquête sur le système de suivi des dépenses publiques jusqu’à destination dans le secteur de la santé au Mali », Juillet. Ye, Xiao and Sudharshan Canagarajah (2002) “Efficiency of Public Expenditure Distribution and Beyond: A Report on Ghana’s 2000 Public Expenditure Tracking Survey in the Sectors of Primary Health and Education�. Africa Region Working Paper Series No. 31, The World Bank, Washington, D.C. (2007) “Madagascar Service Delivery in the Education and Health Sector: Results of the 2006/2007 Public Expenditure Tracking Survey�, December 31.