WPS5033 Policy Research Working Paper 5033 Measuring the Quality of Education and Health Services The Use of Perception Data from Indonesia Basab Dasgupta Ambar Narayan Emmanuel Skoufias The World Bank Poverty Reduction and Economic Management Network Poverty Reduction Group August 2009 Policy Research Working Paper 5033 Abstract Satisfaction surveys offer a potentially convenient facility and the household's expected standard for quality, and cost-effective means for measuring the quality of which is influenced by its socioeconomic characteristics. services. However, concerns about subjectivity and The findings show that, after taking into account the selection bias impede greater use of satisfaction data. expectations of households, reported satisfaction does This paper analyzes satisfaction data about health and vary significantly with objective indicators of quality. The educational services from the 2006 second round analysis also checks for possible selection bias affecting of the Governance and Decentralization Survey in the results by using a two-stage selection model. The Indonesia to assess whether satisfaction data can serve model yields policy-relevant insights into the aspects of as reliable indicators of quality, despite dubiously service delivery that most affect satisfaction, highlights high levels of reported satisfaction. The authors use differences across rich and poor districts, and shows that an expectation disconfirmation model that posits that once the role of expectations has been factored in, the a user's satisfaction with a facility improves with the variation in user satisfaction can be highly informative for (positive) difference between the actual quality of the policymakers and researchers alike. This paper--a product of the Poverty Reduction Group, Poverty Reduction and Economic Management Network --is part of a larger effort in the group to analyze poverty and monitor and evaluate the effectiveness of poverty reduction programs. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at eskoufias@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Measuring the Quality of Education and Health Services: The Use of Perception Data from Indonesia Basab Dasgupta Ambar Narayan Emmanuel Skoufias The World Bank Keywords: Satisfaction Surveys; Perception Measures; Citizen Report Cards; Community Score Cards; Decentralization; Service Delivery; GDS-2; and Governance. Corresponding Author: Emmanuel Skoufias, The World Bank, Mail Stop MC4-415, 1818 H Street NW, Washington, DC 20433, USA. tel: (202)458-7539. fax: (202)522-3134. e-mail: eskoufias@worldbank.org. Acknowledgments: The authors are grateful to Elizabeth Bryan for her excellent research assistance and to Susan Wong, Claudia Rokx, and Javier Luque for comments in earlier drafts of this paper. Kai Kaiser, Blane Lewis, Emmanuel Jimenez, and Daan Pattinasarany also provided valuable suggestions. The findings, interpretations, and conclusions in this paper are entirely those of the authors and do not reflect the views of the World Bank. I. Introduction Can satisfaction-related questions be valuable in measuring the quality of public services, specifically in health and education? In examining this question, the paper develops a model for identifying the range of factors that influence satisfaction with services among users and examines how these determinants of satisfaction may vary by the type of service and economic status of regions within a country.1 Measuring the quality of public services (or changes in their quality over time) has become increasingly important, particularly to monitor or evaluate the impact of fundamental reforms in service delivery, such as decentralization. This has resulted in a range of initiatives attempting to measure "performance" indicators in service delivery. Many of such indicators are conceptually complex and expensive to collect information on. Satisfaction surveys can offer a cheaper and more convenient alternative, but only if they can be shown to have information content that is meaningful to measure performance in service delivery. In recent years, significant progress has been made in the realm of public service delivery measurement (see Amin, Das and Goldstein, 2007 for an overview of these instruments). An increasing number of the tools developed include subjective instruments that gauge citizen perceptions. These encompass citizen report cards, community scorecards, facility exit polls, and citizen satisfaction surveys. We focus on one type of subjective tool for gauging citizen perceptions, namely citizen satisfaction with services after Indonesia's "Big Bang" decentralization in 2001, which 1This paper draws significantly from a companion paper by Amin, Dasgupta, and Skoufias (2008) that presents a detailed analysis of satisfaction with health services in Indonesia, using the same dataset. It is also complementary to the study of Lewis and Pattinasarany (2009) focusing on satisfaction with educational services but abstracting from the issue selection bias. 2 transformed a highly centralized government to one that gave broad autonomy to the regions in most domains. We use nationally representative data on health and education services gathered in 2006 from the second round of the Governance and Decentralization Survey (GDS-2), which contains detailed questions related to governance and service delivery. The survey collected data on satisfaction from households and information on objective quality directly from facilities. In addition to the methodological question of whether satisfaction surveys yield useful results on facility quality, the findings of this paper also have more direct implications for policymakers in Indonesia. Our analysis of the determinants of satisfaction among service users suggests that users value certain dimensions of "quality" in public services (health and education) more than others, and that these determinants of satisfaction vary between rich and poor districts. This information is likely to be useful to inform policies engaged to improve basic services, well beyond what generic and aggregated measures of satisfaction typically provide. Section II explains the motivation for this study, examining the limitations of satisfaction surveys and outlining some of the concerns this paper seeks to address. Section III describes the dataset used for our analysis and the model used. In Section IV, we describe the models and the results on the determinants of satisfaction with both health and education facilities. Section V discusses the implications of our results for policy and the design of satisfaction surveys. 3 II. Why do satisfaction surveys merit study? Satisfaction surveys merit study for a number of reasons.2 Even though perceptions of citizens are imperfect indicators of quality of services, satisfaction surveys have considerable appeal as a practical way of measuring the impact of governance reform and decentralization. While other tools have been developed to collect more objective information on service quality (e.g. facility surveys, public expenditure tracking surveys),3 they are typically more time and labor intensive than satisfaction surveys. Thus the latter can be a quick and easy way for policymakers to measure the impact of governance reforms on government performance, particularly for sectors where measurement of service quality is not easy, provided citizen satisfaction is closely correlated with the actual quality of services. Even as satisfaction surveys are increasingly being used to measure the impact of governance reforms,4 there is little consensus on whether citizens' satisfaction reflects the actual quality of services satisfaction surveys. More research is therefore merited on the question of (i) whether data from perception surveys are useful in measuring quality of services, and (ii) if there is information content, how should such data be used and interpreted to measure quality of services. Moreover, understanding what factors influence citizen satisfaction is crucial in order to evaluate the impact of decentralization. In the typical model linking decentralization to 2 See Amin et al (2008) for a more detailed discussion 3 Facility surveys are used to directly measure the quality of infrastructure and resources. An excellent example is found in Banerjee et al (2004) ­ their study of 100 villages in the Indian state of Rajasthan combined a household survey and a village census with a detailed facility survey of public and private health providers. Public expenditure tracking surveys (PETS) measure the efficiency of fund flow through different levels of government ­see, for example, World Bank (2005) for results from PETS in the primary education sector in Cambodia. 4 Since the first citizen report card (CRC) initiative was adopted in 1994 in Bangalore, India similar initiatives have been adopted around the world including in Bangladesh, the Philippines, Sri Lanka and Vietnam (Paul, 1999). 4 improved outcomes, an important premise is that local governments would improve their performance on account of improved accountability, which in turn hinges on citizens being able to discern between good and bad government and then influence their local authorities.5 This implies that in order to understand whether decentralization is likely to improve service delivery, it is important to understand what household and community level factors (other than quality of services) determine citizen satisfaction. The above questions are addressed here by exploring the relationships between satisfaction, household and community characteristics, and the actual quality of service delivery as measured using objective indicators. Concerns with satisfaction surveys and the example of GDS-2 Despite the growing prevalence of surveys administering satisfaction-related questions, there are serious concerns regarding the information content of the data, fueled by results from survey data that often appear puzzling. One example is Indonesia, where the nationally representative GDS-2 household survey reveals extremely high satisfaction with health and education services. More than 90 percent of households report being at least somewhat satisfied with the overall quality of health services (Figure 1), while 72 percent feel that health service delivery has improved in the last 2 years (Figure 2). For education services, more than 80 percent report being at least somewhat satisfied and 73 percent report improvement in the last 2 years. This happy picture is quite inconsistent with the poor reputation of health and education services in Indonesia, which is also supported by more objective measures of quality 5 See, for instance, Ahmad et al. (2005) and Grindle (2007). 5 from surveys. Such inconsistency, along with the apparent lack of variation in the response to the satisfaction question for two different types of services (education and health) seems to limit the usefulness of the GDS-2 data on satisfaction. High reported satisfaction has been attributed more to cultural norms or social pressure rather than the superior quality of service delivery in Indonesia. The problem with the apparent lack of variation in Indonesia satisfaction data is not unique to Indonesia, but nor is it universal. High variation in satisfaction with education and health services among respondents is seen, for example, in a number of countries where Core Welfare Indicators Questionnaire (CWIQ) surveys have been administered.6 Researchers and policymakers alike have long harbored doubts regarding the accuracy of perception-based measures of quality, due to the subjective nature of these instruments (e.g. Bertrand and Mullainathan, 2000). The absence of a common baseline against which respondents' ratings can be benchmarked makes interpreting these data difficult and comparing data-points across regions and countries even trickier. These problems may explain, for example, why respondents from citizen report card surveys in the Indian state of Bihar have reported higher levels of satisfaction with schooling than those in Kerala, even though Kerala vastly outperforms Bihar in most measures of access to and quality of public education and education outcomes. Support for skepticism about perception-based measures of quality can be found in several studies demonstrating little or no correlation between objective indicators of quality and satisfaction levels (Brown and Coulter 1983; Stipak 1979). Other studies, such as Deichman and 6In Pakistan, a survey (2006-07) based on CWIQ with a district-representative sample of 73,000 households showed satisfaction rates of 35 and 61 percent for government basic health facilities and schools, respectively. A CWIQ survey in Sierra Leone (2007) yielded satisfaction rates of 38 and 42 percent among those attending primary and secondary schools respectively. 6 Lall (2003), Kelly and Swindell (2000) and Parks (1984), show a statistically significant correlation. But studies have shown that factors other than quality also influence satisfaction, including demographic factors like age, gender, education, income and ethnicity, as well as attitudes and predispositions related to political beliefs, consumer expectations or past experiences (see Amin et al, 2008 for a fuller discussion). Many of the demographic factors may be easy to observe and control for, but others such as expectations, experiences and predispositions are harder to measure or proxy. This makes it difficult to isolate the impact of quality on satisfaction and makes the interpretation of satisfaction data a complex exercise. These problems have led many to argue that perception-based instruments may be useful for initiating public debate about government performance and enhancing accountability, but not for measuring actual quality of public services. This paper examines if this is true in the Indonesian context, using GDS-2 data. This survey is uniquely suited for such analysis, given that it is one of those rare instruments that provide data on satisfaction of households along with (from a facility level survey) objective indicators of quality of the facilities the households are using. The analysis also sheds some light on how satisfaction data can be used and interpreted as a measure of quality ­ a complex question for all the reasons discussed above. Finally, our detailed analysis of the determinants of satisfaction will also shed some light on a question likely to be of interest to policymakers in Indonesia: What are the specific dimensions of quality that seem to matter more than others for user satisfaction? For obvious reasons, indicators of satisfaction would be unable to address this question by themselves. The unique opportunities offered by GDS-2 allow us to address this question in a framework that corrects for some of the most typical problems in using and interpreting satisfaction data. 7 III. Data and Model The second round of GDS was conducted from May to September 2006, to assess the state of governance and local public service delivery in Indonesia by collecting data on quality and satisfaction from households, communities, and facilities. The GDS-2 household sample is nationally representative, selected using a stratified random sampling approach. A total sample of 8544 households were distributed equally among 1068 hamlets (dusun) that served as the primary sampling unit (PSU), which were in turn distributed between 89 districts (kabupaten/kota), 267 sub-districts (kecamatan) and 534 villages.7 The sample of health and education facilities was not selected at random, and was instead guided by which facilities were reported as most frequently used by households. For health services, the 6 community health centers (puskesmas) that were most frequently mentioned by surveyed households were selected for secondary data collection within each district included in the survey. Excluded from the facility survey sample were private facilities, public general hospitals and the ancillary facilities (known as the pustu, polindes, and pusling) that form the extended network supporting the main puskesmas.8 Education facilities were selected for secondary data collection (through a facility survey) in a way similar to that for puskesmas. The most frequently used public elementary school in a 7 89 districts (kabupaten/kota) were randomly selected from the 408 that remained after excluding all districts in Aceh and Jakarta, and the 3 districts used for pre-testing. Three sub-districts (kecamatan) were randomly selected from each district using probability proportionate to size sampling; the same method was used to randomly select 2 villages (desa) within each sub-district; and 2 hamlets (dusun) were randomly selected within each village. Latest population lists provided by the hamlet heads were used to randomly select 8 households in each hamlet. 8 Out of 8,544 households, 4,358 use puskesmas and its related networks. Of these, facility/secondary data are available for facilities used by 2,269 households (see Appendix A, Table A-1), since the less frequented puskesmas and most of the affiliated facilities were excluded from the facility sample. 8 village was selected based on information from interviews with households and the heads of the two hamlets sampled in each village. Similarly, the most frequently used public junior high school in each sub-district was selected based on interviews with households in the two villages sampled in each sub-district and the two hamlet heads for each village.9 Data on both household satisfaction and objective measures of facility quality are available for 52 percent of households using public health facilities and 57 percent of households using public education facilities. The partial overlap between the household and facility samples and the non-random selection of facilities is a source of concern about possible selection bias. As mentioned earlier, average satisfaction with government health and education facilities is surprisingly high in GDS-2. Among the five options households were given (1=Satisfied, 2=Quite Satisfied, 3=Quite Unsatisfied, 4=Unsatisfied, and 5=NA or Unknown) for rating the performance of services, option 1 or 2 was chosen by around 90 and 80 percent of households for health and education services respectively. Although most respondents in GDS- 2 reported being "satisfied", many of them chose the unequivocal response "Satisfied" versus the lower, more qualified option "Quite Satisfied". To capture the variation, we define our dependent variable as a binary indicator of satisfaction (S) where all those who chose 1 is classified as "Satisfied" and everyone else as "Dissatisfied". Table 1 shows the distribution of S for health and education services. 95,877 households have children of school age (6-19) ­ a necessary pre-condition for using schools. Among them, facility/secondary data are available for schools used by 2,955 households, including 503 households using non-public institutions (see Appendix A, Table A-2). 9 1 if satisfied (code 1) Satisfaction (S) 0 otherwise (codes 2,3,4,5) Our next step involves devising a robust model for interpreting and evaluating the data on satisfaction. Most analyses of satisfaction data begin by comparing objective indicators of quality of service delivery with satisfaction levels. Recognizing that objective quality is rarely a sufficient explanation of satisfaction (Stipak 1979), many of these models incorporate the role of expectations, pre-dispositions and perceived quality in determining satisfaction (Serra 1995; DeHoog et al 1990; Van Ryzin 2004). A commonly-used model is the expectancy disconfirmation theory (Cardoso 1965; Deichmann and Lall 2003; Oliver 1980, 1997), where satisfaction is determined by the degree to which objective performance of service providers meets the expectations of consumers. Where positive disconfirmation occurs (performance surpassing expectations) households are satisfied; conversely, negative disconfirmation leads to dissatisfaction. Positive if Actual quality Expectation Disconfirmation Negative (1) if Actual quality Expectation The performance model is a more dynamic variant of the disconfirmation model, where consumer expectations are constantly recalibrated based on recent experiences of service use (Boulding et al 1993; Johnson et al 1995). Satisfaction is therefore a result of (i) actual service quality and (ii) expectations, where the latter is determined by a range of factors, namely a household's characteristics, information available to the household and its experiences with past consumption of services. Given this broad base of household specific determinants of 10 expectation and the possibility of measurement error in expectations, we consider a range between minimum and desired expected standard to be the latitude of acceptance or expectation (Figure ). Households are `quite satisfied' when actual quality falls within this area, i.e. does not differ significantly from household's expectation. Actual quality beyond the desired expected standard (the upper bound of expectation) yields positive disconfirmation or satisfaction to a household. This approach can be implemented if data on satisfaction and expectations can be successfully matched with data on actual service delivery. Models based on this approach tend to ignore the difficulty of measuring both objective quality and consumer expectations when it comes to basic services, and do not adequately account for the role of governance in determining satisfaction. Moreover, given that satisfaction with a service can be observed only for a user of the facility, such a model also has to account for the possibility of sample selection bias, related to the decision of a user to opt for a particular service facility. Typically, two main sources of sample selection bias are self-selection by users into the sample, and non-random sample selection of service delivery facilities by data analysts and survey administrators. Satisfaction surveys often suffer from both these problems and this is true for GDS-2 as well. As discussed in Section III, selection bias in GDS-2 arises from facility data being collected from only the most frequently used public health and education facilities instead of a random sample of all facilities used by households, which creates the possibility of bias arising from self-selection of households into facilities included in the sample. Bias can occur because households choosing the highly frequented public health facilities may systematically differ from users of facilities excluded from the sample, and households choosing 11 the surveyed public schools may systematically differ from those using other schools (such as, the users of private schools10). Selection bias would be more of a concern in analyzing satisfaction with (and usage of) basic services, as opposed to utilities. Basic utilities like water or electricity tend to be universally used and often have single providers servicing entire areas, with limited scope for users to exercise choice. In the case of basic services such as health and education these conditions are less likely to hold. This increases the likelihood of selection bias and makes the typical approach of matching users' satisfaction with services with objective data from public service providers problematic. To address these issues, a modified version of the disconfirmation model is presented in the next section to identify the determinants of satisfaction, using the restricted sample of households who can be matched with the objective data for each type of facility. A reduced- form version of this model is estimated using the matched sample of household and facility level data. In addition, to examine whether selection bias is a significant concern, a Heckman 2- stage selection model is estimated where satisfaction is estimated on the matched dataset only after the factors that determine a household's choice of a facility are taken into account. IV. Results An accurate analysis of the determinants of reported satisfaction will require modeling the determinants of satisfaction levels, after taking into account the household specific indicators 10For example, Lankford et al (1995) shows that socio-economic characteristics, including income and parental education and family composition, along with the location of a household and school characteristics, influence strongly the choice between public and private schools in the United States. 12 that are likely to influence a household's expectations of service quality. As discussed earlier, expectations are based on a range of factors that includes, but may not be limited to, household demographic characteristics, access to and availability of information about the service and the provider (governance) and the past experiences. The inability to capture every determinants of expectation creates some "latitude" of measurement error. Given this, we define household's utility from expected quality of service as: V (QE ) QE x 1 (2) Where x represents the determinants of expectation of service quality and 1 represents the latitude of measurement error. is the utility of a household from actual quality of services, comprised of different dimensions of actual objective quality like coverage, infrastructure, services and governance. Similar to expectation, we define as: n V (Q Ai ) Q Ai 2 i 1 where n is the dimension of actual service quality. Given this set up, we modify the disconfirmation model of equation (1) as: 1 0, that is equivalent to: 1 1 2 . (3) This leads to a discrete choice problem with satisfaction defined in terms of the difference between actual and expected quality. We assume that the actual dimensions of quality (QAi) of a facility are known to only those who have used that facility. This is akin to assuming that the services from a facility is an "experience" good, information on whose quality is known to only those who have consumed 13 that service.11 This allows us to restrict our modeling of satisfaction to the sample of households who are users of health/education facilities that can be matched with corresponding facility level data from the survey. This assumption it necessarily rules out any "reputational" effect of service quality on the satisfaction of non-users as well. However, it is reasonable given our objective, which is to identify the informational content in the satisfaction reported by the users of a service.12 Notably, this assumption also has a key implication for how we test the robustness of our results to selection bias, as described later in this section. Following equation (3) above, our reduced form discrete choice models posit that expectations play a role in defining a benchmark to measure satisfaction with the facility. The estimated equations examine how the binary variable of satisfaction (S) with public health and education facilities varies with objective quality of the facility and the governance environment within which the facilities operate, once the expectations of quality are taken into account. Satisfaction with public health facilities Given the assumption that the actual dimensions of quality of a facility are known to only those who have used that facility, the model is estimated for households who have reported using a public health facility for which data is available from the facility survey (a sample of 1,786). The dependent variable S takes the value 1 if the household is satisfied with a particular health facility and 0 otherwise. To capture possible differences in expectations of households 11 An experience good is a product or service where product characteristics (such as quality) are difficult to observe in advance, but can be ascertained upon consumption. The concept is originally due to Nelson (1970). 12 Moreover, even if reputational factors affect the satisfaction reported by non-users as well, the underlying model explaining their satisfaction levels is likely to be quite different from that of users, which would support the case for dropping non-users from our analysis. 14 and quality of facilities across rich and poor districts, the models are also estimated separately for households from poor and rich districts, with sample sizes of 682 and 1,104 respectively.13 The 88 districts that GDS-2 households belong to are classified into rich and poor districts using gross regional domestic product (GRDP) per capita for each district ­ the bottom 50 percent of districts by GRDP per capita are defined as poor districts, while the top two quartiles are defined as rich districts. For each of the three samples, two types of models are used that are different only in the way household's expectation of quality or QE(x) is proxied. In model (1), in the absence of a single, readily-identifiable indicator for expectations, we proxy expectations with a range of household characteristics, such as gender, age, education levels, religion, household expenditure and social status. An index of whether respondents have information on bribery and corruption in health services, budget and development plan at the sub district level and the source(s) they rely on for such information is also included. This index indicates the information a household has on governance environment in which the facility operates. Model (1) allows us to see how household level factors influence satisfaction with a health facility, because of the way they influence household's expectations regarding health services. In model (2), we create a single index from all the variables used as a proxy for expectation (based on the first principal component) to represent expectation. The single index for expectation is useful to see how expectations on the whole matter for household's satisfaction with a health facility. Quality of public health services is modeled using the following dimensions of quality measured from the facility survey: (a) the coverage area of the facility; (b) the types of medical 13For example, Duffy (2000) finds that there are a few services that deprived area residents are less satisfied with compared to their counterparts in less deprived districts. 15 support provided; (c) the quality of inputs in terms of human resources and medical supplies; and (d) facility infrastructure. Principal component (PC) analysis is used to construct, for each of these categories, a single index that is a composite of multiple indicators from the facility surveys. Each index (with a mean of 0 and a standard deviation of 1) is the first PC and increases with higher levels of quality (see Appendix B for the definition of each index). In addition, we use reported perceptions of households on the quality of service at the health facility they have last visited, namely: (e) whether the household faced any discrimination in the facility, (f) the speed of service provided and (g) the waiting time to receive services. The institutional and governance environment in which health services function is captured by the following indicators derived from the household survey: (i) level of accountability (proxied by the responsiveness of service provider to complaints); (ii) an index of active participation in community level initiatives providing and improving health services.14 Finally, binary variables for rural/urban areas and regional location are included in all models to take into account any effect of location of a household on expectations about quality and unobserved differences in facility quality, both of which may systematically vary across regions and rural/urban areas. See table 2 for the detailed results from the estimation of models (1) and (2) for pooled, rich district and poor district samples and Appendix B for the description of variables. After conditioning for expectation, satisfaction varies significantly with various indicators of objective quality and governance in the expected directions, with distinct differences across rich and poor districts. Firstly, satisfaction is significantly correlated with the 14The index of active participation is comprised of multiple indicators related to participation in community level initiatives for providing and improving health services ­ namely, being present in the community meetings, providing and sharing ideas, and making monetary contributions. 16 level of support available to the main puskesmas from ancillary facilities for the pooled sample and the rich district samples, while the correlation is weak in poor districts. Secondly, the quality of inputs ­ measured in terms of human personnel and medicinal inputs ­ is an important determinant of satisfaction in the pooled sample and poor districts, but not in rich districts. Thirdly, higher speed of service delivery and lower waiting time are associated with significantly higher satisfaction in the pooled sample and rich districts, but not in poor districts. Interestingly, the quality of infrastructure has no impact on satisfaction in all three samples. A number of studies have indicated that participation at the local level and accountability of service providers affect satisfaction with services (including DeHoog et al, 1990 and Licari et al, 2005). We find that higher levels of participation in the health services and responsiveness to complaints among service providers are significantly associated with higher satisfaction in the pooled sample. The effect is particularly strong for poor districts, but nearly insignificant for richer districts. Thus citizen satisfaction with health service delivery seems to respond to the availability of ancillary facilities to support the main puskesmas, quality of inputs (staff and medicinal inputs) and speed and timeliness of service, but not to the quality of infrastructure. For rich districts, support from ancillary facilities and speed and timeliness of service are important for satisfaction. On the other hand for poor districts, quality of inputs, community level participation of users in service provision and higher accountability of service providers are important correlates of satisfaction. The results appear to support the use of the modified expectations disconfirmation model we posited above. The coefficients of the "index for expectation" in model 2 (see Table 2) show that lower overall expectation of quality significantly increases the probability to be 17 satisfied for the pooled sample and across rich and poor districts alike, with the relationship being the strongest for households from poorer districts. The negative and significant relationship between expectations and actual satisfaction is consistent with the prediction of the model defined by equations (1) to (3). Among the variables that influence household expectation of quality, demographic characteristics play an important (and statistically significant) role in influencing satisfaction, particularly in poor districts. Respondents who are older, less educated, belong to female- headed households and are not associated with the elite class are more likely to have higher satisfaction in poor districts. Given equation (3), these results are consistent with expectation of quality being lower among these groups­ which make intuitive sense. By similar reasoning, greater knowledge about governance and corruption issues (and access to information sources) appears to be associated with higher expectation of quality and lower satisfaction in poor districts. This coefficient may however be misleading, since it may not necessarily reflect the impact of information on expectation of quality, but rather that dis-satisfied users are more motivated to seek out information about governance and corruption (endogeneity). In rich districts, education is the only household level factor that seems to matter for satisfaction, with higher education of a respondent associated with higher expectation and lower satisfaction. Finally, the regional location of a household plays no significant role in influencing satisfaction with health services. However, urban households are more likely to be satisfied with public health facilities than rural households, particularly for rich districts. On the whole, households from rich districts are more likely to be satisfied with health facilities as compared to those from the poorer districts. 18 Robustness of results to selection bias. Given the concerns about possible selection bias (see discussion Section IV) affecting our results, we use the Heckman two-stage model to control for selection bias in the matching sample of households with both facility and satisfaction data. The model corrects for the fact that the sample of households for which we have corresponding facility data is defined non- randomly through a combination of self-selection and the method of sample selection in GDS-2. The first stage selection equation predicts the propensity of households to use a public health facility for which objective data on quality are available. The second stage outcome equation examines how satisfaction (S) varies with various indicators of quality and governance, conditioned on the selection of the facility. The binary dependent variable in the selection equation takes the value 1 if the household uses a public health facility for which facility data is available and 0 otherwise.15 The level of satisfaction (S) with a facility is the dependent variable in stage 2, where the definition of S is identical to that in the reduced form model above. The propensity of a household to choose a health facility is estimated as a function of household and community characteristics, perceptions of households about health services, governance and institutional environment and location (regional) fixed effects. Conditional on the selection of a facility, household's satisfaction with the facility is estimated as a function of objective indicators of facility quality, perceptions of households about health services, governance and institutional environment, a few household characteristics (that can influence the household's expectation of quality) and 15Out of the total household sample of 7,686 households used for the first-stage selection model, the dependent variable takes the value one for 2,064 households. 19 regional fixed effects ­ identical to the reduced form probit model estimated above. A few household and community characteristics enter into the first stage selection model but not the second-stage regression, since they are likely to influence a household's choice of facility but not its satisfaction with the facility (see Appendix C for a more detailed discussion of the identification strategy). The assumption stated earlier ­ that the services from a facility is an "experience" good, whose actual quality is known to only those who have used that facility ­ implies that the variables measuring facility quality influence satisfaction with the facility but not the household's choice of facility.16 We do not find significant selection bias from our estimated results,17 which implies that the reduced form discrete choice model discussed above is appropriate for our analysis. The detailed model and results from our 2 stage-Heckman selection model are reported in Appendix C (Table C-4). The results on determinants of satisfaction are quite similar for the reduced form model and the 2-stage Heckman selection model, which lends confidence to the reduced form model results. Satisfaction with public schools The model for satisfaction with public education facilities (schools) is similar to that for health facilities. The estimated equations examine how the binary variable of satisfaction (S) with public schools varies with objective quality of the facility and the governance environment 16 This assumption can be justified since accurate information on the quality of a facility is likely to be available only to the users of the facility. That said, it is somewhat restrictive since the actual quality of the facility can have an impact on its reputation, which in turn can influence the household's choice of a facility. 17 The coefficient of Inverse Mill's Ratio () in the second stage regression in each case is insignificant in two cases, and only weakly significant for the pooled sample. These results are also quite robust to changes in specifications of the 1st stage selection model. 20 within which the school operates. Household expectations are proxied in an identical manner as in the case of health, in model (1) with a range of household characteristics and in model (2) with a single index constructed using Principal Components from the household characteristics. The sample is restricted to 2,557 households with at least one child of school age who attends a public school for which facility level data are available. Like in the case of health, the models are also estimated separately for households from poor and rich districts, with sample sizes of 1,094 and 1,463 respectively. The first key difference between the education and health cases is that unlike for health, the list of independent variables to estimate satisfaction with schools does not include indicators of service quality reported by households, such as level of discrimination, speed of service and waiting time. Secondly, quality of public schools is proxied using the following information from the facility survey: (a) the extent of participatory decision-making in school; (b) quality of infrastructure in school; (c) the quality of teaching staff; (d) student performance (in terms of dropouts and repeats); and (e) coverage of students by the school (the size of enrollment and rate of attendance). For each of these categories, Principal Component analysis is used to collapse the multiple indicators provided by the facility surveys into a single index (see Appendix for a more detailed description). Thirdly, the institutional and governance environment in which public schools function is captured by the following indicators reported by households: (i) level of accountability, proxied by the responsiveness of service provider to complaints; (ii) an index of participatory management 21 of schools;18 and (iii) an index of coverage and implementation of the School Operation Assistance Program (BOS or "Bantuan Operasional Sekolah") ­ a school grant program administered by the central authority introducing school-based management and targeting non-salary operational expenditures of schools. Variables (i) and (ii) are analogous to those used to proxy the governance and institutional environment of health facilities. Variable (iii) is introduced because implementation of the BOS program can potentially bring about a significant change in the governance environment of the school, by affecting how non-salary expenditure decisions are made. See Table 3 for the detailed results from the estimation of models (1) and (2) for all three samples and Appendix B for the full description of variables. Like in the case of satisfaction with health facilities, we find that after conditioning for expectation, satisfaction (the binary variable S) with public schools is correlated with certain indicators of objective quality and governance. There are key differences between rich and poor districts in terms of the factors that influence satisfaction with public schools. For the full sample of households, none of the objective indicators of quality available from the facility survey are significantly correlated with satisfaction. Interesting patterns however emerge from models estimated separately for rich and poor districts. Better infrastructure facilities in schools (e.g. condition of classrooms, library, sports hall, computer rooms and availability of books) and higher coverage of students by schools (level of enrollments and attendance) are associated with significantly higher level of satisfaction in poor districts, but has no effect for rich districts. Conversely, higher teacher quality (in terms of The index of active participation indicates whether decisions about school's mission and vision were 18 made together by the principal, teachers and community. 22 experience, number and student-teacher ratio) leads to significantly higher satisfaction in rich districts, but has no effect in poor districts. Indicators of student performance like dropout and repetition rates do not seem to matter for satisfaction for any group of households. Among the variables that proxy governance and institutional environment of public schools, satisfaction of users in rich districts is significantly higher when the decision-making for school's mission and vision is participatory ­ taken jointly by school principal, teachers and the community. The positive association between satisfaction and participatory decision- making is weak for the full sample and does not exist in poor districts. The index representing coverage of a school by the BOS program and the extent to which the implementation of BOS has progressed has positive and significant effect on the level of satisfaction in the full sample. Clearly, this association is driven by the rich districts where the "BOS effect" on satisfaction is significant, while no such effect is seen for poor districts.19 Increased responsiveness of provider to complaints is only weakly correlated with higher satisfaction and that too just for the poor districts. The results above seem to suggest that users in poor districts are more concerned with the basic features of a school (e.g. facilities in the building, enrollments and attendance of students), whereas in richer districts satisfaction is influenced by factors reflecting more "second-generation" issues, like quality of teaching staff, reforms related to school-based management and the extent to which the decision-making process is participatory. One explanation for this is the fact that schools in poor districts lag in facilities and enrollments than 19The index for BOS includes indicators related to adequacy of BOS allotment, number of students covered in BOS, and the extent of implementation of BOS in aspects like preparation of the school's development plan and budget, implementation of school based management system, dissemination of required information and number of disbursements (see Appendix B). 23 those in richer districts, which leads to households in poor districts placing a premium on these features, whereas those in other areas focus on other aspects of quality.20 The coefficients of the "index for expectation" in model 2 (see Table 3) show that lower overall expectation of quality increases the probability to be satisfied, with the correlation being significant for the pooled sample and rich districts. Like in the case of satisfaction with health services, the negative association between expectations of quality and likelihood to be satisfied is consistent with the prediction of our modified expectation disconfirmation model. Looking at variables that are likely to proxy a household's expectation of quality, demographic and socio-economic characteristics seem to matter less in the case of education than that of health. For the pooled sample, the only household characteristic that significantly influences satisfaction with public schools is the household's status as (or association with) elite in the community. For households in poor districts, higher education attainment and status as elite (or being associated with the elite) are associated with lower satisfaction with public schools. This is consistent with expectation disconfirmation, which would suggest that higher education and status in the community is likely to induce higher expectations of quality that would lead to lower satisfaction with the service. In poor districts, greater knowledge about governance/corruption issues and budget and development plan at the sub district level (and access to information sources) increases satisfaction significantly, suggesting that greater knowledge of these issues may lead to lower expectation of quality. This association is the opposite of what was seen for health, but hard to interpret (as in the case of health) because of 20There are statistically significant gaps between rich and poor districts in school infrastructure, and enrollment and attendance of students (see Table A-3, Appendix A). There is a significant gap in teacher availability and experience as well; these aspects of school quality however do not seem to influence perceptions in poor districts. 24 concern about endogeneity of the variable to satisfaction. In rich districts however, none of the household characteristics matter for satisfaction with public schools. The coefficients of regional dummies indicate that after controlling for all observable factors, households from poor districts in Kalimantan, Sulawesi and Sumatra regions are less likely to be satisfied with public schools compared to those from poor districts in Java, while the regional effect is insignificant for rich districts. This suggests that in these three regions, there are significant differences in unobserved location-specific factors (that influence users' satisfaction with schools) between rich and poor districts, compared to Java. Unlike in the case of health, urban households are not more likely to be satisfied with public schools than rural households, and households from rich districts are about as likely to be satisfied with public schools as households from poorer districts. Robustness of results to selection bias. As in the case of satisfaction with health facilities, we use the Heckman two-stage model to control for selection bias in the matching sample of households with both school facility and satisfaction data. The sample for the first-stage selection model is restricted to households that have at least one child of school age. The dependent variable in the first stage selection model is a binary variable that takes the value 1 if the household has a child going to a public school for which facility data is available and 0 otherwise.21 The level of satisfaction (S) is the dependent variable in stage 2, where the definition of S is identical to that in the reduced form model above. The propensity of a household to choose a public school that is in the facility sample is 21Out of 5,599 households in the sample for the first-stage regression, the dependent variable takes the value one for 2,595 households. 25 estimated as a function of household and community characteristics, perceptions of households about governance and institutional environment and location (regional) fixed effects. Conditional on the selection of a facility, household's satisfaction with the facility is estimated as a function of variables that are same as in the reduced form probit model. As in the case of satisfaction with health facilities, we find no significant selection bias from our estimated results and the results are similar to those from the reduce form model discussed above. This implies that the reduced form discrete choice model is appropriate for our analysis. The detailed model and results from our 2 stage-Heckman selection model are reported in the Appendix C (Table C-5). Comparing health and education results For health and education alike, satisfaction with public facilities is significantly correlated with specific objectives measures of quality of a facility, as well as with indicators of governance and institutional environment of the facilities. The direction of correlation is as expected (satisfaction positively correlating with quality), which suggests that the satisfaction indicator, in the way we have defined it, has considerable information content. Along with the broad similarities, there are important differences between the determinants of satisfaction in health and education facilities. One difference relates to the roles of infrastructure and quality of service. While quality of infrastructure seems to have no influence on satisfaction with health facilities, the quality of school infrastructure is a significant determinant of satisfaction with schools in poor districts. Indicators of quality ­ like availability of personnel and medicinal inputs ­ as well as speed of service and waiting time are key determinants of satisfaction in health services, albeit with important differences between rich 26 and poor districts. For education facilities, on the other hand, indicators of student performance (dropout and repetition rates) have no correlation with satisfaction and an index of teacher quality (education, experience and student-teacher ratio) is correlated with satisfaction only in rich districts. On the whole, the relationship between indicators of facility quality and satisfaction is much weaker in the case of public schools than it is for public health facilities. For both education and health facilities, greater participation of users in the management of the facility seems to induce higher satisfaction among users. There are differences, however, in how the results shift between rich and poor areas. For health facilities, households in poor districts are more likely to be satisfied with higher participation in the administration of health services. For schools, households in rich districts are more likely to be satisfied when management of schools is more participatory or the implementation of BOS program (implementing a school-based management system) is more advanced. Higher responsiveness of provider to complaints about facilities improves satisfaction with health facilities but does not seem to influence satisfaction with schools. Based on the information we have, it is hard to find a consistent story to explain all these patterns. But taken together, the results confirm that (i) the governance and institutional environment of public services matters for user satisfaction with services, and (ii) the interactions between these factors and satisfaction with services are complex and depends on the type of service and characteristics of the area. Another important difference between health and education is in the role of regional location of a household in determining satisfaction. Regional location does not matter for satisfaction with health facilities. In contrast, satisfaction with public schools is likely to be much lower in the poor districts in Kalimantan, Sulawesi and Sumatra regions compared to the poor areas of Java region, while no such regional effect is seen for rich districts. This seems to 27 suggest that there is greater heterogeneity (between rich and poor districts) in the unobservable aspects of public school quality in these three regions compared to Java region. On the other hand, satisfaction with public health facilities is likely to be higher in urban areas than in rural areas, whereas urban/rural location has no impact on satisfaction with public schools. Finally, there are significant differences in how expectations on service quality are formed for health and education facilities. Two broad patterns emerge. Household characteristics appear to proxy expectations of quality much better in the case of health facilities than schools and for poor districts than for rich districts. Which characteristics matter for expectation (and thus for satisfaction) and how they matter vary significantly between health and education as well. For example, knowledge about governance, corruption and local planning/budget issues seem to matter for satisfaction in poor districts only for education and health alike; but the correlation is significantly negative for health while being significantly positive for education. These differences indicate that a household's expectation of quality from different types of services can be formed very differently, depending on the type of service and the economic condition of the area. That said, for both health and education facilities, the composite index of expectation is negatively correlated with the likelihood to be satisfied with a facility. The results on the whole support our core hypothesis of expectation disconfirmation ­ satisfaction of a user with a facility improves with the (positive) difference between actual quality of the facility (that the user has experienced) and the household's desired expected standard for quality. This would imply that the satisfaction is more likely when the quality of the facility is better and the household's expectation of quality is lower ­ patterns that emerge clearly from our regressions estimating satisfaction with public health and education facilities. 28 V. Conclusions and Policy Implications Our analysis of data from the GDS-2 has useful implications for the design and use of surveys that measure satisfaction levels of users of public services. The analysis shows that proper interpretation of satisfaction data would require finding meaningful variation in satisfaction responses and econometric models that account for subjectivity of responses and examine the possibility of selection bias. Once the role of expectations in determining satisfaction with facilities has been factored in, the variation in satisfaction level can be highly informative for policymakers and researchers alike. Satisfaction surveys are likely to remain a popular method for monitoring the quality of services because of the ease of administering such surveys. Thus it is useful to reflect briefly on what our analysis suggests for the design of such surveys, especially in the "second-best" scenario where collecting data from households and facilities is not possible for practical reasons. Firstly, our findings confirm that a range of factors other than those related to the quality of facilities play a key role in determining the satisfaction level of households. Thus even when an accompanying facility survey is not a practical option, there are clear benefits in having a satisfaction survey collect as much information on the characteristics of households and communities as possible, including sources of information and social status of a household. This would allow an analyst to econometrically correct for some of the subjectivity in the satisfaction data. The models estimated in this paper control for factors that proxy a household's expectations of quality from a service, which would in turn influence the reported level of 29 satisfaction. In the absence of such correction, an interpretation of satisfaction data, across space or time, can be misleading. Secondly, data on satisfaction from a random sample administered at the household level is likely to yield more representative results in most cases than a typical "user survey", namely a survey of a sample of users of a particular type of facility. A random household sample may allow for satisfaction data to be collected from a representative sample of a country or a sub-region, or allow for the correction of selection bias arising from the household's choice of a particular type of facility (if satisfaction is reported by only a subset of households using that facility). In contrast satisfaction reported by a sample of users of a particular type of facility can be subject to an undetectable selection bias, which would yield misleading results.22 In this paper, while we estimated the reduced form models only on the sample of users of certain types of facilities, the larger household sample allowed us to check for selection bias in our results. The concern about selection bias in a survey limited to just users especially applies to services like health and education that present potential users with some degree of provider choice. Incorporating questions on satisfaction with basic services in household surveys is becoming increasingly popular. For example, the aforementioned CWIQ surveys, fielded in a large number of countries (mostly in Africa) combine questions on access, usage and satisfaction with basic services with those on household and community characteristics, on a nationally (or regionally) representative sample.23 In cases where user surveys are the only 22 For example, if one is comparing satisfaction data from a user survey over time, a change in satisfaction levels may reflect a "real" change, or just may be a result of a changing profile of households opting to use a particular type of facility, with differences in characteristics that influence their satisfaction. 23 Numerous African countries have had at least one round of CWIQ. Some, including Ghana, Malawi, Mozambique and Rwanda have had more than one round. In Pakistan, two rounds of a survey modified from CWIQ have been fielded in 2004-05 and 2006-07, which measures satisfaction with public services. 30 practical option due to time or cost constraints or because the service is used by a miniscule proportion of the population24 the data must be interpreted with caution, qualified with the necessary caveats. Thirdly, our analysis suggests that even if a large majority of respondents appear to be more or less satisfied (or dissatisfied), useful information can still be extracted by using the variation in responses rather than the actual responses directly. There are simple implications for survey design as well. Variation in response is more likely to occur when surveys phrase satisfaction-related questions as multiple-choice questions (as in GDS-2), as opposed to a simple "yes/no" or "satisfied/dissatisfied". Another way to induce variation in responses is by framing separate questions on satisfaction with different aspects or features of a school or health facility (CWIQ surveys usually adopt this approach), as opposed to a single question on satisfaction with education or health services. Our analysis also sheds light on which aspects of health and education services matter the most for user satisfaction in the case of Indonesia. How these determinants differ across rich and poor districts (the upper and lower 50 percent of the districts in the GDS-2 sample, ranked by GRDP per capita) also suggests differences in priorities among users depending on the economic condition of an area. Firstly, access to ancillary medical facilities supporting the main public health facilities (Puskesmas) is a universally important determinant of satisfaction with health services, whereas infrastructure and coverage are not. Infrastructure, on the other hand, seems to matter for user satisfaction with public school in poor areas. Local governments seeking to increase satisfaction with health facilities may therefore need to concentrate on 24For services that are rarely used, like courts in many developing countries, a random household sample will need to be very large to ensure that a reasonable number of users are included, which can be impractical or too costly. 31 improving the extended network, rather than on coverage and physical infrastructure of the main puskesmas. Improving the infrastructure of schools in poor areas, on the other hand, is likely to improve satisfaction levels among parents of students. Secondly, the results suggest that improving the quality of human resource and medicinal inputs can be a potential priority area for the government to improve satisfaction with health services in poor districts. In education, in contrast, indicators of teacher quality matter for satisfaction in rich districts while indicators of student performance do not seem to matter at all. This should not be taken to imply that households do not attach importance to the quality of education offered by a school; instead, it may be the case that these indicators do not reflect the aspects of "quality" the households care most about. The results do suggest, however, that factors other than the directly measurable indicators of school quality are important for satisfaction among parents. Thirdly, a greater degree of community participation in the decision-making processes for facilities appears to improve satisfaction with public health and school facilities alike, albeit with significant differences between rich and poor districts. Notably, in rich districts, satisfaction with schools also significantly improves with the greater implementation of the BOS program (whose key features include school-based management, allocations of funds to schools, participatory planning and budgeting). Thus increased decentralization of service delivery appears to improve user perceptions about school quality, but only among the better- off districts. The indicators related to the participatory decision-making and extent of decentralization may partly reflect the aspects of "quality" of a school that are valued by users, or that households attach an intrinsic value to being involved in the management of the facilities. Why these indicators matter for satisfaction, what explains the variations between rich 32 and poor areas, and what that implies for the priorities of a government are important questions for future research. Finally, satisfaction with public schools is significantly correlated with the household's location, with satisfaction levels in poor districts likely to be significantly lower in regions outside of Java. This suggests the need for prioritization in efforts to improve the quality of education, particularly in the poor districts of these regions. Satisfaction with public health facilities, on the other hand, is likely to be much higher in urban areas than in rural areas, which suggests that health services in rural areas to be a clear area of concern. We conclude by noting that the useful results from the GDS-2 data should diffuse some of the skepticism regarding the utility of satisfaction surveys. Although perception-based satisfaction data do not lend themselves easily to direct interpretation, when used with care they can be effective in providing insights to policymakers on the quality of services and citizens' priorities and for evaluating the impact of reforms such as decentralization. Analysts have a key role to play in interpreting the data using appropriate models and estimation techniques, given the inherently subjective nature of such data. 33 Figure 1: High satisfaction with health and education services in Indonesia Health services Education Services 100 100 % of households 80 80 58.1 60 60 50.2 32.2 40 40 30.1 20 7.7 20 10.7 7.2 1.2 0.8 1.7 0 0 NA NA Unsatisfied unsatisfied Satisfied Satisfied Satisfied Satisfied Satisfied unsatisfied Quite Q uite Not Quite Quite Source: GDS-2 (2006) 34 Figure 2: Positive Perceptions of changes in the quality of health and education services in the last two years in Indonesia Health services Education Services 100 100 71.7 72.9 80 80 60 60 40 40 20.6 20 20 14.1 5.8 7.2 3.0 5.8 0 0 Better Same Worse NA Better Same Worse NA Source: GDS-2 (2006) 35 Figure 3: Expectancy disconfirmation model Negative Disconfirmation Latitude of Expectation Positive Disconfirmation (Not Satisfied) (Quite Satisfied) (Satisfied) (-) (+) Minimum Expected Desired Expected Standard Standard 36 Table 1: Distribution of binary variable for satisfaction Binary Health Education variable services services S=1 58.1% 50.2% S=0 41.9% 49.8% Total 100 100 Note: S=1 if code=1; S=0 if code=(2,3,4,5) Source: GDS-2 (2006) 37 Table 2: Discrete choice models (Probit) for satisfaction with health services Full sample Poor districts Rich districts VARIABLES (1) (2) (1) (2) (1) (2) Dimensions of facility quality Index for coverage -0.006 -0.006 -0.084* -0.067 0.012 0.010 Index for support 0.059*** 0.060*** 0.081* 0.078* 0.052** 0.055*** Index for quality of inputs 0.058*** 0.064*** 0.071* 0.110*** 0.024 0.020 Index for infrastructure 0.007 0.001 0.001 -0.010 0.022 0.018 Level of discrimination -0.153 -0.176 -0.363 -0.302 -0.074 -0.086 Speed of service 0.223*** 0.202*** 0.209* 0.105 0.239*** 0.232*** Waiting time -0.003*** -0.003*** -0.003 -0.003 -0.004*** -0.003*** Institutional and governance environment Provider responsiveness 0.307*** 0.325*** 0.414*** 0.454*** 0.186 0.224* Index for participation 0.073*** 0.073*** 0.139*** 0.142*** 0.033 0.030 Components of expectation Age of the household head 0.006** 0.006 0.005 Gender: female 0.185*** 0.233* 0.099 Level of education: -0.053 -0.244 -0.079 Primary Junior High -0.287** -0.676*** -0.181 Higher education -0.222* -0.143 -0.304* Religion: Catholic -0.212* -0.200 -0.303 Christian -0.079 -0.441** 0.062 Other 0.211 -0.016 0.178 Log per capita monthly 0.032 -0.040 0.066 exp. Elite/Association with -0.153* -0.326** -0.059 elites Index for sources and info -0.036 -0.138*** 0.001 Index for expectation -0.089*** -0.150*** -0.055** Location dummies Urban area 0.279*** 0.354*** 0.270 0.267 0.261*** 0.303*** Region: Kalimantan -0.013 -0.041 -0.351 -0.299 -0.002 -0.081 NTT 0.212 0.075 -0.024 -0.079 0.274 0.205 Sulawesi 0.093 0.024 -0.194 -0.161 0.170 0.124 Sumatra -0.063 -0.081 -0.386 -0.264 -0.073 -0.136 Constant -0.792 -0.234** 0.453 -0.071 -1.108 -0.191 Observations 1786 1786 682 682 1104 1104 Note: *** 1% level of significance, ** 5% level of significance, * 10% level of significance 38 Table 3: Discrete choice models (Probit) for satisfaction with education services Full sample Poor districts Rich districts VARIABLES (1) (2) (1) (2) (1) (2) Dimensions of facility quality Index for infrastructure 0.027 0.030 0.086*** 0.082*** -0.007 -0.005 Index for teacher quality 0.029 0.033 -0.037 -0.028 0.066** 0.066** Index: student -0.018 -0.021 -0.032 -0.031 -0.026 -0.024 performance Index for student 0.008 0.006 0.086** 0.073* 0.001 -0.001 coverage Institutional and governance environment Index for BOS 0.049** 0.051** -0.003 0.007 0.085*** 0.089*** Provider responsiveness 0.007 0.079 0.146 0.206* -0.125 -0.015 Index for participation 0.109* 0.102 -0.081 -0.086 0.211** 0.231*** Components of expectation Age of the household -0.001 -0.001 -0.003 head Gender: female 0.025 0.050 0.024 Level of education: 0.029 -0.072 0.197 Primary Junior High -0.108 -0.404** 0.191 Higher education -0.152 -0.316* 0.040 Religion: Catholic -0.068 0.178 -0.233 Christian 0.068 0.116 0.019 Other -0.116 0.285 -0.139 Log per capita monthly -0.014 -0.049 0.004 exp. Elite/Association with -0.142** -0.241** -0.052 elites Index for sources and 0.036 0.151*** -0.020 info Index for expectation -0.050*** -0.036 -0.059** Location dummies Urban area -0.124 -0.114 -0.109 -0.035 -0.114 -0.108 Region: Kalimantan -0.230*** -0.260*** -0.536*** -0.540*** -0.015 -0.034 NTT -0.068 -0.112 -0.238 -0.198 -0.234 -0.289 Sulawesi -0.221** -0.241*** -0.480*** -0.522*** 0.036 0.034 Sumatra -0.349*** -0.369*** -1.122*** -1.160*** -0.147 -0.135 Constant 0.464 0.198** 1.276* 0.485*** -0.068 -0.054 # Observations 2557 2557 1094 1094 1463 1463 Note: *** 1% level of significance, ** 5% level of significance, * 10% level of significance 39 REFERENCES Ahmad, Junaid, Shantayanan Devarajan, Stuti Khemani and Shekhar Shah. 2005. "Decentralization and Service Delivery." Policy Research Working Paper 3603, World Bank, Washington, DC. Amin, Samia, Jishnu Das and Markus Goldstein. 2007. Are You Being Served? New Tools for Measuring Service Delivery. Washington DC: World Bank. Amin, Samia, Basab Dasgupta and Emmanuel Skoufias. 2008. "(Dis)Satisfied with Services? The Information Content of Perception-Based Surveys". Mimeo. Washington DC: World Bank. Banerjee, Abhijit, Angus Deaton and Esther Duflo. 2004. "Wealth, Health, and Health Services in Rural Rajasthan". The American Economic Review. 94 (2): 326-330. Bertrand, Marianne, and Sendhil Mullainathan. 2000. "Do People Mean What They Say?" Implications for Subjective Survey Data," Mimeo, University of Chicago, 2000. Boulding, William, Ajay Kalra, Richard Staelin, and Valerie A. Zeithaml. 1993. "A Dynamic Process Model of Service Quality: From Expectations to Behavioral Intentions." Journal of Marketing Research. 30 (1): 7-27. Brown, Karin and Philip Coulter. 1983. "Subjective and Objective Measures of Police Service Delivery." Public Administration Review 43 (1): 50-58. Cardoso, Richard. 1965. "An Experimental Study of Consumer Effort, Expectation, and Satisfaction." Journal of Marketing Research 2 (3): 244-49. DeHoog, Ruth, David Lowery, and William Lyons. 1990. "Citizen Satisfaction with Local Governance: A Test of Individual, Jurisdictional, and City Specific Explanations." Journal of Politics 2(3): 807-37. Deichmann, Uwe and Somik V. Lall. 2003. "Are You Satisfied? Citizen Feedback and Delivery of Urban Services." Policy Research Working Paper 3070, World Bank, Washington, DC. Duffy, Bobby. 2000. "Satisfaction and Expectations: Attitudes to Public Services in Deprived Areas." CASE Working Paper 45, Center for Analysis of Social Exclusion, London School of Economics, London, UK. Federal Bureau of Statistics. 2007. Pakistan Social and Living Standards Measurement Survey Report. Government of Pakistan. Islamabad. Grindle, Merilee. 2007. Going Local: Decentralization, Democratization and the Promise of Good Governance. Princeton and Oxford: Princeton University Press. Heckman, J. 1979. "Sample selection bias as a specification error". Econometrica 47: 153­61 Johnson, Michael, Eugene Anderson and Claes Fornell. 1995. "Rational and Adaptive Performance Expectations in a Customer Satisfaction Framework." The Journal of Consumer Research 21 (4): 695-707. Kelly, Janet and David Swindell. 2000. "Linking Citizen Satisfaction Data to Performance Measures: A Preliminary Evaluation." Public Performance and Management Review 24 (1): 30-52. Lankford R., E. Lee E. and J. Wyckoff. 1995. "An Analysis of Elementary and Secondary School Choice". Journal of Urban Economics 38(2): 236-251 Lewis, Blane. D., and Daan Pattinasarany. 2009. "Determining Citizen Satisfaction with Local Public Education in Indonesia: The Significance of Actual Service Quality and Governance Conditions," Growth and Change, Vol. 40(1) (March): 85-115. 40 Licari, Michael, William McLean and Tom Rice. 2005. "The Condition of Community Streets and Parks: A Comparison of Resident and Nonresident Evaluations." Public Administration Review 65 (3): 360-68. Nelson, Philip. 1970. "Information and Consumer Behavior". Journal of Political Economy 78(2): 311-329. Oliver, Richard. 1997. Satisfaction: A Behaviorial Perspective on the Consumer. New York, NY: Irwin-McGraw-Hill. Oliver, Richard. 1980. "A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions." Journal of Marketing Research 17 (4): 460-69. Parks, Roger. 1984. "Linking Objective and Subjective Measures of Performance." Public Administration Review 44 (2): 118-27. Serra, George. 1995. "Citizen-Initiated Contact and Satisfaction with Bureaucracy: A Multivariate Analysis." Journal of Public Administration Research and Theory 5 (2): 175-188. Stipak, Brian. 1979. "Citizen Satisfaction with Urban Services: Potential Misuse as a Performance Indicator," Public Administration Review 39 (1): 46-52. Van Ryzin, Gregg. 2004. "Expectations, Performance, and Citizen Satisfaction with Urban Services." Journal of Policy Analysis and Management 23:433-448. World Bank. 2005. Cambodia ­ Public Expenditure Tracking Survey (PETS) in Primary Education. Report No. 34911-KH. 41 APPENDIX A Table A-1: Distribution of households across different heath facilities Type of health facility Facility/secondary information Total Not Available Available Puskesmas and ancillary facilities 2,089 2,269 4,358 Puskesmas 613 2,253 2,866 Pustu 1,242 15 1,257 Polindis 212 0 212 Pusling 22 1 23 State General hospital 333 3 336 Private hospital 198 0 198 Private clinic 138 0 138 Private doctor 800 4 804 Private midwives 1,299 1 1,300 Private nurse 1,030 0 1,030 Never used 117 0 117 No information for most frequented facilities NA NA 263 Total 6,004 2,277 8,544 Note: 2269 households that reported Puskesmas and ancillary facilities as the most frequently used facilities for which facility data was available were used for the analysis Source: GDS-2 (2006) i Table A-2: Distribution of households across different education facilities Type of education facility Facility/secondary information Total Not available Available Public facilities 1,870 2,452 4,322 Public Elementary 1,008 1,410 2,418 Public Junior High 484 659 1,143 Public Senior High 345 354 699 Public Diploma/College 33 29 62 Private Elementary 319 1 320 Private Junior High 154 4 158 Private Senior High 104 1 105 Private Diploma/College 9 0 9 Other 466 497 963 Total 2,922 2,955 5,877 Source: GDS-2 (2006) ii Table A-3: School facilities and coverage across rich and poor districts Indicators of quality Rich Districts Poor Districts difference Average number of teachers 13.44 10.66 2.78** Average teaching hrs per teacher per week 23.85 25.4 -1.55** Average years of experience per teacher 9.60 8.44 1.16** Proportion of good theory room to total 0.65 0.56 0.09** Proportion of good computer room to total 0.12 0.06 0.06** Proportion of good sports hall 0.67 0.50 0.17** Average number of registered students 37.30 31.60 5.70** Average number of attending students 33.01 26.94 6.07** Level of satisfaction (0, 1) % of households (with child in school) satisfied with public education facility 54.47 54.23 0.24 Source: GDS-2 (2006) iii APPENDIX B: Description of all variables used in regressions Welfare status: Per capita monthly household expenditure (in Indonesian currency) Demographic variables Age of the household head in years Gender of the household head (female=1) Dummies representing level of education of the household head Up to Primary school (=1) Up to Junior High school (=1) Higher education (=1) Dummies representing Catholic (=1) Christian (=1) Other religion (includes Hindu, Buddhist and other) (=1) Household head's social status as elite or association with them (=1) Household composition Percentage of children in the family below 5 years Percentage of male members in the family between 20 to 59 years of age Percentage of male members in the family between 20 to 59 years of age Location dummy: Dummy for urban area (=1) Regional dummies (Java as reference): Kalimantan, NTT, Sulawesi, Sumatra Household's experience (specific to health facilities) about Discrimination: Did you or other HH member experience discrimination during services (=1)? Speed of services: Was the service faster, in accordance with, or longer than what you or other HH member expected (=1)? Waiting time: How long did you or other HH member have to wait until they received services (in minutes)? Health cost: What was the total cost that had to be paid you or other HH member for services (excluding transportation costs)? Provider's responsiveness: the follow-up from the management against criticism, complain and advice? II. Description of variables created using Principal Component Analysis (PCA) The components of all indices that are created using PCA are given below, separately for education and health regressions. Health Index representing coverage (pc1_coverage): Total area covered by the puskesmas (public health facility). Total population served by the puskesmas. Number of households served by the puskesmas. iv Number of kelurahan served. Index representing support (pc1_support) Number of puskesmas support (ancillary facilities) per thousand population served. Number of village maternity houses (polindes) per thousand population served. Number of medicine posts per thousand population served. Index representing service (pc1_service) Proportion of doctors in total medical staffs in the puskesmas. Proportion of nurse in total medical staffs in the puskesmas. Proportion of dentists in total medical staffs in the puskesmas. Average hours of services by doctors as a proportion to that by medical staff Average hours of services by dentists as a proportion to that by medical staff Average hours of services by nurses as a proportion to that by medical staff Dummy variable: if any medical staff has outside private practice. Number of doctors with PNS type employment. Dummy variable to represent number of weeks the puskesmas ran out of stock of vaccine Dummy variable to represent number of weeks the puskesmas ran out of stock of medicine Location of where health service is provided Index representing infrastructure (pc1_infra) Number of bathrooms in the puskesmas per thousand population served. Number of generators in the puskesmas per thousand population served. Number of computers in the puskesmas per thousand population served. Number of beds for public treatment in the puskesmas per thousand population served. Number of delivery beds in the puskesmas per thousand population served. Number of patient used inpatient facilities in the puskesmas per thousand population served. Number of beds days generated in the puskesmas per thousand population served. Index representing participation (pc1_participation) whether the household shared ideas in meetings for health programs. whether the household was present in meetings for health programs. whether the household contributed money for health programs. Index representing expectation (pc1_exp) [only used for probit model (2)] Household Head's age, gender (female=1), level of education (primary, junior high and higher education), religion, social status as elite. Per capita monthly expenditure, Whether the household knows about any health program for improvement of health services. Whether the household has information about kelurahan budget Whether the household has information about kelurahan development plan. Whether the household has information about corruption in health /education service Whether the household has information about bribery in health/education service. Source of information: radio Source of information: TV Source of information: News paper Source of information: Lurah/village head, subdistrict head etc. Index for village level information and source (pc1_source) v Whether the household knows about any health program for improvement of health services. Whether the household has information about kelurahan budget Whether the household has information about kelurahan development plan. Whether the household has information about corruption in health service Whether the household has information about bribery in health service. Source of information: radio Source of information: TV Source of information: News paper Source of information: Lurah/village head, subdistrict head etc. Education Index for participatory mode of management (Pc1_participation) Decision about school's mission and vision was made together by principal and the teachers Decision about school's mission and vision was made together by principal, teachers and community. Index for village level information and source (Pc1_source) HH has information on kelurahan budget HH has information on kelurahan Development Plan HH has information on kelurahan BPD HH has information on any complaint delivered to Kelurahan. Sources of information: radio Sources of information: TV Sources of information: News paper Index representing expectation (pc1_exp) [only used for probit model (2)] Household Head's age, gender (female=1), level of education (primary, junior high and higher education), religion, social status as elite. Per capita monthly expenditure, Whether the household has information about kelurahan budget Whether the household has information about kelurahan development plan. Whether the household has information about corruption in education service Whether the household has information about bribery in education service. Source of information: radio Source of information: TV Source of information: News paper Source of information: Lurah/village head, subdistrict head etc. Index for teacher quality (Pc1_teacher) Number of teachers Average term of a teacher Average teaching hours Teacher to student ratio Average years of experience of a teacher Index for school infrastructure (Pc1_facilities) Proportion of good theory rooms to total theory rooms Proportion of good computer rooms to total computer rooms vi Proportion of good library rooms to total library rooms Proportion of good sports hall to total sports hall Whether the school has proper electricity Books available per student Total number of class rooms Index for student performance (Pc1_stud_quality) Total male dropouts Total female dropouts Total male repeats Total female repeats Index for coverage (Pc1_stud_coverage) Number of registered students Number of attending students Index for BOS coverage (Pc1_bos_coverage) Whether school based management is being implemented Prepared school development plan together Prepared school budget together Received adequate information on BOS Whether socialization for BOS was adequate Number of BOS disbursement Whether BOS amount was adequate Number of poor students covered under BOS vii APPENDIX C: Robustness of results to selection bias: Heckman two-stage model for selection bias The possibility of selection bias arises from the fact that the sample of households for which we have corresponding facility data is defined non-randomly through a combination of self- selection and the method of sample selection in GDS-2. To correct for this bias in our results on the determinants of satisfaction with facilities, we estimate a revised model where: (1) household and community level factors and regional and rural/urban location (fixed effects) determine the choice of the particular (health or education) service provider; (2) conditional on the choice of a provider, reported satisfaction with the service facility is a function of indicators of actual quality of the facility and governance, as well as some household characteristics and regional and rural/urban location. The stage-2 equations are nearly identical to the reduced form probit models of the corresponding cases.i Analogous to the reduced form models, we run the 2-stage models separately for samples from poor and rich districts, and in a pooled sample with all districts, for satisfaction with health and education facilities separately. The assumption stated earlier ­ that the services from a facility is an "experience" good, whose actual quality is known to only those who have used that facility ­ implies that all variables measuring facility quality and governance enter the stage-2 regression that estimates satisfaction, but not the stage-1 selection model. Variables that enter into the stage-1 selection equation but not stage-2 are: variables related to household composition by age and gender (for i There is but one exception: the stage-2 models for education do not include variables on household head's religion, whereas the reduced form probits for education do so. This is however a minor difference, since these variables are highly insignificant in the reduced form probit. The reason for not including them in the 2nd stage of the selection model is that they are good candidates for "instruments" in the selection model (see footnote ii below). viii facility choice in both health and education), whether a facility is in the village or not and fees per visit to the facility (for choice of health facility only), and religion of the household head (for choice of education facility only). These variables influence facility choice of a household, but do not have significant effect on with satisfaction with a facility conditional on that choice.ii The results from the 2-stage Heckman selection model for satisfaction with health and education are presented in Tables C-4 and C-5 respectively. In each case, the model is run for the pooled sample and for samples from rich and poor districts, analogous to the reduced form probit models. The results for the determinants of satisfaction (the 2nd stage model) are quite similar to those for the corresponding reduced form probit models (comparing Table C-4 and Table C-5 with Tables 2 and 3). This is consistent with the fact that the coefficient on inverse Mills ratio () is statistically insignificant in the 2nd stage regression for most cases in health and education alike (the pooled sample for health is the only case where it is weakly significant). This implies that the null hypothesis that the coefficient on is zero (no bias due to sample selectivity) cannot be rejected with 95 percent level of confidence in all cases, and can be rejected with 90 percent level of confidence in only one case. The results are also quite robust to changes in specifications of the 1st stage selection model in all cases. iiThese variables play the role of "instruments" in the 2-stage Heckman selection model, since they are found to be not significant in the 2nd stage probit, while being significant in the 1st stage selection model in most cases. ix Table C-4: Two-stage Heckman selection model for satisfaction with public health facilities Pooled districts Poor districts Rich districts 2nd 2nd 2nd stage Selection stage Selection stage Selection VARIABLES model* model* model model model model Index for coverage -0.001 -0.032* 0.005 Index for support 0.019*** 0.031* 0.016** Index for services 0.022*** 0.024 0.010 Index for infrastructure 0.003 0.000 0.010 HH experience about discrimination -0.073 -0.081 -0.119 -0.467*** -0.030 0.067 HH experience about speed of service 0.089*** 0.069** 0.068* 0.044 0.100*** 0.099** HH experience about waiting time -0.001*** 0.001** -0.001 -0.003*** -0.001** 0.004*** Provider's responsiveness 0.117*** 0.022 0.146*** 0.022 0.074 0.057 Index for participation 0.024*** -0.027** 0.052*** -0.054*** 0.011 -0.007 Age of household head 0.002** 0.004*** 0.002 -0.004* 0.002* 0.008*** Gender of Household head 0.054** -0.111*** 0.0921* -0.229*** 0.028 -0.053 Level of education: primary -0.025 -0.114* -0.087 -0.336*** -0.026 0.058 Level of education: junior high -0.114** -0.051 -0.243*** -0.369*** -0.062 0.137 Level of education: higher education -0.096** -0.136** -0.045 -0.383*** -0.116* 0.036 Religion dummy (Muslims as reference) Catholic -0.070 0.096 -0.076 0.413*** -0.108 0.064 Christian -0.024 0.072 -0.160** 0.078 0.036 0.171** Other religion 0.101* 0.386*** 0.012 -0.018 0.089 0.365*** Log of per capita monthly exp. 0.003 -0.127*** -0.022 -0.105*** 0.010 -0.209*** Elite/Association with elite -0.063** -0.091** -0.114** -0.073 -0.025 -0.093 Index for sources of information -0.013 0.013 -0.051*** -0.010 0.001 0.024 Dummy for urban area=1 0.114*** 0.250*** 0.102 0.243** 0.106*** 0.146*** % of male members bet.20-59 yrs 0.285** 0.979*** -0.086 % of female members bet.20-59 yrs 0.037 0.479** -0.264* Health cost -0.000*** 0.000 -0.000*** Location dummy for puskesmas inside the village=1 -0.520*** -0.918*** -0.366*** Region dummy (Java as reference) Kalimanthan 0.021 0.384*** -0.140* 0.358*** 0.017 0.274*** NTT 0.084 0.120 -0.031 0.243** 0.131 0.472*** Sulawesi 0.074 0.585*** -0.102 0.956*** 0.084 0.210** Sumatra -0.006 0.280*** -0.165* 0.759*** -0.022 0.068 Mill's ratio: lambda 0.105* -0.029 0.103 Constant 0.172 0.659** 0.791** 0.653 0.092 1.547*** x Observations 6737 6737 2720 2720 4017 4017 Uncensored obs 1779 680 1099 Note: *** 1% level of significance ** 5% level of significance * 10% level of significance Dependent variables: First stage selection model: choice of public health facilities with available objective information; second stage: binary variable of satisfaction with facility xi Table C-5: Two-stage Heckman selection model for satisfaction with public school facilities Pooled districts Poor districts Rich districts 2nd stage Selection 2nd stage Selection 2nd stage Selection VARIABLES model* model* model* model* model* model* Index for infrastructure 0.00804 0.0301** -0.00607 Index for teacher quality 0.0112 -0.014 0.0252** Index: student coverage -0.000766 0.0324** -0.00674 Index: student performance -0.0085 -0.0121 -0.013 Index for BOS 0.0195** 0.000295 0.0326*** Index for participation 0.0422* -0.0348 0.0873*** Provider's responsiveness -0.00114 0.183*** 0.0687* 0.219*** -0.0733 0.172** Age of the household head -0.000467 -0.00053 -0.000136 -0.000923 -0.00114 -4.75E-06 Gender: Female 0.0133 0.00736 0.021 0.0274 0.0207 -0.0152 Level of education: Primary 0.00528 0.0915 -0.0197 0.0921 0.0522 0.103 Junior High -0.0437 0.115 -0.142** 0.0961 0.0534 0.154 higher education -0.0568 -0.031 -0.116* -0.0341 0.0158 0.00181 Index: sources of information 0.0151 -0.0292* 0.0522*** -0.0488* -0.00255 -0.0305 Elite/association with elites -0.0537** -0.0526 -0.0990** -0.1 -0.0194 0.0096 Log of percapita monthly exp. -0.00671 0.0242 -0.00967 0.107*** 0.0184 -0.108*** Religion: Catholic -0.535*** -0.678*** -0.292** Christian -0.234*** -0.349*** -0.175* Other religion -0.0971 -0.546** 0.0387 % children in HH below 5 yrs -0.146 -0.382* 0.0447 % of male in HH betn. 6-12 yrs -0.0677 -0.501** 0.235 Urban location dummy -0.0384 -0.459*** -0.0438 -0.0791 0.0441 -0.526*** Region dummies (Java as reference) Kalimanthan -0.0959** 0.325*** -0.193*** 0.0881 -0.0803 0.488*** NTT -0.0307 0.151** -0.0629 0.287*** -0.11 0.229 Sulawesi -0.0796** 0.281*** -0.158*** 0.201** -0.0613 0.607*** Sumatra -0.135*** 0.241*** -0.391*** 0.317*** -0.0956 0.341*** lambda -0.0458 0.106 -0.259 Constant 0.729*** -0.404 0.780** -1.248*** 0.504* 0.982** Observations 5458 5458 2315 2315 3143 3143 Uncensored obs 2541 1091 1450 Note: *** 1% level of significance ** 5% level of significance * 10% level of significance Dependent variables: First stage selection model: choice of public school facilities with available objective information; second stage: binary variable of satisfaction with facility xii