WPS4017 Do Subsidized Health Programs in Armenia Increase Utilization among the Poor? Diego F. Angel-Urdinola and Shweta Jain World Bank Abstract This article analyzes the extent to which the Basic Benefit Package (BBP), a subsidized health program in Armenia, increases utilization and affordability of outpatient health care among the poor. We find that beneficiaries of the BBP pay approximately 45 percent less in fees for doctor visits (and display 36 percent higher outpatient utilization rates) than eligible users not receiving the BBP. However, even among BBP beneficiaries the level of outpatient health care utilization remains low. This occurs because the program mainly provides discounted fees for doctor visits, but fees do not constitute the main financial constraint for users. Our estimates suggest that other non-fee expenditures, such as prescription medicines, constitute a more significant financial constraint and are not subsidized by the BBP. As a result, outpatient health care remains expensive even for BBP beneficiaries. World Bank Policy Research Working Paper 4017, September 2006 The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org. The authors benefited from valuable feedback from Aleksandra Posarac, Cem Mete, Panagiota Panopoulou, Emil Tesliuc, and participants to the ECSHD (Europe and Central Asia Human Development) Brown Bag Lunch series. 1. Introduction This article analyzes the extent to which the Basic Benefit Package (BBP), a subsidized health program in Armenia, increases health care utilization and affordability among health users. Health care utilization rates in Armenia have been declining at alarming rates. According to the WHO (2006), while in the early 1990s outpatient utilization in Armenia was at 7 contacts per person per year, in 2005 it had declined to 2 contacts per year (see Figure 1). This indicator is below the regional average, which oscillates between 6 and 10 contacts per person per year. Inpatient utilization is also very low. While inpatient care admissions in the region vary between 15 and 20 per 100 inhabitants per year, in Armenia the rate is less than 8 per 100 inhabitants (i.e. about half of the regional average compared to the 1990 or 1995 level). Declining utilization has occurred mainly as a consequence of a quasi-privatization of health services in Armenia after the end of the Soviet era, which permitted hospitals and doctors to generate revenues by selling health services to the public. Decreasing levels of health care utilization in Armenia may also reflect both the low level of public expenditure on health and affordability constraints (Armenia National Statistical Services and World Bank, 2006; Angel-Urdinola et al., 2006; Bonilla-Chacin et al., 2005; Hayrapetyan and Khanjian, 2004). According to Armenia National Statistical Services and World Bank (2006), household survey estimates suggest that while public expenditure on health as a percentage of GDP in Armenia was at 1.32 percent in 2004 (one of the lowest in the Europe and Central Asia, ECA, region), private expenditure as a percentage was at 4.55 percent of GDP. This is a reflection of the fact that health service patients in Armenia generally pay for their treatment "out-of pocket". One could argue that health care utilization in Armenia has reached efficiency and that the higher rates of utilization that existed in the past were a result of excessive usage of the system during the Soviet period. It may also be that the system may have become underutilized as affordability constraints forced sick individuals to seek informal care or not to seek care at all. Angel-Urdinola et al. (2006) find that expenditures on health among poor 2 households with at least one health service recipient account for 70 percent of their total non-food consumption. In other words, such households have limited resources for alternative non- food/non-health consumption expenditures. This feature is not exclusive among the poor. Non- poor households with health care recipients also claim to spend about 47 percent of their overall non-food consumption on health (equivalent to roughly 30 percent of their income). In response to decreasing utilization, the government of Armenia has designed a social assistance program to provide health services (free-of-charge or highly subsidized) to the poor and most vulnerable groups of the population.1 International experience suggests that public health care subsidies may contribute to higher utilization and improve health outcomes such as under-five-mortality (Newman et al., 2002; Alderman and Lavy, 1996; Lavy et al., 1996, and Mwabu et al., 1993). Furthermore, Gertler et al. (1989) argue that the price elasticity of the demand for health care decline as income rises, which implies that user fees tend to reduce access to care proportionally more for the poor than for the rich. The Basic Benefits Package was introduced in 1998. Since January 2001, the government has extended the free-of-charge BBP program to all beneficiaries of the poverty family benefit system, which is a government adopted means-tested benefit program that provides a series of health and non-health services to the poor. The BBP package comes in two forms: the first targets a segment of the population considered vulnerable (based on an eligibility criteria), providing them with free health care services at all levels, with the exception of some specific costly services (e.g. transplants) and less essential services (e.g. expensive dental services and cosmetic surgery). The second, more limited, type of BBP is provided to the non-vulnerable or general population. This package requires co-payments for certain home visits, antenatal and postnatal care (provided by gynecologists or nurses), a large part of dispensary outpatient care, and a selection of hospital services. This article focuses on the first type of BBP assistance only. 1 The program targets families of disabled persons, war veterans, single-parent and orphaned children under the age of 18, disabled children under the age of 16, families with four or more children under the age of 18, prisoners, participants of the Chernobyl disaster liquidation activities, among others. 3 In principle, BBP recipients should get free health care at hospitals, while the rest of the users pay fees (no official co-payments apply to the first type of BBP services). Some exceptions apply to emergency treatment, sexually-transmitted diseases (STDs), tuberculosis (TB), and malaria. In practice, due to funding shortages, BBP beneficiaries continue to pay out-of-pocket for medicines and for informal payments (known as under-the-table payments) to doctors and practitioners for treatment. Informal payments are common because government reimbursements for BBP services, which are transferred from the State Treasury directly to health care facilities, are generally low and take time to disburse. Estimates (HERA, 2004) claim that 30 to 50 percent of the costs of BBP treatments are actually reimbursed. In this article we use data from the 2004 Armenia Integrated Living Standard Survey (ILSC) to analyze the extent to which access to poverty family benefit system (and thereby to free-of-charge BBP) promotes health care utilization. Previous research using household survey estimates (Murrugarra et. al, 2004) found that patients eligible for free services under BBP displayed higher utilization rates that non-eligible ones. The authors simulated program eligibility based on a means-testing algorithm. However, not all eligible users actually get BBP. Data used by the authors did not permit them to identify whether individuals were actually collecting benefits. In practice, only those who register and pass the eligibility criteria receive benefits. Besides providing data necessary to infer about eligibility to the program, the 2004 ILSC data provide information on whether families are registered in the poverty family benefits system and, if registered, on whether they actually receive benefits or not. As such, ILCS 2004 data allows making a more accurate comparison of the impact of the program based on better-defined treatment and control groups (e.g. eligible and registered users getting benefits vs. eligible and registered users not getting benefits). Data from the 2004 ILSC provides an ideal scenario to perform a quasi-random evaluation of the program. First, observable characteristics between users in the treatment and control groups are likely to similar because the program administration determines eligibility based on a points-based-score according to the socio-economic 4 characteristics of the applicants (see Appendix). Second, households who register to receive the benefits are also likely to have similar unobservable characteristics since registration to the program occurs through self-selection. Finally, data suggest that some of the users who should get the benefit (i.e. those registered and eligible) do not. This may occur because the program administration in some localities displays long processing-times and/or capacity constraints. Although fully randomized social experiments are considered more appropriate for program impact evaluation (Grossman, 1994; Holland, 1986; and Newman et. al, 1994), non experimental evaluations such as the one presented here provide an alternative evaluation method for programs like the BBP which are already implemented and which, due to ethical considerations, do not allow for more reliable experimental design. Fortunately, the characteristics of the data used here allow for impact evaluation on the basis of propensity score matching techniques. First, observable characteristics in terms of socio-economic status, demographics, and health status do not differ much between our treatment and controls groups. Second, our data passes favorably the balancing tests necessary to conduct matching (see Dehajia and Wahba, 1999 and Dehajia and Wahba, 2002) so that comparisons in outcomes between the treatment and control groups verify that the average score (or probability) between control and comparison units within each stratum are statistically the same. Due to data limitations, the results presented here focus on utilization of outpatient services only. We focus our analysis mainly on the question of whether treatment costs for doctor visits and health care utilization rates change as a result of eligibility for BBP benefits. The main findings can be summarized as follows: eligible users registered in the poverty family benefit systems who receive BBP benefits (the treatment group) pay approximately 45 percent less for outpatient treatment than eligible users registered in the program but not yet receiving benefits (the control group). As a consequence of such price discounts, eligible beneficiaries of the BBP display a 36 percent higher probability of seeking outpatient health care when needed sick as compared with non-beneficiaries. However, even among those eligible for 5 BBP outpatient utilization remains low. This happens because although the BBP relieves some financial burden in relation to doctor fees, outpatient health care requires large additional out-of- pocket expenditures (mainly on prescription medicines). The rest of the paper is structured as follows: section 2 describes the data and our methodological strategy; section 3 discusses the main results; the article ends with a brief conclusion. 2. Data and Methodology Data used for the purposes of this study comes from the 2004 Armenian Integrated Living Standard Survey (ILCS). The survey has been conducted since 1996 by the National Statistical Service of Armenia, with support from the World Bank, USAID, and other donors. The survey collects information on household characteristics, such as household assets, intra- household transfers, and housing; and on individual characteristics, such as migration, health, education, and employment. Data are stratified, nationally representative, and provide detailed information on household income and consumption. The sample of the 2004 ILCS is based on the population structure of the 2001 population census. It includes 6,816 households and 26,614 individuals. The data contains 43 urban sample units and 216 rural sample units, which makes the survey representative at the level of the eleven main regions in Armenia (or Marzs). The health module of the survey collects information on morbidity, access, and expenditures on health by type of doctor, type of expenditure (i.e. official vs. unofficial payments), type of health provider, self-perceptions of health status, and postnatal services. The following variables were used in the analysis: individual and household characteristics (such as per capita consumption, region, strata, sex, age, and employment status), cost of doctor visits, overall health expenditures, health care utilization, and type of treatment. We define morbidity as a dummy variable taking the value of 1 if the individual claimed to have been sick/injured in the past four weeks. Among those, an additional dummy was constructed for those who claimed having been seriously ill. Another important variable used is a dummy for those who were 6 seeking care if they reported being sick. This helps to differentiate preventative care from required care. Individuals who received any treatment when sick reported to have used one of the following providers: doctors, healers, home treatment (self-prescription), private physicians, and hospitals. Given the limited sample size, we were unable to gather detailed information for those who were hospitalized or used a healer (among individuals registered for poverty family benefits only 64 were hospitalized and 24 received heath treatment from a healer). Data from the 2004 ILCS allows simulating whether or not households are entitled to receive family poverty benefits according to a means testing algorithm that defines eligibility (see appendix).2 Based on this algorithm, we constructed a dummy variable that takes the value of one if a user is eligible for family poverty benefits and the values zero if otherwise. Our sample is delimited to those users identified as being eligible. To identify a control group we look at whether "eligible" households are a) registered to receive BBP benefits and b) whether they actually get the benefits. Users with access to BBP were identified using two primary questions: the first identifies whether a user lives in a household registered to get family poverty benefits, and the second asks whether the individuals in the household actually receive the benefits. In relation to expenditures on health, the survey contains detailed questions on the cost of treatment received from different providers. Since the survey was conducted over a 12-month period, the comparability of the quintiles should not be affected by seasonal fluctuations. Sample size and simple descriptive statistics are presented in Table 1. Our analysis is conducted on those individuals eligible for family poverty benefits who are registered for the BBP (3,316 individuals). About 77 percent of them (2,422 individuals) actually receive BBP benefits while the remaining 13 percent (714 individuals), being registered, do not yet receive benefits. Given that rural areas are more vulnerable to poverty, it is not surprising that eligible 22004 ILCS data allow to replicate the main variables needed to estimate household eligibility scores for the poverty family benefits system in Armenia. However, data do not contain information to estimate some of parameters included in the eligibility formula (they are Pr and Pf; see Appendix for more details). As such, while our eligibility dummy is a good proxy for actual eligibility, it may contain some errors of inclusion and exclusion that are beyond our control. 7 families registered (receiving and not receiving) benefits are mainly living in rural areas (41 and 57 percent respectively). Only 18.8 percent of the eligible BBP beneficiaries live in Yerevan ­ the capital. The proportion of eligible users who are employed and registered but who do not receive benefits is higher than that of employed individuals receiving benefits (36.15 vs. 21.7 percent). This result is somehow expected because households under situations of shock, such as unemployment, or facing constraints to participate in the labor force, such as disability, are more likely to be granted assistance. The average age of the individuals in the treatment group is 37 years old vs. 44 in the control group; 40 percent of the individuals in the treatment group are males vs. 42 percent in the control group. About 33 (40) percent of all individuals in both the treatment (control) groups declared to have been sick in the past four weeks. Among those in the treatment (control) group, about 42.21 (32.92) percent were severely ill and about 54 (51.5) percent received any treatment. Among those receiving medical treatment, 26.9 (23.1) percent in the treatment (control) group visited the doctor and 2.91 (3.93) percent were hospitalized. The main empirical question consists in testing whether utilization rates and cost of outpatient treatment differs among individuals in the treatment and control groups. To answer the question we rely on propensity score matching techniques. The results presented above seem to indicate that morbidity, utilization rates, and cost of treatment does not differ much among individuals in the treatment and control groups. Such results are likely to put in question the effectiveness of the program (i.e. the extent to which it can reduce heath care costs and increase utilization). In order to verify these rather counterintuitive results (as we would expect that targeted programs such as this one would have some impact in outcomes) we rely on propensity score matching techniques. We denote the eligible individuals registered for the basic benefit package as BBP. Let BBP = 1 for those among them who receive benefits and BBP = 0 for those not yet receiving benefits. For values of BBP equal to j = {0,1} we get a vector of outcomes Yj. The parameter of 8 interest, denoted by POI, shows the difference between outcomes of interest in the treated state (Y1) with the outcomes in the control state (Y0) conditional on receiving treatment. For any individual i, the vector of outcomes Yij (in this case, outcomes such as utilization and cost of treatment) is defined asYij = j (Z) +ij where Z denotes a vector of characteristics (observables and not observables) and denotes an error term. The average expected outcome for both the treatment and control groups can be defined as j (Z) = E(Yj | Z). We are interested in the parameter POI, which measures the effect of the program on the participants [or the average effect of treatment on the treated]: POI = E(Y1 -Y0 | BBP =1) = E(Y | BBP =1) = E(Y1 | BBP =1) - E(Y0 | BBP =1) (1) We are able to observe E(Y1 | BBP =1) from the data but not E(Y0 | BBP =1) , which is the counterfactual of interest. However, we can observe the average outcome in the control state E(Y0 | BBP = 0) , which we can use as an estimate for the counterfactual. One should expect in general that E(Y0 | BBP =1) E(Y0 | BBP = 0) because of selection bias. The central problem becomes then to obtain a good estimate for the unobservable component. We cannot simply assume that BBP is independent of the factors that influence Y. In other words, participants and non-participants are different in many ways, including the effect of the program. Therefore, E(Y | BBP =1) E(Y1 | BBP =1) - E(Y0 | BBP = 0) . (2) The last term in equation (2) can we re-written as: E(Y1 | BBP =1) - E(Y0 | BBP = 0) = (3) E(Y | BBP =1) +[E(Y0 | BBP =1) - E(Y0 | BBP = 0)] Propensity score matching provides a way to estimate E(Y | BBP = 1) under the assumption that, conditional on observable characteristics X, participation is independent of outcomes (e.g. treatment status is random conditional on X). 9 (Y0,Y1) BBP| X (4) This property in known as the conditional independence assumption (CIA). In our case, the vector X includes demographic characteristics, such as sex, age, square of age; socio- economic characteristics, such as whether or not the individual lives in poor households and whether the individual is employed or not; and the health status of the individual, proxied by whether or not the individual declares to have been sick in the four weeks prior to the interview. Using Dehajia and Wahba's (2002) methodology, tests on our data suggest than the CIA property holds. This is not surprising since, as mentioned before, the differences in mean values of variables included in X are not significant between the treatment and control groups.3 If the assumption in equation (4) holds true, then: POI(X ) = E(Y | BBP =1) = 1(X ) - 0(X ) (5) In other words, the estimated impact of the program on participation is the difference in mean Y, conditional on X, between participant and non-participants. If the CIA holds, we can estimate (5) using propensity score matching techniques (Rosembaum and Rubin, 1983) so that instead of conditioning on X we can condition on the propensity score P(X) = Pr(BBP=1|X). An advantage of this approach is that the dimension of the propensity score is one. In such a case the CIA can be expressed as: E(Y0 | BBP =1,P(X )) = E(Y0 | BBP = 0,P(X )) (6) If (6) holds, we can estimate the parameter of interest in equation (1) as follows: POI(X ) = E(Y | BBP =1,P(X )) = E(Y1 | BBP =1,P(X )) - E(Y0 | BBP = 0,P(X )) (7) To estimate (7) we rely on kernel matching so that all treated observations are matched with a weighted average of all controls that are inversely proportional to the distance between the propensity scores of treated and controls. Additionally, to check the robustness of our results we 3Stratum (urban vs. rural) dummies were excluded from the analysis because they did not satisfy the balancing property. This result was somehow expected because the sample has an intrinsic rural bias. 10 use radius and local linear matching techniques. Our estimates impose the common support restriction so that observations with matches outside of the boundaries of the common support (or probability of being treated given that they are not) are excluded. In this case, by imposing this restriction we do not lose many degrees of freedom because most of our matches happen to be within support.4 Table 2 presents the estimates of the bias in the dependent variable due to observable characteristics and provides the region of common support. To correct the bias that may arise from differences in observable characteristics between the individuals in the treatment and control groups, our propensity score matching model controls for socio-economic conditions of the individuals (whether the individual is poor and employed), health status if the individuals (whether the individual reported being ill in the past 4 weeks and whether or not he/she received treatment), and demographic characteristics of those registered in the program (such as age, square of age, and gender). Figure 2 presents the distribution of matches within support of the propensity score for the entire sample. The figure suggests good properties of our sample: first, that most matches are within support; second, the degrees of freedom in the different intervals of the propensity score are sufficient for proper matching. 3. Results Using the procedure described above we analyze three main outcomes that could be associated with the BBP benefits: cost of treatment, utilization rates, and economic burden of health utilization proxied by the share of health expenditure cost in the non-food consumption budget. Results summarized in Table 3 provide information on differences in outcomes between individuals in the treated and control groups (both unmatched and using propensity score 4 With radius matching each treated person is matched only with the control person whose propensity score falls in a predefined neighborhood of the propensity score of the treated unit. The smaller the size of the neighborhood the better is the quality of the matches, but the probability that a treated person does not find a control person would also increase. Local-linear matching is similar to kernel matching but includes a linear term for the balancing score, the results of which are helpful in cases asymmetric data. 11 matching) and display the number of observations used for comparison. The first set of results suggest that individuals in the control group (those eligible and registered in the BBP but not receiving benefits) pay 45.15 percent more for health treatment than individuals actually receiving BBP benefits. The same result holds if we restrict the analysis to those individuals who are poor but the magnitude of the effect is lower (26 percent). These results are intuitive because BBP beneficiaries pay subsidized fees while other users do not. Note that unmatched results give a different message: that all (poor) eligible BBP recipients would pay on average 26 (25) percent less (more) for treatment. These findings are robust to the choice of matching techniques as presented at the bottom of Table 3. Given that the cost of health treatment is lower for eligible BBP beneficiaries, one would expect them to have higher utilization rates. Indeed, results in Table 3 suggest that utilization rates for doctor visits are 8 percentage points higher (about 36 percent) for the treatment group as compared to the control group. A similar result holds if we delimit the sample to the poor. Yet, utilization rates for BBP beneficiaries are low at 25 percent. Low rates of utilization in developing countries are often related to affordability constraints. The last set of numbers in Table 3 present health expenditure outcomes for households in the treatment and control groups. Note that while the previous set of outcomes was estimated at the individual level, affordability outcomes are estimated at the household level (usually we expect the household head ­ and not necessarily the patient ­ to pay for health services of family members). Kernel matching results show that households spend on average about 2.5 percent of their monthly non-food consumption every time a family member visits a doctor. In total, the average household having a sick family member spends about 12 to 17 percent of their monthly non-food consumption on health. These results demonstrate that in addition to high outpatient costs, households face large non-fee related expenditures, mainly on prescription medicines. As presented in Table 3, prescription medicines (proxied by the expenses for home treatment) account for 12 percent of all non-food expenditures. Angel-Urdinola et al. (2006) find that 12 medicines in Armenia represent 25 to 32 percent of all expenditures on health among households with at least one health service recipient and constitute the main expenditure on health after expenditures on hospitalizations. 4. Conclusions This article analyzes the extent to which subsidizing health care in Armenia increases utilization among the poor. Our analysis suggests that eligible individuals registered in the family poverty benefits system who are BBP recipients pay about half of the price for doctor visits (and display higher utilization rates) than similar individuals who are not BBP recipients. In this sense, the program seems to be achieving its goals. However, there is scope for improvement since overall utilization rates remain low even among BBP recipients. This phenomenon occurs because health care treatment remains expensive (mainly because of the high cost of prescription medicines) even for those benefiting from subsidized health under the poverty family benefit system. This result is common among programs which provide financial relief for items that are not necessarily the main affordability constraint. World Bank (2005) finds that the abolition of school fees in Mozambique did not contribute to higher levels of enrollment as the affordability constraints faced by students were not associated to school fees but to high costs of uniforms and transportation to school. If the goal of the BBP is to promote utilization more aggressively, the program could subsidize heath costs beyond doctor fees. Furthermore, the government should make sure that payments for medical treatment that are transferred from the state treasury to doctors and hospitals upon the provision of services to BBP beneficiaries are more aligned with market prices. A large difference between the actual costs of treatment and those that the program disburses to medical institutions provides incentives to doctors and medical practitioners to charge large informal (under-the-table) fees. Finally, the program could consider the possibility to provide subsidies for prescription drugs, which constitute a large component of the overall expenditures on health among those households having health users. 13 References Alderman, H. and Lavi, V. (1996). Household Responses to Public Services: Costs and Quality Trade-offs. World Bank Research Observer, Vol. 11(1):3-22 Angel-Urdinola, D.; Jain, S.; and Prina, S. (2006). Social Sectors and Poverty In Armenia: From Equity in Access to Equity in Quaility. Mimeo. World Bank. Armenia National Statistical Services and World Bank (2006). Social Snapshot and Poverty in Armenia. http://www.armstat.am/Publications/2006/PV_04/PV_04_0.pdf. Yerevan. Bonilla-Chacin, M.; Murrugarra, E.; and Temourov, M. Health care during Transition and Health Systems Reform: Evidence from the Poorest CIS countries. Social Policy and Administration, Vol. 39(4): 381-408 Diaz, J. and Handa, S. (2004). An Assessment of Propensity Score Matching as a Non Experimental Impact Estimator: Evidence from a Mexican Poverty Program. Mimeo. University of Maryland. Dehajia, R. and Wahba, S. (2002). Propensity Score Matching for Non Experimental Causal Studies. Review of Economic and Statistics, Vol. 84: 151-61 Dehajia, R and Wahba, S. (1999). Causal Effects in Non-Experimental Studies: Reevaluating the Evaluation of Training Programs. Journal of the American Statistical Association, Vol. 94: 1053-62 Gertler, P.; Locay, L.; Sanderson, W. (1989) Are User Fees Regressive? The Welfare Implications of Charging for Medical Care in Peru. Health economics: Latin American perspectives: 86-106 Grossman, J. (1994). Evaluating Social Policies: Principles and U.S. Experience. World Bank Research Observer, Vol. 9 (2): 159-80 Hayrapetyan, S. and Khanjian, A.(2004). Health care in Armenia: Challenges and Prospects. Mimeo, Armenian International Policy Research Group. HERA (2004). Review of health financing and provider payment systems in Armenia. Final Report. HERA, Germany. Holland, P. (1986). Statistics and Causal Inference. Journal of the American Statistical Association, Vol 81: 945-60 Lavy, V.; Strauss, J.; Thomas, D.; and De Vreyer, P.(1996). Quality of Health Care, Survival, and Health outcomes in Ghana. Journal of Health Economics, Vol. 15(3):333-57 Murrugarra, E., Chaudhury, N.; Cnobloch, R.; and Hammer, J. (2004) Poverty Targeted Social Assistance and Health: Impact of Eligibility Expansion on Health care Utilization in Armenia. Mimeo. World Bank. 14 Mwanbu, G.; Ainsworth, M.; and Nyamete, A. (1993). Quality of Medical Care and Choice of Medical Treatment in Kenya: An Empirical Analysis. Journal of Human Resources, Vol. 28(4): 838-62 Newman, J.; Pradhan, M.; Rawlings, L.; Ridder, H.; Coa, R; and Evia, J. (2002). An Impact Evaluation of Health, Education, and Water Supply Investments by the Bolivian Social Investment Fund. The World Bank Economic Review, Vol. 16 (2): 241-74 Newman, J.; Rawling, L.; and Gertler, P. (1994). Using Randomizes Control Designs in Evaluating Social Sector Programs in Developing Countries. World Bank Research Observer, Vol. 9(2): 181-2001 Posarac (2006). Armenian Experience with Proxi Means Testing. Mimeo. World Bank. Rosembaum, P. and Rubin, D. (1983). The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrica, Vol. 70:604-20 World Bank (2005). Mozambique PSIA. Mimeo. World Bank, Washington D.C. 15 Table 1: Descriptive Statistics and Sample Size. Eligible and Eligible and Not Registered Total registered with registered with benefits no benefits Demographics % Male 40.33 42.35 46.26 45.59 [0.01] [0.02] [0.40] [0.34] Average Age 37.07 44.43 34.21 34.05 [0.56] [1.08] [0.17] [0.15] % Employed 21.65 36.15 38.64 36.34 [0.01] [0.02] [0.39] [0.33] Region % Yerevan 18.81 15.30 38.42 31.76 [0.01] [0.02] [0.40] [0.32] % Urban 40.54 28.20 29.23 30.66 [0.01] [0.02] [0.33] [0.29] % Rural 40.65 56.50 32.35 37.58 [0.01] [0.02] [0.39] [0.34] Morbidity % Sick 32.50 39.65 18.11 20.80 [0.01] [0.02] [0.31] [0.71] % Severe Ill if Sick 42.21 32.92 39.04 37.32 [0.02] [0.03] [0.94] [1.48] % Treated if sick 54.00 51.48 56.96 54.91 [0.02] [0.03] [0.95] [1.55] Type of treatment if Sick % Visit doctor 26.99 23.07 34.62 31.38 [0.02] [0.03] [0.91] [0.69] % Hospitalized 2.91 3.93 4.97 4.10 [0.01] [0.01] [0.41] [0.29] % Private Physician 2.63 2.45 5.03 4.27 [0.01] [0.01] [0.41] [0.29] % Treated at home 38.77 40.75 37.86 37.65 [0.02] [0.03] [0.93] [0.72] % Healer 0.95 0.27 0.95 0.89 [0.00] [0.00] [0.18] [0.13] Total Observations 2422 714 18777 26614 Source: Authors using Armenia ILCS data. Standard errors in brackets. 16 Table 2: Direct estimates of the bias in the dependent variable due to observable characteristics [Probit results]. Dep. Var.: Receiving benefits if Matching model for outcome on Matching model for outcome on eligible and registered in BBP utilization if sick burden if treated (If sick) All Poor All Poor Socio Economic Condition Individual is Poor 0.13** - 0.151** - 0.05 - (0.059) - Health Status Morbidity is last 4 weeks 0.04** (0.026)** - - 0.06 0.087 - - Visit doctor if Sick - - 0.060 - - (0.064) Demographics Male dummy -0.09 (0.028)** -0.093 -0.024 0.05 0.074 (0.060) (0.073) Age 0.01** 0.011** 0.000 0.003 0.00 0.007 (0.005) (0.006) Square of age (0.00)* 0.000 0.000 0.000 0.00 0.000 (0.000) (0.000) Individual is Employed (0.40)** -0.456 -0.359** -0.330** 0.06 0.096 (0.067) (0.084) Constant 0.83 0.880 0.391** 0.401 0.08 0.109 (0.107) (0.133) Number of Observations 3123 1590 2022 1346 Prob. P> chi 2 0.00 0.00 0.000 0.000 Region of Common Support [0.5 - .86] [0.59 ­ 0.86] [0.45 - 0.71] [0.50- 0.70] Source: Authors using Armenia ILCS data. Standard errors in parentheses. 17 Table 3: Propensity score results using kernel matching [eligible and registered in BBP] All Poor All Poor NM M NM M NM M NM M Cost of treatment (in $LC) Visit Doctor (in percentage) Outcome for Treated 790.37 741.79 177.46 182.08 0.29 0.29 0.25 0.25 Outcome for Control 1068.80 1352.44 141.41 245.66 0.24 0.21 0.14 0.20 Difference -278.43 -610.65 36.05 -63.58 0.05 0.08 0.11 0.05 (267.91) (68.66) (0.02) (0.03) % Difference -26.05 -45.15 25.49 -25.88 20.38 35.93 77.28 25.37 Total observations Number of Treated 789 - 355 - 789 - 355 - Number of Controls 253 - 99 - 253 - 99 - Observation on support Number of Treated 780 - 346 - 779 - 335 - Number of Controls 253 - 99 - 253 - 99 - Doctor visit as % of nonfood Health as % of nonfood Outcome for Treated 2.53 2.37 1.52 1.51 17.38 17.13 12.27 12.59 Outcome for Control 2.89 3.36 1.62 0.71 17.12 15.16 12.16 10.49 Difference -0.37 -0.99 -0.10 0.80 0.27 1.97 0.11 2.10 (0.62) (0.57) (1.26) (1.95) % Difference -12.70 -29.56 -6.20 111.94 1.55 13.03 0.94 20.02 Total observations Number of Treated 491 - 196 - 491 - 196 - Number of Controls 170 - 57 - 170 - 57 - Observation on support Number of Treated 475 - 182 - 475 - 182 - Number of Controls 170 - 57 - 170 - 57 - Treatment at home (medicines) as % of nonfood Outcome for Treated 11.62 11.70 8.08 8.38 - - - - Outcome for Control 11.89 9.08 8.56 4.27 Difference -0.27 2.63 -0.49 4.11 - - - - (0.97) (1.32) % Difference -2.30 28.94 -5.72 96.16 - - - - Total observations Number of Treated 491 - 196 - - - - - Number of Controls 170 - 57 - Observation on support Number of Treated 475 - 182 - - - - - Number of Controls 170 - 57 - Other Matching Methods: Cost of treatment (in $LC) Radius matching Local Linear Matching Outcome for Treated 630.42 630.42 194.89 195.34 630.42 630.42 194.89 195.34 Outcome for Control 1141.43 1022.44 421.03 360.77 1141.43 1113.51 421.03 351.57 Difference -511.01 -392.02 -226.14 -165.42 -511.01 -483.09 -226.14 -156.22 (139.98) (65.49) (141.39) (37.17) % Difference -44.77 -38.34 -53.71 -45.85 -44.77 -43.38 -53.71 -44.44 Total observations Number of Treated 782.00 - 485.00 - 782.00 - 485.00 - Number of Controls 1240.00 - 861.00 - 1240.00 - 861.00 - Observation on support Number of Treated 782.00 - 485.00 - 782.00 - 485.00 - Number of Controls 1240.00 - 859.00 - 1240.00 - 859.00 - Source: Authors using Armenia ILCS data. Standard errors in parentheses. NM= not matched, M=matched. 18 Figure 1: Statistics on Health care Utilization in Armenia Source: WHO. 19 Figure 2: Distribution of matches within support of the propensity score [Eligible and registered in BBP] .5 .6 .7 .8 .9 Propensity Score Untreated Treated 20 Appendix Eligibility to Family Poverty Benefits in Armenia From Posarac (2006) In January 1999, Armenia introduced monthly family poverty benefit in cash, targeted at the very poor households using a proxy means-testing (PMT) mechanism. The proxy means-testing formula consists of the following variables: - social category of each of the family members, or individual "social risk" (Pk) and related average "social risk" for the family (Pm); - number of the family members not capable of working (Pc); - place of residence (Pr); - housing situation (Ph); - car ownership (Pa); - private business (Pb); - the document issued by respective territorial center for social services verifying the social and economic situation of the applicant family (Pf) and its eligibility for the benefit ("eligible" and "not eligible"); - family income (Pi). The score of the family need (P) is calculated using the following formula: P = Pm* Pc * Pr * Ph * Pa * Pb * Pf * Pi (a) Socio-economic category of each of the family members (individual social risk)--Pk and related average "social risk" for the family (Pm). This is the most important element in calculating the score of the family need. Each family member is screened for a certain social category, e.g. for whether she/he belongs to a certain category of "social risk". Each category brings a certain number of points. The number reflects the assumed level of need of each category. The list of categories and corresponding number of points is presented in the following table. Social category Points 1 Biological orphan (no parents) 50 2 First category disabled 48 3 Child invalid (up to 16) 45 4 Biological orphan (one parent deceased) 43 5 Second category disabled 39 6 Pensioner (75+) 39 7 Single pensioner 36 8 Child below 2 year of age 35 9 Pensioner 34 10 Child 2-18 33 11 Pregnant women (20+ weeks) 30 12 Third category disabled (below pension age) 28 13 Unemployed 27 14 Single mother child 26 15 Child of divorced parents 26 16 Public university student 22 17 No social category 20 21 In cases where one person belongs to several social categories, a weighted average is calculated. The weights are as follows: for the category with the highest number of points 1.0; for the second highest 0.3 and for the third and all the rest 0.1. For instance, a person can be a 17-year old child (category 10), with divorced parents (category 15), a student (category 16) and a third category disabled (category 12). Her/his individual social category (or "social risk") score is calculated in the following way: Pk(ind) = P10 + 0.3*P12 + 0.1*(P15+P16) = 33+0.3*28+0.1(26+22) = 46.2 The average "social risk" score for a family is calculated as arithmetic mean of the family members scores. (b) Number of family members not capable of working (Pc). The value of this factor is calculated in the following way: Pc = 1.0 + 0.02*m, where m is the number of the family members incapable of working, namely children up to 16, women over 63, men over 65 and first and second category disabled. (c) Place of residence (Pr). For most of the settlements in Armenia, the value of this factor is one. However, there is a list of 173 settlements (in the earthquake zone and border territories) for which the coefficient ranges between 1.03 and 1.05. (d) Housing situation (Ph). Housing situation is classified into 6 categories with the following coefficients: "domik" (temporary shelter such as a carriage, a barrack, etc., in particular in the earthquake zone) ­ 1.2; homeless ­ 1.07; unsafe dwelling ­ 1.05; collective center for internally displaced persons ­ 1.03; other ­ 1.02; permanent dwelling ­ 1. (e) Filter variables (0 or 1). The following factors are used as filters: a car ownership (Pa), private business (Pb), and a document issued by respective territorial center for social services verifying the social and economic situation of the applicant family and its eligibility for the benefit (Pf). Their value can be either 1 or 0. Obviously, 0 for any of the three (the family has a car and uses it, the family or its members are running private business and the social services center has assessed the family as ineligible for the benefit) eliminates the family from the list of beneficiaries. (f) Family income. The family income coefficient is calculated using the following formula: Pi = 1.2 ­ 0.04*(Sj/m*M) (j=1...n) Where n is the number of the household members, sj is the income of the j-th household member, m is the number of the present household members, and M is the minimum wage (regulated by the Government). The income includes wages and salaries, income from self-employment, pensions, stipends and unemployment compensation. Income from farming is estimated based on cadastral income, while income from cattle breeding is estimated separately using a methodology regulated by the Government. The qualifying score: Currently, the score that qualifies the household for the benefit is 34.01. This score is determined once per year in the following way: all applicant households are ranked by their scores. The cut off point is decided upon based on available resources. 22