EVALUATING THE IMPACT OF INFRASTRUCTURE REHABILITATION PROJECTS ON HOUSEHOLD WELFARE IN RURAL GEORGIA Michael Lokshin1and Ruslan Yemtsov ABSTRACT This paper evaluates the effect of various community level infrastructure rehabilitation projects undertaken in rural Georgia on household wellbeing. The analysis is based on combining household and community level survey data. Our empirical approach utilizes the panel structure of the data to control for time-invariant unobservables at the community level by applying propensity-score-matched double difference comparison. Our results indicate that improvements in school and road infrastructure produce nontrivial welfare gains for the poor at the village and country levels. The impact of water rehabilitation projects is ambiguous. School rehabilitation projects produce the largest gains for the poor. The methodological lesson from this analysis is that ad hoc community surveys matched with ongoing nationally representative surveys can provide a feasible and low cost impact evaluation tool. World Bank Policy Research Working Paper 3155, October 2003 The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org. 1 Address for correspondence: Michael M. Lokshin, Development Economics Research Group, World Bank, 1818 H Street NW, Washington, DC 20433, email: mlokshin@worldbank.org; Ruslan Yemtsov, Eastern Europe and Central Asia Poverty Reduction and Economic Management, World Bank, 1818 H Street NW, Washington, DC 20433, email: ryemtsov@worldbank.org. This research was conducted as a part of analytical work for Georgia: Poverty Update. Support of PRMPO trust fund is gratefully acknowledged. We thank Louise Cord, Martin Ravallion, and Dominique van de Walle for useful and constructive comments. We are also grateful for support and contributions of Alexander Kolev, Nodar Kapanadze, and Zurab Sajaia. Special thanks to the Georgia Social Investment Fund (GSIF) team, and in particular to the GSIF project team leader Alexandra Posarac for providing support and making a wealth of information available to the authors. 1. Introduction Economic and political turmoil have led to a dramatic fall in living standards in Georgia, once one of the richest republics of the former USSR. Despite robust economic growth (averaging 5 percent per year in the last seven years), poverty remains stubbornly high, fueled by rising income inequality and widening disparities between socioeconomic groups, regions, and urban and rural areas. At any point in time over the last four years, about 20 percent of the population has lived in absolute poverty, and around 5 percent has survived in extreme poverty consuming at less than a dollar a day (World Bank 2002). The stress of transition has also resulted in deteriorating non-income indicators of poverty, particularly those related to child welfare ­ school enrollment rates have fallen, maternal mortality have risen and infant mortality rates have remained high (World Bank 2003b). Why has economic growth failed to produce any sizeable reduction in poverty? Several recent studies (World Bank 2002, Government of Georgia 2003, World Bank 2003a, Cord et al. 2003) found that in Georgia the lack of a well-functioning public infrastructure is a major impediment to a broad-based growth. The poor state of infrastructure affects poverty both indirectly and directly. A deficient public infrastructure limits growth and the opportunities open to the poor to benefit from that growth.2 Independent Georgia inherited a developed infrastructure network that provided a broad access to key public services for the majority of the population. Public infrastructure spans across a wide range of facilities: water and sanitation systems, transportation, schools, kindergartens and health clinics, and other types of public services. The state of these infrastructure facilities has been deteriorating rapidly since the beginning of the transition. Health and education facilities are badly in need of repair and lack of drinking water and good sanitation has increased the risk of the spread of disease. Rural areas in Georgia have been hit particularly hard, suffering from an increasing marginalization and impoverishment, worsening access to roads, information, 2See, for example, Boarnet, 1997; De Frutos, Garcia Diez and Perez Amaral, 1998; Sanchez-Robles, 1998; Castaneda Sabido, Cotler and Gutierrez, 2000. 2 energy, healthcare facilities, and markets (World Bank 2003e). Dilapidation of health and education establishments was more pronounced in rural than in urban areas of Georgia. Georgia has little, if any, resources available for rehabilitating its severely degraded infrastructure. Even with substantial improvements in revenue collection, Georgia will continue to face a severe shortage of public investment funds. As in the past, donor funds will be required to undertake basic maintenance of roads, repair of water and sanitation systems, and urgent rehabilitation of school facilities. But the mounting rehabilitation bill is so huge due to deferred and neglected maintenance that it is unrealistic to expect that even the most critical needs will be met with available public, private or donor resources. The Government will face difficult tradeoffs of weighting investment priorities against critical current spending needs, such as poverty alleviation programs and law and order services. Little is known about the impact of infrastructure rehabilitation activities in Georgia. Although every donor-financed operation includes an evaluation module, these evaluations often focus on the project-specific outputs only, overlooking the impact of the project on welfare of beneficiaries. In the rare cases, when beneficiaries have been the center of attention, the lack of data, insufficient information on welfare indicators, and ad hoc choice of the control groups have limited the usefulness of impact evaluations. This paper aims at providing the Government of Georgia and the international donor community with information that would help evaluate public investment tradeoffs and thereby channel very limited and increasingly scarce resources to their best uses. We investigate the welfare impact of various types of rural infrastructure rehabilitation projects in Georgia and evaluate targeting (or placement) of projects. This paper also attempts to provide evidence on whether the poorest benefit from such activities, and among interventions that benefited the poor, which ones have the largest impact. In addition, we assess how well the micro-credit programs funded by the World Bank- supported Georgian Social Investment fund (GSIF) perform compared with other donor- and State-supported interventions. The interest in evaluating the effectiveness of community-based projects has been expanding in response to the increasing popularity of such projects for channeling development assistance. Several recent papers focus on measuring the effects of 3 improved infrastructure on various dimensions of wellbeing. Improvements in water and sanitation facilities are analyzed by Jalan and Ravallion (2003), Lee at el., (1997), Brockerhoff and Derose (1996). The impacts of school infrastructure rehabilitation projects are evaluated in Glewwe (1999), Hanushek (1995), and Kremer (1995). Jacoby (2002) and van de Walle and Cratty (2002) analyze the effect of improvements in access to roads. The World Bank's multi-country analysis, Social Funds 2000, has resulted in several studies that look at the effectiveness of Social Investment Fund (SIF) projects in Honduras (Walker et al. 1999), Zambia (Chase and Sherburne-Benz 2001), Bolivia (Newman et al., 2002), Armenia (Chase 2002), Nicaragua (Paxson and Shady 2002), and Peru (Pradhan and Rawlings 2002). In comparison with other SIF project evaluation studies, our analysis has several distinct features. First it is based on combining longitudinal community- and household- level surveys. The availability of longitudinal observations in many cases allows us to deal with unobserved heterogeneity in project placement ­ a problem that is often difficult to address using cross-sectional data. Second, our analysis relies on a comprehensive coverage of all infrastructure rehabilitation activities in every community regardless of the way they were financed. Many SIF evaluation studies concentrate on a particular type of projects, omitting from the analysis possible synergies arising from the interaction of SIF and other projects. We also look at both direct and indirect effects in which projects affect the wellbeing of the population in a community. Finally, our research focuses on the impact of rehabilitating of existing infrastructure facilities rather than on the construction of the new facilities ­ an issue usually taken up by the SIF studies. This focus has important implications for the choice of empirical methodology we employ in the paper. Based on the data from the Rural Community Infrastructure Survey (RCIS) and the Survey of Georgian Households (SGHH) we conduct a series of tests to assess the effectiveness of various types of projects on a wide range of welfare outcomes. Our results indicate that improvements in school infrastructure produce nontrivial gains in school enrolment rates, raise school attendance, and reduce health risks of school-age children. These effects are strongly biased toward the poor. Road and bridges rehabilitation projects generate clear economic benefits at the community level. The most 4 definite effect of water-system rehabilitation is the reduction of the incidence of water- borne diseases. Non-poor households seem to benefit more from water and road rehabilitation projects. The paper is organized as follows. The next section describes the different types of community investment projects carried out in rural Georgia. Section 3 introduces the dataset used in this analysis. Section 4 discusses the impact evaluation methodology. Section 5 describes the impact assessment results. Section 6 concludes. 2. Community-level investment projects in Georgia Rehabilitation activities in many countries are mostly implemented as part of a public investment program. In Georgia, however, this is an exception rather than a rule, because of a dire state of local budgets. Most community-level (micro) projects are supported by local and international donors. The database compiled by the UN Office for the Coordination of Humanitarian Affairs (UNOCHA 2003) lists 218 organizations that have been active in carrying out such projects. Based on RCIS estimates, the total number of micro projects affecting the rural population in 1996-2002 could have been as high as 8000. The GSIF supports one of the largest public investment programs in Georgia. Funded by the World Bank's International Development Association (IDA) it provides micro credit to communities for rehabilitating critical infrastructure (GSIF 2001). Before stage one of the GSIF program came to a close in 2002, about 400 micro-projects had been completed. To achieve an objective of poverty reduction, GSIF targets its activities to the poor. This targeting is carried out on a geographical basis. GSIF allocates funds among districts using a scoring formula. This placement mechanism takes into account the population size, an extent of poverty, geographical characteristics, and state of infrastructure in a district. Another major supporter of local initiative projects in Georgia is the UN World Food Programme (WFP). It has lunched a particularly large number of micro projects, including the rehabilitation of water systems, schools, roads and bridges. The purpose of WFP was to provide emergency food aid through public works. Between 1997 and 2000 5 WFP delivered food assistance to over 250,000 beneficiaries through about 300 micro projects (WFP 2002). Many other international organizations (including among others TACIS, World Red Cross, International Save the Children Alliance, OXFAM, and USAID) and several private donors were also involved in their own micro projects in Georgia. Given the multiplicity of agencies involved in the community level rehabilitation projects, a comprehensive evaluation of the impact of rehabilitation project is necessary. Such an assessment would include analysis of complex interactions among various rehabilitation activities and their multidimensional repercussions for the living standards of Georgians. 3. Data The analysis in this paper is based on two sources of data. The household-level information comes from the ongoing multi-topic survey (SGHH) ­ the main official household survey in Georgia. The survey began in July 1996 and is conducted quarterly by the State Department of Statistics of Georgia with a support of the World Bank, the UK Department for International Development (DFID), and EC Food Security Programme. The SGHH is representative at the national, urban, rural, and regional levels3. The survey is based on a two-stage stratified rotating sample of 2,800 households. At the first stage, 282 primary sampling units (PSUs) are randomly selected from the stratified list of 12,000 census units with probability proportional to size4. In the rural areas PSUs roughly correspond to villages. In total, the survey covers 174 rural population sites. On the second stage, 7 to 20 households are randomly selected from each PSU. These households stay in the survey for four consecutive quarters and then are replaced by different households from the same sampling unit. This process continues until the list of households in the sampling unit is exhausted. At that point another PSU is chosen from 3Three sparsely populated administrative regions are combined in stratification with larger regions. Racha- Leckhumi and Lower Swaneti are included in Imereti, Mtskheta-Mtianeti is sampled with Inner Qartly region. Abkhazia and South Ossetia were not part of the sampling frame. 4The size of a PSU could range from 70 to 300 households. 6 the same stratum. Each PSU tends to remain in the survey for years, making it possible to monitor changes in living standards at the village level over a long period of time. The SGHH survey instrument combines features of a regular household budget survey with the features of an integrated living standard measurement survey. It collects information on the demographic characteristics of the household members, their labor market activities, and their access to social services. A special section of the questionnaire gathers information on income and consumption expenditures as well as on ownership of assets. Modules collecting information on health and education outcomes were introduced in the first quarter of 2000. This defines the timeframe for our study. Community-level information for our study comes from the Rural Community Infrastructure Survey (RCIS) conducted in May-June 2002 by Georgian Opinion Research Business International (GORBI) with the support of the State Department of Statistic Georgia (SDSG) and the World Bank (PRPMO Trust Fund). RCIS supplements SGHH data with community-level information for rural areas. To that end, RCIS covers all 174 rural population sites from the SGHH. In addition, 75 villages not covered by the SGHH were selected from the list of 360 villages where GSIF projects have been implemented by 20025. These 75 villages were chosen to participate in the RCIS based on a very limited set of characteristics available in GSIF database: regional dummies, altitude of the village above sea level, the distance from the district center, and the size of the village. We use these characteristics to match villages with GSIF projects to villages from the SGHH sample without a GSIF projects. By doing this we hope to improve the precision of our estimates of project impact. One of the main purposes of the RCIS is to collect retrospective information for evaluating infrastructure rehabilitation projects in Georgia. The RCIS questionnaire includes sections on geographic and climatic conditions, the state of transport infrastructure, water supply systems, schools, kindergartens and healthcare facilities, and the sources of livelihood for the local population based on detailed modules on agricultural and non-farm activities. A special section of the questionnaire contains 5Detailed information on RCIS sample frame, methodology and questionnaire design could be found in the Final Community Survey Report published by GORBI and the State Department of Statistics (2003). 7 detailed questions on all infrastructure rehabilitation projects carried out in the survey sites since 1996. The RCIS collected this information, which include date of completion of each project, during multiple interviews with key informants in rural sites. The respondents were representatives of local authorities, informal leaders, NGOs, and social assistance workers. Responses for every section of the questionnaire were solicited from the most informed respondents, i.e., those in a position to know the current (2002) situation as well as conditions in the past (1998). School directors, for example, provided information on school facilities. While this design of the RCIS compensates for the absence of proper baseline data in some indicators we use in the analysis, it could introduce possible bias due to the use of a long recall period for the village-level outcome measures. An important feature of the RCIS data is that it covers all villages in the SGHH. Thus, we have a complete overlap between these two sources of data. This allows us to use both community and household level information for our evaluation. Sample for evaluation Based on information from the project module of the RCIS we classify villages in the sample by the type and number of projects. Our sample includes 249 rural population sites or villages. A typical village in the sample benefited from multiple infrastructure projects ­ 57 percent of survey sites reported having two or more projects carried out between 1998 and 2002 (with a maximum of 15 projects in a village). Forty-nice villages (20 percent of the sample) had no projects. RCIS collected information on 549 projects funded by various local and international agencies. These rehabilitation projects can be grouped into six major categories: schools, which comprise 28 percent of the total number of projects; road infrastructure (27 percent); water supply systems (11 percent); medical facilities (6 percent); kindergartens (3 percent); and others. We evaluate the three largest groups of interventions ­ schools, road infrastructure, and water system rehabilitation projects. To fit the recall period of RCIS, our analysis is based on a sub-sample of the projects started on or after 1998 (our 8 baseline) and completed by January 2001. This gives us a total of 144 projects in 106 villages to evaluate the community level impact of the projects. 4. Methodology A true measure of the impact of an intervention is the difference between the observed outcome for a group of beneficiaries and the (counterfactual) outcome for the same group without the benefit of intervention. Because counterfactuals are never observed, the challenge of the evaluation work is to find the plausible proxies for such unobserved outcomes. We address this challenge by comparing outcomes for beneficiaries with the outcomes for an appropriate comparison group. Both groups should have characteristics common with each other. These characteristics would influence both the outcomes of an intervention and group selection into the program. Funds are allocated to micro-projects in Georgia based on the preferences of a community and/or on the requirements of a project-implementing agency. The placement rationale varies among agencies, but in most cases it takes into account the extent of poverty or its correlates (such is the case with WFP, e.g.), the state of infrastructure in a village (GSIF, e.g.), or regional characteristics. Beyond these concerns, many projects rely on demand-driven targeting mechanisms. Whether the funds are allocated to a particular village can depend on its ability to actively seek support from the implementing agencies. In order to apply for GSIF financing, for example, a village community has to mobilize a counterpart cash contribution that amounts to about 10 percent of total project costs6. Therefore, villages are chosen by project managers based on characteristics, both observable and unobservable that could be correlated with the expected outcomes of a project. Because of such nonrandom placement, a simple comparison of outcomes between villages that benefit from infrastructure rehabilitation projects and villages without projects would not measure correctly the impact of an intervention. 6An average community contribution towards a GSIF project amounts to about $5000, which is a very substantial sum of money in cash-striped rural Georgia. 9 If selection of a village into a project is based purely on observable characteristics, we can use a propensity-score matching (PSM) method to remove the selection bias due to differences between villages with and without projects (Rubin 1973). Using the PSM method, villages with projects (the "treated" group) are matched with the villages without projects (the "control" group) on the basis of the propensity score. This score measures the probability that a project is implemented in a village as a function of that village's observed pre-intervention characteristics. Rosenbaum and Rubin (1983) show that an exact matching of propensity scores implies that villages from control and treated groups have the same distribution of covariates. When the relevant differences between any two villages are captured in the observable covariates (i.e., outcomes are independent of assignment to treatment, conditional on pre-treatment covariates), PSM yields an unbiased estimate of the intervention's impact. However, some unobserved characteristics of the village that correlate with project outcomes might also correlate with project placement. This correlation can introduce bias in the estimation of project impact. For example, an active parent group might lobby the village authorities to pursue a school rehabilitation project, and, at the same time, parent participation in the education process could positively affect school outcomes of their children. In this case, the effectiveness of the school project will be overestimated if the evaluation procedure does not take into account the differences in parental activity between treated and control villages. Under the assumption that pre-intervention differences between the control and treated villages are the result omitted variables that do not change over time in their impact on outcomes, we can use the difference in difference (DD) method to correct the possible bias. In DD the pre-project difference in outcomes may be subtracted from the post-project differences for the same villages. The underlying assumption of DD is that the time trend in the control group is an adequate proxy for the time trend that would have occurred in the treated group in the absence of an intervention. In this study we use the matched DD method, which combines the PSM and DD. Recent studies by Heckman, Ichimura and Todd (1997), and Heckman et al. (1998) have argued that combining PSM and DD can substantially reduce the bias found in other non- experimental evaluations. First, we match villages from the control and treatment groups 10 using PSM. This matching removes the selection bias due to the observed differences between the treated and control villages7. Then we apply DD to correct for possible bias due to the differences in time-invariant unobserved characteristics between the two groups. To evaluate the impact of the project we compare the changes in the outcome indicators between matched villages from the treatment and control groups. Following Chen and Ravallion (2003), we can define outcome measure Iit for a project in i'th treated village (Di=1) at date t as: (Iit | Di =1) = Iit + Git +it (i =1,...,N;t = 0,1), * I I (1) where Iit is the counterfactual outcome for a treatment village if the program had not * been implemented, Git is the gain in an outcome attributable to a project, and it is a I I mean-zero error term uncorrelated with the project placement. While the counterfactual outcome is unobservable, its estimates I^it could be obtained from a comparison group. * However, mismatching arising from differences in unobserved characteristics between treated and control villages might bias this estimate. If the selection bias is time-invariant and separable, it could be removed from the estimate by taking differences over time. The mean difference-in-difference for the outcome is estimated by taking the expectation of (1) over all N: E[(Ii - I^i ) - (Ii - I^i ) | Di = 1] = E[Gi -Gi | Di = 1] * * I I (2) 1 1 0 0 1 0 If the outcomes at period 0 are not correlated with the project assignment, equation (2) estimates the mean changes in outcomes for the treated villages. There is anther form of bias that PSM and DD cannot remove. It can arise from time-variant unobservable characteristics correlated with the project placement and the outcomes of the intervention8. In particular, project placement could be based on unobserved community characteristics that are correlated with changes in the expected project outcomes. However, in the context of micro-projects in rural Georgia this bias may not arise. The project-placement procedures used by the implementing agencies are 7The observations from control and treatment groups are matched with replacement. Thus, more than one village from the treatment group could be matched with a village in the control group (Dehejia and Wahba 1999). 8This problem thought to be severe for infrastructure programs in poor areas if the deficient state of infrastructure in the initial period not only attracts the program, but also reduces future growth (Jalan and Ravallion 1998). 11 based on formal criteria that capture exclusively the current state of affairs. Thus, it can work in our favor reducing (but not completely eliminating) possible bias in our evaluation. Impact indicators Any infrastructure project has multiple welfare repercussions for the households affected by a project. To assess the impact of a project one should track changes across different welfare dimensions. In practice, several indicators need to be constructed for each type of intervention and the choice of these indicators is determined by the practicalities of the evaluation and data collection. Impact or outcome indicators have to be measurable with the data at hand, and be linked directly to, and respond relatively quickly to the intervention in question. How effective an intervention is depends on the availability of inputs that compliment project goals and/or on the behavioral responses of beneficiaries. In most cases the interaction of public and private inputs makes the task of estimating ultimate outcomes complex. If public inputs can displace private resources, the potential benefits of a project may not be realized in practice. For example, Jalan and Ravallion (2003) show that the impact of an improved access to piped water on prevalence of diarrhea among children depends on the willingness of parents to devote household resources to disease prevention. This is why evaluation studies often include indicators that reflect private inputs. The outcome indicators might be complemented by output indicators that measure the progress towards the implementation of the project. The difference between output and outcome indicators is that outcome indicators are directly linked to the project objectives while the output indicators are related to the means of achieving these objectives. For example, the output of a water supply rehabilitation project could be an increase in the number of households connected to piped water. An outcome of a water project might be a reduction in the incidence of water-borne diseases. In many situations, however, output and outcomes can coincide or be measured by similar indicators. 12 We analyze three types of infrastructure rehabilitation projects in this study: schools, roads and bridges, and water systems. In the short run, these interventions are expected to have a number of positive and objective effects on household standard of living. We use the measures of these effects as impact indicators for our analysis. Two sets of impact indicators are identified for each project. The first set is drawn from RCIS and the second set from SGHH. Community-level indicators based on RCIS measure changes between 1998 and 2002. Since our treatment group includes villages with projects completed before the beginning of 2001, we are allowing for more than a year to pass before assessing project benefits. Because SGHH did not cover all RCIS villages, the analysis of SGHH impact indicators is conducted on a smaller sample compared with the analysis of the RCIS indicators. We use household averages based on SGHH to compare outcomes in 2001 and 2000. This timeframe is dictated by data availability. However, the way our treatment group is defined may cause some problems with interpretation. While the majority of projects in our treatment group were completed in 2000, some villages could have projects implemented as early as 1998. For such villages the comparison of household- level outcomes between 2000 and 2001 would trace the long-term growth effect rather than the immediate impact of an intervention, as in case of projects completed in 2000. Excluding these villages from our sample would substantially reduce the number of useable observations and therefore decrease precision. We choose to retain the villages but exercise care in interpreting results. Our main indicators for all project types are shown in Table 1. The design of an indicator set in this paper aims to measure (a) immediate output of a project (such as new channels of drinking water supply resulting from a water projects), (b) project-specific outcomes (such as changes in the school enrollment for school projects), (c) changes in private inputs related to a project (such as transportation expenditures for road projects), (d) the indirect economic effects (such as a change in the SMEs resulting from road projects), and (e) the impact of projects on other dimensions of well-being (such as changes in the incidence of respiratory diseases among school age children because of school repairs). Figures reported in Table 1 are simple averages across all villages in the sample calculated at the beginning and at the end of the timeframe chosen for the analysis 13 of projects. The last column in Table 1 reports changes (differences) in the main outcome indicators.9 To deal with the problem of several influential outliers, changes in some continuous variables were recoded into simple categorical variables reporting the balance between positive and negative changes. For example, Table 1 shows that an average village in RCIS had 293 pupils in its secondary schools in 1998. This average has fallen to 283 by 2002. But the balance between positive and negative changes is greater than zero because the number of villages in which number of pupils has increased exceeds the number of villages in which it has decreased by 24.4 percent of the sample. Some indicators reported in Table 1 reflect alarming trends in school enrollment and attendance, quality of road infrastructure and access to piped water in rural areas. Only 68.3 percent of villages had all school age children in schools in 1998; this rate deteriorated further to 59.0 percent by 2002. Household level data suggest that close to 8 percent of children in an average village were missing over 30 days of classes in 2000. This indicator improved for all villages in the sample by the end of 2001. In 1998 as many as 91.1 percent of villages reported that the quality of their main roads was inadequate. This indicator improved quite significantly by 2002, but 71.1 percent of villages still complained about the road quality. In as many as 66.8 percent of villages it took more than 4 hours for an ambulance to arrive in 2000. This indicator also improved somewhat by 2001, but remained at a very high 52.8 percent of the sample. Despite improvement in the road infrastructure, employment opportunities for village population stagnated. On average only 16.1 percent of village adults performed any off- farm work in 2000 and this share has declined in 2001. The access to piped water is far below 100 percent: in 1998 56.1 percent of rural households were connected to piped water sources with practically no change by 2002. Even if households are connected to water pipes, water is available only 8 hours per day, on average. Among indictors tracing the impact of water supply quality on well-being high and increasing incidence of water-borne diseases among children is of a particular 9Note the difference in definition of "Before" and "After" time points in RCIS- and SGHH-based indicators. Some villages have missing values for an indicator after or before project completion; thus the indicator of change may differ from a simple ratio between pre- and post-project indices. 14 concern. By 2001 as many as 2.2 percent of children below 7 reported illnesses related to poor water quality in a single month preceding the survey. 5. Results In this section we discuss the results of impact evaluation analysis for school rehabilitation projects, improvements in road infrastructure, and water system rehabilitation projects. We first present a description of the project placement mechanism. We model the probability that a village is selected into a project as a function of a large set of variables from RCIS that include village-level aggregates on geographic, demographic, and socio- economic conditions (listed in the last column of Table 2). Our specification controls for the project-specific pre-intervention outcome indicators. Since we are not making any a priori assumptions about which indicators have an impact on village selection into a project, we use similar variables for each project type. We also try to control for the presence of other projects in the same village. The probit estimates are shown in Table 2. Adjusted pseudo-R2 of these probits ranges from 0.156 for the school projects to 0.393 for the water projects. These are acceptable levels of explanatory power; an R2 that was too high could indicate the existence of fundamental differences between the characteristics of project and non- project villages, which would make the formation of a proper control group very problematic. Only a few coefficients in the table are significant, the indicator of natural calamities, for example. However, one should not take this as a sign of the problem because our empirical specifications include many correlated variables and the purpose of the estimation is to calculate the propensity score and not the structural modeling of an underlying selection mechanism. Using these results kernel density estimates of the propensity score for the villages from the treatment and control groups are constructed. The shape of the probability density function (Figure 1) differs among the project types, but for all three types of rehabilitation projects similar tendencies are observed. The shape of the PSM distribution suggests that before matching the characteristics of the villages with and without the project were quite different (left panels in Figure 1). However after the 15 matching, the kernel density estimates of the propensity score show that PSM eliminates most of the observed differences between the villages with and without projects (right panels in Figure 1). Therefore, for each village in the treatment group we find the pair among the villages in the control group that is similar to the project village in all respects. School rehabilitation projects Typically investments in micro-projects in Georgia involve rehabilitation and repair of existing facilities rather than construction of new facilities. School projects focus on improving school buildings: repairing roofs, windows, and floors, replacing inner pipes, installing sanitary and heating equipment, and repainting walls. These projects may yield several types of benefits to the community. School rehabilitation may affect positively school enrolment and attendance rates ­ the indicators most often used in the SIF impact evaluation studies (e.g., Newman et al. 2002; Chaise 2002; Paxson and Schady 2002; Pradhan and Rawlings 2002). Better heating and repaired windows in the classroom may reduce the incidence of respiratory diseases. This could be particularly important for Georgia, where some rural schools close for several weeks in winter because of frigid classrooms (Orivel 1998). Increases in household expenditures on schooling can be used as an indicator of the positive private response to investment in school rehabilitation. And the subjective assessments of schooling conditions provide a useful check for the results based on objective measures. School rehabilitation projects were completed in 61 villages (about a quarter of all villages) by 2001 in the RCIS sample. Thirty-seven villages out of this group were also covered by the SGHH. We formed the initial control group using villages without school projects those with incomplete school projects at the end of 2001. The DD estimation of the impact of school rehabilitation projects is shown in Table 3 for unmatched and PSM constructed control groups. Three outcome indicators are reported at the community level. The first indicator shows that the share of villages reporting that all children are presently enrolled in school and are attending classes declined between 1998 and 2002. In the matched comparison the average change in the outcome indicator is the same for the treatment and control groups and shows that school 16 enrolment ceased to be universal in 6.6 percent of the villages. The next indicator, the number of pupils in village schools, gives a different picture. In approximately a third of project villages the number of pupils has increased, compared to an increase in just over 20 percent of the non-project villages in the unmatched sample. The matched comparison shows an even larger, statistically significant difference. The number of secondary school graduates increased in 37 percent of the villages with the rehabilitation projects10. This outcome demonstrates a significant improvement over the changes in number of graduates in villages without projects (24 percent). The village- level subjective assessment indicator shows statistically significant change in the perception in beneficiary villages that access to education has improved. There is no such change in the unaffected villages. By combining information from RCIS and SGHH, we obtain a more detailed set of outcome indicators. Changes in all outcome indicators unambiguously point to positive long-term effects of the school rehabilitation interventions. In project villages, primary and secondary school enrolment rates increased by 6 percent between 2000 and 2001. Enrolment rates decreased in control group villages for the unmatched estimation and were close to zero for the PSM estimation. However, the difference in changes in this outcome indicator is only marginally significant. One indicator of school attendance, a change in the share of pupils missing more than 30 days in the school year, shows clearer benefits from school improvements. The share of pupils missing classes dropped in project villages by 5.7 percent while it increased by 2 percent in the matched control group. The health impact of school rehabilitation is substantial. In the treatment group villages the incidence of respiratory diseases among school age children declined by 12 percent compared with a decline of only 5.6 percent in villages without the project. Despite overall improvement in the objective schooling indicators, the rehabilitation of school infrastructure fails to produce significant changes in parents' assessment of schooling conditions. The indicator of private inputs in education tracks an increase in household education expenditure across the country. The differences in changes in expenditures between the treatment and control group villages are insignificant. 10We define the change in the number of graduates in the village as a ratio of the number of graduates in 2001 and 1998. 17 Overall, the estimation results fit our priors. Indicators that are closely related to the project outputs, e.g., school attendance, show more significant and stronger effects. The project impact spans across dimensions of wellbeing by improving the health status of school-age children. While school rehabilitation projects have produced clear benefits, the size of the effects and project coverage were often insufficient to counter the general tendency toward worsening school conditions in Georgia. In cases where certain indicators have improved, this change can be fully accounted for by the impact of the school rehabilitation projects. The improvements in school conditions between 2000 and 2001 reduced the number of school-age children missing more than 30 days of school by 5.7 percentage points for project sites. At the country level this translates into a 1.7 percentage points reduction in the number of children missing school. As a result, the observed share of pupils missing school in rural Georgia was reduced to 6.3 percent, compared to the 8.0 percent share that would have been observed in the absence of the project. On the other hand, the aggregated number of pupils enrolled in school changed only slightly due to the project effects. The number of pupils increased in 24.4 percent of villages between 1998 and 2002. Without the project the number would have increased in 21.4 percent of villages, suggesting that other non-project factors (e.g., demographic) were key drivers in the observed changes in the number of students in schools. Improvements of road infrastructure Road and bridge rehabilitation often means pavement of existing roads, restoration of road structures damaged or destroyed by flooding and earthquakes, and widening of road intersections and bridges. The rehabilitation of roads and bridges could reduce the time spent commuting and ease access to market places. This may lead to an increase in the value of productive assets held by households. For example, land prices, a robust measure for welfare impact of rural road projects, could increase with better transportation infrastructure (Jacoby 1998). Investments in roads and bridges are likely to generate new income opportunities for agricultural households, trickling far beyond the project site.11 11According to studies conducted elsewhere in the ECA region, the poor road quality can add 28 to 44 percent to usage costs. Also, the lack of either the means of transportation or satisfactory roads to reach markets makes commuting difficult for households. Hence, lowering transportation costs can determine 18 Several labor market studies have identified off-farm employment as the key driving force of welfare changes (Yemtsov 2001; Bernabe 2002). But access to rural labor and product markets appears to be an important constraint to disseminating the benefits of economic growth in rural Georgia (Cord et al. 2003). In assessing project effectiveness, it is also important to account for the indirect nature of benefits and their dependence on interactions with other investments, existing infrastructure, and geographical, community and household characteristics (van de Walle and Cratty 2002). Projects improving road infrastructure were completed by the end of 2000 in 41 villages, or 19 percent of the RCIS sample; 36 of these villages were also covered by the SGHH. We formed the initial control group from all villages without road and/or bridge projects completed between 1998 and 2001. Table 4 reports the estimation results for road and bridge infrastructure rehabilitation projects. The most immediate indicator of a road project outcome ­ time spent commuting to the district center shows a reduction by 35 minutes in project villages as opposed to only 29 minutes in the unmatched control group and 27 minutes in the PSM control sample. These differences, however, are not statistically significant. The change in indicators that are linked to the economic impact of the projects, as estimated from RCIS, is more pronounced. The share of villages reporting barter exchange among the main channels for marketing agricultural products has dropped significantly as a result of the road projects. Barter actually increased in both matched and unmatched control group villages. The share of villages with active small and medium non- agricultural enterprises has increased in project villages; this impact is insignificant when compared with the unmatched control group, but statistically significant compared with the PSM-matched control group. The last village-level indicator ­ changes in the subjective assessment of road quality ­ fails to react to road rehabilitation intervention. Similar to the community-level indicators, the indicators constructed using household-level data demonstrate both direct and indirect economic benefits from road rehabilitation projects. Time for an ambulance to arrive decreased in 23 percent of cases in the beneficiary villages. This compares favorably with the worsening of this indicator whether producers can remain competitive. Furthermore, it will have a dramatic effect on the poor because 19 in the PSM-matched control group. Focusing on the economic effects of the road projects, off-farm employment and female wage employment rates show increases in villages affected by road rehabilitation. These indicators declined in the control villages where these rates have declined. Indicators reflecting changes in the per capita market sales of agricultural products, however, show no improvement in the treated villages. The differences between the control and treatment groups in several economic outcome indicators, as well as differences in the rates of road accidents and in changes in average private per capita expenditures on transport, are not statistically significant. As we noted already, the effects of road rehabilitation projects could be difficult to capture. Nevertheless, we find some indications of positive changes due to projects: an increase in the number of small and medium non-agricultural enterprises, the reduction in commuting time, and a decline in the importance of barter trade. Other indicators related to conditions in the labor market and to changes in the value of market sales may take longer to materialize than the time horizon of our data allows. Finally, indicators of improved road safety impose high demands on data coverage because accidents rarely occur. Most of the impact indicators aggregated at the country level show only minor improvements over the time horizon of this analysis. But where positive changes occurred, they can be attributed to road project effects. Two-thirds of the increase in the number of villages with operating small and medium enterprises results from the effect of road rehabilitation interventions. The share of villages where an ambulance takes more than four hours to arrive has drooped by 11.3 percent between 2000 and 2001. As much as 5 percentage points of this change, i.e., close to half, can be related to the effect of road rehabilitation projects. Water system rehabilitation projects Water projects include a wide range of works ­ installation of new or repairs of existing communal water tanks, water treatment equipment, new pumps, repair or installation of new pipes and, to some extent, rehabilitation of wastewater management networks. As far it is poor households that generally tend to be located in very remote areas. See The World Bank 2003a. 20 as benefits from investments in water supply systems are concerned, we may expect a reduction in the incidence of water-borne disease (Jalan and Ravallion 2003), as well as a greater time available for schooling among children and for productive activities among adults, particularly women. Improvements in water quality could enable households to save resources going to alternative drinking water supplies (bottled or trucked-in water). The coverage of villages by water rehabilitation projects was less extensive in comparison with school or road projects. In RCIS, 17 villages (7 percent of the sample) had a water system rehabilitation project completed by the end of 2001. Only nine villages in this group were also covered by SGHH. The results of impact evaluation estimations are presented in Table 5 for the unmatched and PSM matched treatment and control groups. The list of drinking water supply channels expanded in 23 percent of project villages. In the control group, only 8 percent of villages in the unmatched and 5 percent in the matched sample reported a new water supply channel between 1998 and 2002. Household-level indicators also suggest improvements in access to water in villages with projects. The coverage of piped water supply has increased by 11 percent in the treated villages compared with no change or even slight deterioration of coverage in the control groups. The amount of time piped water is available increased substantially in the project villages as opposed to a sizeable decline in the matched control group. The comparison of changes in the incidence of water-borne diseases shows a significant beneficial effect of projects on health outcomes. Other impact indicators show changes in the expected direction (with the exception of changes in the female employment rate), but the differences between the treatment and control group averages are insignificant. Only a few impact indicators show significant effect of the water projects. This could be linked to two circumstances. First, our sample is too small to capture the effect of this type of intervention and only a limited set of indicators is available from RCIS and SGHH. For example, a low-income elderly household for which bottled water always has been too expensive would not be expected to report any increase in female wage employment or any savings on bottled water, after receiving piped water. But this household might experience significant improvements in quality of life along the dimensions not captured by the surveys, e.g., less time and effort spent in water 21 collection, easier housecleaning and bathing. Second, a distinct feature of water projects is a partial coverage of the population: in many villages only certain clusters of houses are connected to pipes and therefore direct beneficiaries of this intervention. As a result, the effect observed at the village level may not fully reflect the actual impact among the project beneficiaries. This issue will be addressed in the section dealing with the distributional impact of the projects. GSIF projects compared with other donors' projects In Georgia, projects of the same type are implemented by different agencies. For example, school rehabilitation projects are carried out by local administration, GSIF, USAID, Save the Children, and private donors. It might be relevant to compare project effectiveness across different donors. GSIF is believed to be the largest public investment program in Georgia and it is important to assess its relative performance. While applying the same methodology as described in previous sections, we form the treatment and control groups differently for this analysis. The treatment group consists of all villages with GSIF projects of a certain type completed by the end of 2000. Our control group is formed from the villages with completed projects of the same type financed by other donors. So, the measured impact shows the effectiveness of the GSIF interventions relative to that of the other agencies. The data availability (sample size) limits our analysis to school and water system rehabilitation projects. For the matched comparison GSIF school rehabilitation projects generated growth in school attendance and enrolment in excess of 30 percent over projects supported by other donors. The subjective assessments of access to education also demonstrate the superiority of GSIF interventions. The evidence for the relative effectiveness of water projects is mixed. GSIF water rehabilitation projects achieve better results in improving access to piped water, but the economic impact of GSIF-financed interventions is similar to the impact of other donors' projects.12 Several factors explain why GSIF supported projects demonstrate somewhat better results. GSIF, being the largest micro-finance operation in Georgia has a number of 12Detailed set of results is available from authors on request. 22 advantages. First, the size and number of GSIF micro-projects compare favorably with projects sponsored by other donors. For example, GSIF community-level funding averages USD 40,000-100,000 (depending on the type of project), while other donors typically contribute about USD 10,000 per site. Secondly, economies of scale within GSIF operations allow provision of high-quality engineering services to beneficiaries, a feature, that other donors running ad hoc small-scale interventions cannot guarantee. Finally, and probably most importantly, GSIF actively engages communities in design, implementation and maintenance of project sites, particularly emphasizing the aspect of sustainability. Distributional impact of infrastructure rehabilitation projects So far, we have looked at the impact of infrastructure rehabilitation projects at the village level. It is possible, however, that households within a village would benefit differently from a particular project. Jalan and Ravalion (2002) showed that in India, the impact of piped water projects is different for poor and non-poor households. Poor families might benefit relatively more from a project than wealthier households if publicly and privately provided inputs are substitutes. Because of the low level of private investment, the publicly subsidized substitute inputs would have higher marginal returns for the poor. For example, a high educational fee could be an obstacle for school enrolment. If part of these fees is subsidized (e.g., through a publicly financed school rehabilitation project) the enrolment rate among children from poor households could increase relatively more than that of children from wealthier households, for whom these fees are affordable in any case. If, on the other hand, public and private inputs play complimentary roles, one would expect households with higher incomes to benefit more from the project than households with low incomes. The impact of an intervention would depend on the amount of privately provided inputs. For example, households with cars could gain more from road rehabilitation. To assess the impact of infrastructure rehabilitation projects on the living standard of poor and non-poor households we apply DD methodology used in the previous 23 sections. We reconstruct our main outcome indicators using sub-samples of poor and non-poor households from each village covered by SGHH. We omit from this analysis community-level impact indicators from RCIS because these cannot be differentiated between poor and non-poor. Then we compare the impacts of different types of projects on poor and non-poor sub-samples. The results are shown in Table 6 for villages from treatment and control groups. Comparing the impact of school rehabilitation projects on poor and non-poor households, one can conclude that the poor benefit more from this type of intervention. In particular, the most sensitive indicator ­ improvements in school attendance ­ shows a significant effect of the school rehabilitation on the poor. The share of children from poor households missing classes declined by 11 percent as opposed to 3.5 percent for the non- poor households. Similarly, health outcomes improved more for children from poor households, an expected outcome given poor people's lack of warm clothes. Surprisingly, school enrolment rates demonstrate better response for children from non-poor households. Differences in changes in private educational expenditures are ambiguous. The distributional impact of road rehabilitation projects varies for different outcome indicators. The non-poor clearly benefited more from improvements of road infrastructure in terms of time needed for emergency medical assistance to arrive and number of opportunities for non-agricultural employment. Female off-farm employment rates, on the other hand, show greater project impact among the poor. An interpretation of the estimates based on the agricultural product sales indicator is difficult, as pointed out by other users of SGHH (World Bank 2003a). In recent years the sales of agricultural products plummeted for the whole country and the decline was particularly strong for the rich households. This is exactly what our analysis shows and we suggest that road quality is not the main driver in this process. Our results indicate that the key benefits from water projects are related to the improvements in the health status. Non-poor households gain more in relative terms than the poor: the incidence of water-borne diseases has dropped significantly for non-poor households living in villages covered by projects. Changes in this indicator for poor households were not statistically different between the treatment and control groups. 24 6. Conclusions This study investigates the impact of investments in infrastructure rehabilitation in rural Georgia. It evaluates the effect of various community-level projects on household wellbeing and assesses the resulting distributional impact. The study also attempts to provide some evidence on what type of rehabilitation yields the largest benefits for the poor. The analysis in the paper is based on combining household and community level survey data. Our empirical approach for project impact evaluation utilizes the panel structure of the data to control for time-invariant unobservables at the community-level by applying propensity-score-matched double difference comparisons. Our results indicate that improvements in school infrastructure produce nontrivial gains in school enrolment rates, raise school attendance, and reduce health risks of school-age children both at the village and country levels. Subjective assessments of access to education also improve. Road and bridge rehabilitation projects generate clear economic benefits at the community level. The number of small and medium enterprises increases while the importance of barter trade falls as a result of this type of interventions. At the household level, the access to emergency medical assistance improves unambiguously. However, the impact of road projects on labor market conditions is positive but insignificant. The results of water system rehabilitation interventions are less clear. The most unambiguous effect is observed for reduction of the incidence of water-borne diseases. The impact of water projects measured by other indicators is less clear-cut. To a large degree the ambiguity is related to the small number of project villages in our sample. We also found that projects of various types have different distributional impacts. School rehabilitation improves school attendance and children's health status among the poor more than it does among the better off. Road projects benefit the poor and non-poor in different ways. The non-poor gain more from improved accessibility to emergency medical assistance. Expansion of non-agricultural job opportunities favors women from poor households. The observed decline in the incidence of water-borne diseases could be fully accounted for by the improved health status of better-off households. This suggests that the benefits of water rehabilitation projects accrue mostly to the non-poor. One of the 25 policy implications of our findings could be the need for complementary behavioural change among poor households, in particular, educating households in the importance of boiling water. We can conclude with certainty that GSIF school rehabilitation projects achieved better outcomes compared with projects supported by other donors. The evidence for the relative effectiveness of water projects is mixed. GSIF water projects achieve better results in improving access to piped water, but their economic impact is similar to that of other donors' projects. The paper demonstrates that robust evaluation of project impact is possible even in the absence of proper baseline survey data. Specially designed community surveys (collecting proper retrospective information) in combination with ongoing nationally representative surveys could provide a feasible and low-cost alternative to standard "before-and-after" techniques. Findings of this study contribute to the Bank's initiatives in evaluating its impact on poverty by bringing new empirical evidence to bear on the welfare and equity implications of infrastructure interventions. The evidence presented in this paper provides an input to the development and implementation of a poverty reduction strategy for Georgia. By focusing on integrating impact on poverty into the project monitoring and evaluation system, the study offers an analytical input for the next stage of GSIF activities. 26 References Bernabč, S., (2002) "A Profile of the Labour Market in Georgia," ILO, UNDP: Tbilisi, Georgia Boarnet, M., (1997) "Infrastructure Services and the Productivity of Public Capital: The Case of Streets and Highways," National Tax Journal, Vol. 50(1): 39-57 Brockerhoff, M., and L., Derose, (1996) "Child Survival in East Africa: The Impact of Preventive Health Care." World Development Vol. 24(12): 1841-57 Castaneda Sabido, A., Cotler, P., and O., Gutierrez (2000) "The Impact of Infrastructure on Mexican Manufacturing Growth," Economia Mexicana, Vol. 9(2): 143-64 Chase, R., and L., Sherburne-Benz (2001) "Impact Evaluation of the Zambia Social Fund." The World Bank, Social Protection, Washington, D.C. Chase, R., (2003) "Supporting Communities in Transition: The Impact of the Armenian Social Investment Fund," World Bank Economic Review, Vol. 16(2): 219-240 Chen, S., and M. Ravallion (2003) "Hidden Impact? Ex-Post Evaluation of an Anti- Poverty Program." World Bank Research Paper Series #3049, The World Bank, Washington, D.C. Cord, L., Lopez, R., Huppi, M., and O. Melo, (2003) Growth and Rural Poverty in the CIS7 Case Studies of Georgia, the Kyrgyz Republic, and Moldova," paper prepared for the Lucerne Conference of the CIS-7 Initiative, January 20-22 Dehejia, R., and S. Wahba, (1999) "Causal Effects in Non-Experimental Studies: Reevaluating the Evaluation of Training Programs." Journal of the American Statistical Association Vol. 94(448): 1053-62 de Frutos, F., Gracia-Diez, M., and T., Perez-Amaral, (1998) "Public Capital Stock and Economic Growth: An Analysis of the Spanish Economy," Applied Economics, Vol. 30(8): 985-94 Glewwe, P., (1999) "The economics of school quality investments in developing countries: An empirical study of Ghana," Jaikishan Desai et al. Studies on the African Economies. New York: St. Martin's Press; London: Macmillan Press; in association with Centre for the Study of African Economies, University of Oxford Government of Georgia, (2003) "Economic Development of Poverty Reduction Program of Georgia," Draft, Tbilisi, Georgia GSIF, (2001) "Georgian Social Investment Fund: Mid-term Report," Tbilisi, Georgia Hanushek, E., (1995) "Interpreting Recent Research on Schooling in Developing Countries," World Bank Research Observer Vol. 10(2): 227-46 Heckman, J., Ichimura, H., and P. Todd, (1998), "Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Program," Review of Economic Studies Vol. 64(4): 605-654 Heckman, J., Ichimura, H., J., Smith and P. Todd, (1998), "Characterizing Selection Bias using Experimental Data," Econometrica, Vol. 66: 1017-1099 Jacoby (2002) "Access to Markets and the Benefits of Rural Roads." Economic Journal Vol. 110(465): 713-37 Jalan J., and M. Ravallion (2003) "Does Piped Water Reduce Diarrhea for Children in Rural India?" Journal of Econometrics, Vol. 112(1): 153-73 __________ (1998) "Are There Dynamic Gains from a Poor-Area Development Program?" Journal of Public Economics, Vol. 67(1): 65-85 27 Kremer, M., (1995) "Research on Schooling: What We Know and What We Don't: A Comment," World Bank Research Observer Vol. 10(2): 247-54 Lee, L., Rosenzweig, M. and M. Pitt, (1997) "The Effects of Improved Nutrition, Sanitation, and Water Quality on Child Health in High-Mortality Populations." Journal of Econometrics Vol. 77(1): 209-35 Newman, J., Pradhan, M., Rawlings, L., Ridder, G., Coa, A., and J. Evia (2002) "The Impact and Evaluation of Health, Education and Water Supply Investments by the Bolivian Social Investment Fund" The World Bank Economic Review, Vol. 16(2): 241-274 Orivel, F., (1998) "Cost and Finance of Education in Georgia." Mimeo, The World Bank Pradhan, M., and R. Rawlings, (2002) "The Impact and Targeting of Social Infrastructure investments: Lessons from the Nicaragua Social Fund," The World Bank Economic Review, Vol. 16(2): 275-295 Rawlings, L., and N., Schady (2002) "Impact Evaluation of Social Funds." The World Bank Economic Review, Vol. 16(2): 213-217 Rosenbaum, P., and D., Rubin, (1983) "The Central Role of the Propensity Score in Observational Studies for Causal Effects." Biometrika, Vol. 70: 41-55 Rubin, D., (1973) "The Use of Matched Sampling and Regression Adjustment to Remove Bias in Observational Studies." Biometrics Vol. 29: 159­183 Sanchez-Robles, B., (1998) "The Role of Infrastructure Investment in Development: Some Macroeconomic Considerations," International Journal of Transport Economics, Vol. 25(2): 113-36 State Department of Statistic Georgia (2003) "Survey of Rural Settlement Infrastructure." Report, Tbilisi, Georgia UN Office for the Coordination of Humanitarian Affairs (2003) "Directory of Humanitarian and Developments Sectoral Programs," Tbilisi, Georgia UN World Food Programme (2002) "Food assistance to Drought Affected People in Georgia," Project document 6302.00, Tbilisi, Georgia van de Walle, D., and D. Cratty (2002) "Impact Evaluation of a Rural Road Rehabilitation Project." Mimeo, The World Bank Walker, I., del Cid, R., Ordonez, F., and F. Rodriguez, (1999) "Ex-post Evaluation of the Honduran Social Investment Fund," Produced by ESA Consultants, Honduras, for the World Bank, Latin America and Caribbean Region. The World Bank (2002) "Georgia Poverty Update", January 10, 2002 (Report No. 22350- GE) __________ (2003 a) "Georgia: Trade Diagnostic Study." __________ (2003 b) "Achieving the Human Development MDGs in ECA." __________ (2003 c) "Georgia: Child Welfare Note." Forthcoming. __________ (2003 d) "Social and Economic Infrastructure in Rural Georgia." Mimeo. __________ (2003 e) "Georgia Country Assistance Strategy" Yemtsov, R., (2001) "Labor Markets, Inequality and Poverty in Georgia," IZA Discussion Paper 251, Institute for the Study of Labor (IZA), Bonn. 28 Table 1: Summary statistics for main outcome indicators. Beforea) After Change Source N Mean Std.Dev. Mean Std.Dev. Mean Std.Dev. All children are enrolled in school RCIS 249 0.683 0.466 0.590 0.493 -0.092 0.353 Number of pupils RCIS 246 293.781 278.019 283.129 292.574 0.244 0.431 Number of grads RCIS 249 23.450 21.955 21.635 22.520 0.339 0.474 Access to education RCIS 0.289 0.454 School enrolment rate SGHH 102 0.973 0.079 0.978 0.044 0.006 0.076 Share of missing more than 30 days SGHH 102 0.081 0.194 0.063 0.159 -0.009 0.111 Unsatisfactory schooling conditions SGHH 102 0.051 0.142 0.054 0.111 -0.015 0.153 Expenditures on schooling SGHH 100 11.755 10.456 48.470 45.612 1.119* 1.114 Incidence of respiratory diseases (child)SGHH 102 0.126 0.182 0.079 0.137 -0.065 0.163 Time to district capital RCIS -30.721 46.553 Subjective assessment of road (bad) RCIS 224 0.911 0.286 0.711 0.454 -0.223 0.428 Barter trade RCIS 249 0.494 0.501 0.498 0.501 0.004 0.168 Number of small enterprises RCIS 249 0.462 0.500 0.486 0.501 0.024 0.268 Time for ambulance to arrive SGHH 68 0.668 0.388 0.528 0.402 -0.113 0.334 Sales of agricultural products SGHH 104 158.122 244.790 114.760 150.807 -0.322* 0.959 Female off-farm employment SGHH 103 0.141 0.141 0.129 0.121 0.001 0.081 Nonagricultural employment SGHH 103 0.161 0.114 0.154 0.105 0.001 0.065 Household transport expenditures p/c SGHH 104 2.196 2.851 1.642 1.814 -0.133* 1.208 Incidence of trauma SGHH 102 0.001 0.005 0.001 0.005 -0.001 0.004 New water sources RCIS 249 0.092 0.290 Number of livestock RCIS 249 0.663 0.474 Piped water in the household SGHH 103 0.561 0.444 0.565 0.429 0.006 0.227 Hours of piped water supply SGHH 103 8.380 9.456 8.788 9.533 0.767 4.852 Incidence of water-borne diseases (total)SGHH 102 0.004 0.008 0.008 0.028 0.000 0.008 Incidence of water-borne diseases (child)SGHH 91 0.008 0.028 0.022 0.097 0.007 0.032 Expenditures on bottled water SGHH 89 1.608 0.872 1.583 0.895 -0.019* 0.421 a)Definition: For RCIS indicators "Before" stands for 1998 and "After" stands for 2002. For SGHH indicators "Before" stands for 2000 and "After" stands for 2001. Notes: *Change in the log of per capita values; Based on a direct change question in the survey; Recoded change indicator, the share of changes in the positive direction minus share of changes in the negative direction. 29 Table 2: Probit estimates of the probability of a village to participate in the project. School project Road project Water project Summary statistics Coeff. Std. Err. Coeff. Std. Err. Coeff. Std. Err. Mean Std. Err. School project 0.202 0.25 -0.727 0.54 0.245 binary Road project 0.129 0.294 -0.359 0.581 0.185 binary Water project -0.494 0.498 0.008 0.462 0.068 binary Total population 0.04 0.147 0.039 0.145 -0.008 0.262 7.046 1.082 If IDP in the village -0.027 0.254 -0.146 0.246 -0.466 0.421 0.482 binary Agriculture only -0.157 0.258 -0.09 0.247 -0.716 0.457 0.394 binary Experienced disaster 0.172 0.057 -0.032 0.06 0.053 0.08 1.956 2.087 Experienced flood -0.479 0.352 0.308 0.325 0.403 0.621 0.129 binary Mountain area 0.054 0.252 0.197 0.253 0.46 0.477 0.442 binary Alpine area 0.165 0.346 0.077 0.334 0.795 0.625 0.241 binary Distance to district center -0.129 0.129 -0.034 0.143 0.143 0.259 2.452 0.808 Distance to market 0 0.002 -0.004 0.005 0.003 0.003 24.388 49.308 Rail road 0.475 0.315 -0.075 0.385 -7.569 . 0.104 binary Interstate highway -0.017 0.267 -0.592 0.274 0.257 0.432 0.390 binary Asfalt road -0.011 0.257 0.702 0.247 -0.505 0.499 0.297 binary Number of schools -0.027 0.138 0.24 0.136 0.476 0.252 1.197 0.802 Number of large enterprises -0.009 0.034 -0.053 0.036 0.088 0.089 14.940 3.097 Number of small enterprises 0.204 0.253 0.084 0.249 -0.183 0.442 0.462 0.500 Police station 0.474 0.319 -0.247 0.334 0.179 0.546 0.169 binary Post office -0.209 0.244 0.293 0.24 0.282 0.402 0.558 binary Restaurant -0.1 0.32 0.118 0.31 -0.931 0.735 0.161 binary Proportion of household with a phone -0.199 0.442 0.501 0.405 1.669 0.548 0.088 0.284 Proportion of household with a toilet -0.589 0.507 -0.156 0.566 -2.468 0.945 0.956 0.206 Unreliable electric power supply 0.128 0.213 -0.026 0.226 1.179 0.416 0.462 binary Proportion of households with piped water 0.413 0.234 0.014 0.276 0.176 0.457 0.229 0.421 Proportion of buildings with wooden walls 0.107 0.346 0.891 0.267 -1.087 0.628 0.333 0.472 Proportion of buildings with dirt floors 0.147 0.449 -0.034 0.519 -7.348 . 0.056 0.231 Trade by the roadside -0.367 0.312 0.692 0.294 0.633 0.533 0.181 0.386 Kaheti (omitted category)* 0.137 binary Inner (Shida) Qartli 1.133 0.448 0.153 binary Lower (Kvemo) Qartli 0.489 0.51 0.141 binary Samskhe-Djavakheti 1.102 0.49 0.141 binary Achara 0.929 0.726 0.068 binary Guria 0.626 0.594 0.092 binary Samegrelo 0.963 0.666 0.076 binary Imereti 1.045 0.496 0.193 binary Constant -1.327 1.332 -1.155 1.302 -2.311 2.526 . . Sample size 249 249 249 249 Pseudo R2 0.156 0.181 0.393 Source: RCIS 2002 30 Table 3: Difference-in-difference estimates of the average impact of school rehabilitation projects. Unmatched sample Matched sample Treatment Control p-value Treatment Control p-value CIS outcome indicators All children are enrolled in school -0.066 -0.101 0.251 -0.066 -0.066 0.500 If Number of pupils increased 0.328 0.216 0.056 0.328 0.190 0.051 If number of grads increased 0.373 0.327 0.268 0.373 0.237 0.059 Access to education has improved 0.361 0.266 0.089 0.361 0.213 0.036 SGHH outcome indicators School enrolment 0.059 -0.004 0.102 0.059 0.000 0.117 Share of missing more than 30 days -0.057 -0.001 0.063 -0.057 0.020 0.019 Unsatisfactory schooling conditions -0.020 -0.014 0.584 -0.020 -0.013 0.611 Expenditures on schooling 1.249 1.094 0.365 1.249 1.544 0.772 Incidence of respiratory diseases (child) -0.120 -0.056 0.083 -0.120 -0.056 0.160 Table 4: Difference-in-difference estimates of the average impact of road and bridges rehabilitation projects. Unmatched sample Matched sample Treatment Control p-value Treatment Control p-value CIS outcome indicators Travel time to district center -35.577 -29.235 0.295 -35.577 -27.692 0.287 Subjective assessment of road (bad) -0.261 -0.214 0.259 -0.261 -0.326 0.731 Barter trade -0.044 0.015 0.038 -0.044 0.044 0.022 Number of small enterprises 0.044 0.020 0.308 0.044 -0.044 0.052 SGHH outcome indicators Time for ambulance to arrive -0.238 -0.070 0.088 -0.238 0.035 0.092 Sales of agricultural products -0.324 -0.322 0.496 -0.324 -0.184 0.287 Female off-farm employment 0.006 -0.001 0.397 0.006 -0.011 0.274 Nonagricultural employment 0.013 -0.004 0.199 0.013 -0.003 0.241 Household transport expenditures p/c -0.165 -0.121 0.442 -0.165 -0.407 0.758 Incidence of trauma -0.001 -0.001 0.396 -0.001 -0.003 0.764 31 Table 5: Difference-in-difference estimates of the average impact of water system rehabilitation projects. Unmatched sample Matched sample Treatment Control p-value Treatment Control p-value CIS outcome indicators New channels of water supply 0.235 0.082 0.086 0.235 0.059 0.041 Increase in livestock 0.647 0.664 0.447 0.647 0.529 0.272 SGHH outcome indicators Piped water in the household 0.110 0.002 0.216 0.110 -0.063 0.243 Hours of piped water supply 0.980 0.779 0.565 0.980 -0.785 0.665 Female wage employment -0.055 0.004 0.181 -0.055 -0.020 0.382 Incidence of water-borne diseases (total) -0.006 0.001 0.196 -0.006 -0.001 0.123 Incidence of water-borne diseases (child) 0.000 0.007 0.037 0.000 0.000 . Expenditures on bottled water -0.018 -0.018 0.500 -0.018 -0.027 0.514 Table 6: Poor versus Non-poor: Difference-in-difference estimates of the average impact of the project for three types of interventions. Poor Non-poor Treatment Control p-value Treatment Control p-value School rehabilitation School enrolment 0.012 0.000 0.403 0.041 -0.008 0.168 Share of missing more than 30 days -0.110 0.023 0.034 -0.035 0.012 0.050 Unsatisfactory schooling conditions 0.000 0.015 0.085 -0.023 -0.020 0.545 Expenditures on schooling 1.586 1.201 0.294 1.267 1.614 0.201 Incidence of respiratory diseases (child) -0.130 -0.054 0.244 -0.081 -0.069 0.436 Road and bridges rehabilitation Time for ambulance to arrive -0.188 -0.250 0.545 -0.237 0.039 0.105 Sales of agricultural products -0.251 0.231 0.150 -0.226 -0.094 0.290 Female off-farm employment 0.053 0.013 0.162 0.002 -0.012 0.317 Nonagricultural employment -0.002 0.018 0.330 0.021 -0.001 0.150 Household transport expenditures p/c 0.077 0.636 0.262 -0.118 -0.476 0.155 Incidence of trauma 0.000 -0.005 0.082 -0.002 -0.002 0.465 Water system rehabilitation Piped water in the household 0.121 0.062 0.368 0.105 -0.099 0.225 Hours of piped water supply 0.747 0.654 0.527 1.058 -1.434 0.310 Female wage employment 0.002 0.063 0.145 -0.083 -0.008 0.317 Incidence of water-borne diseases (total) -0.007 -0.004 0.361 -0.005 0.007 0.142 Incidence of water-borne diseases (child) 0.000 0.000 . 0.000 0.000 . Expenditures on bottled water 0.482 0.168 . -0.054 -0.051 0.494 32 Before matching After matching T reatment Control 3 3 noitubirts 2 2 diyti nseD 1 onitubirts dilor noitubirtsidde 1 onsitubirts di ontcfonae eatrtfo ofnae ean M M M 0 0 0 .21 .35 .5 1 0 .35 .5 1 Propensity score Propensity score Figure 1: School rehabilitation projects. Kernel density estimates of propensity score for villages from the treatment and control groups, before and after matching. Before matching After matching T reatment Control 4 4 noitubirts 3 3 diyti 2 nseD 1 noitubritsidlortn 2 noitubirtsidde 1 onsitubirst di cofo eatrtfo ofnae ean ean M M M 0 0 0 .16 .32 .5 1 0 .32 .5 1 Propensity score Propensity score Figure 2: Road and bridges rehabilitation projects. Kernel density estimates of propensity score for villages from the treatment and control groups, before and after matching. 33 Before matching After matching T reatment 6 6 Control 5 5 noitubirts 4 4 diyti 3 nseD onitubirts 2 dilor noitubirtsidde 3 2 ontc snoitubirtsidfo 1 ofnae eatrtfo ean 1 M M Mean 0 0 0 .05 .3 .5 1 0 .3 .5 1 Propensity score Propensity score Figure 3: Water system rehabilitation projects. Kernel density estimates of propensity score for villages from the treatment and control groups, before and after matching. 34