77477 Social Protection in a Crisis: Argentina’s Plan Jefes y Jefas Emanuela Galasso and Martin Ravallion The article assesses the impact of Argentina’s main socialpolicy response to the severe economic crisis of 2002. The program was intended to provide direct income support for families with dependents andwhose head hadbecome unemployed because of the crisis. Counterfactual comparisons are based on a matched subset of applicants not yet receiving program assistance. Panel data spanning the crisis are also used. The program reduced aggregate unemployment, though it attracted as many people into the workforce from inactivity as it did people who otherwise would have been unemployed. Although there was substantial leakage to formally ineligible families and incomplete coverage of those who were eligible, the program did partially compensate many losers from the crisis and reduced extreme poverty. Income transfer programs are a common social policy response to macroeconomic crises. Stated goals vary, but the common (explicit or implicit) goal is to help protect the living standards of families most adversely affected by the crisis. One of the largest recent programs is Argentina’s Plan Jefes y Jefas, introduced in January 2002 as a public safety net response to the severe economic crisis that hit Argentina at the end of 2001. Unemployment and poverty rates reached record levels (World Bank 2003). Jefes aimed to provide direct income support for families with dependents who had lost their main source of earnings due to the crisis. To ensure that the program reached those in greatest need, work requirements were imposed. With support from a World Bank loan (and equivalent counterpart funds from the government), the program expanded rapidly to cover about 2 million households by late 2002.1 Emanuela Galasso is an economist in the Development Research Group at the World Bank; her e-mail address is egalasso@worldbank.org. Martin Ravallion is research manager in the Development Research Group at the World Bank; his e-mail address is mravallion@worldbank.org. The work reported in this article is part of the ex post evaluation of the World Bank’s Social Protection VI Project in Argentina. The authors thank the staff of the government’s Institute of Statistics and Ministry of Labor, who helped greatly in assembling the data, and the World Bank’s manager for the project, Polly Jones, for her continuing support of the evaluation effort and many useful discussions. Paula Giovagnoli provided excellent research assistance. Helpful comments were received from Pedro Carneiro, Rosalı ´s, John ´a Corte Hoddinott, and anonymous World Bank Economic Review referees. 1. In 2002 the government of Argentina spent about US$500 million on Jefes, about a quarter of it financed through a World Bank loan. For 2003 the estimate is US$600 million, of which the bank loan will probably cover about half. The loan and counterpart funds cover mainly the payments to benefici- aries. Most costs for supplies and equipment for the workfare projects are covered by the local govern- ments or nongovernmental organizations sponsoring the projects. THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3, Ó The International Bank for Reconstruction and Development / THE WORLD BANK 2004; all rights reserved. doi:10.1093/wber/lhh044 18:367–399 367 368 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 Knowledge of the impacts of such programs has often been limited by a number of factors, including the speed with which crisis programs have to be scaled up and the paucity of appropriate survey data. Critics of the Jefes program have made claims about fraudulent participation, pointing to cases of registered participants who do not appear to satisfy the program’s eligibility criteria, or about weaknesses in the implementation and effectiveness of the program’s work requirements.2 At the other extreme, some have argued that the scheme was a big success in reducing poverty and unemployment in the aftermath of the crisis. One assessment claimed that Jefes accounted for the entire reduction in unemployment in the year follow- ing the crisis, which happened to roughly equal the increase in Jefes registrations over the same period (INDEC 2002c; World Bank 2003). Such claims often rest on transparently weak foundations. Anecdotes of abuse attract attention but may not be a sound basis for generalization. Claims about impact (positive and negative) often ignore behavioral responses. For example, it is unlikely that a program such as Jefes would not affect labor force participation choices. It is unlikely that all participants would have otherwise been unem- ployed. Similarly, the impact on poverty will be clearly overestimated if assess- ments ignore the forgone earnings of workfare participants, who are unlikely to be entirely idle in the absence of the program. The common failure to take full account of the costs to participants of targeted programs is known to be a serious deficiency of past evaluations (van de Walle 1998). Several factors make the Jefes program an unusually good case for rigorous study of impacts. Large household surveys were done just before the crisis, in October 2001, and one year later, in October 2002, and the second survey identified Jefes participants. One-third of the October 2001 sample was fol- lowed up in the later survey round. This article uses these survey data and the tools of nonexperimental program evaluation to address the following (related) questions about the Jefes program: . Who got assistance? Were the program’s eligibility criteria enforced? . How did participants respond to the program, such as through labor supply and household composition? Did participants come solely from the ranks of the unemployed? . What was the impact on the incomes of participating households? What share of the income loss due to the crisis was recovered through the program? . What was the distributional impact? . What was the impact of the program on aggregate unemployment and poverty? 2. See, for example, ERES (2004, annex 1) for a qualitative account. Examples of articles from the press include ‘‘Controversia por los planes de trabajo,’’ La Nacion, April 1, 2002; ‘‘En Santa Fe se venden Planes sociales,’’ La Nacion, May 13, 2002; ‘‘Escandalo Cordobes por el reparto de subsidios sociales sin control,’’ Pagina/12, May 21, 2002; ‘‘Denucian que no hay control en la ayuda social,’’ La Nacion, August 28, 2002. Galasso and Ravallion 369 In addressing these questions, a key issue is finding a valid comparison group who have similar characteristics to the Jefes participants but did not enroll in the program. This study exploits the fact that because the program was in a period of rapid scaling up, there were many applicants who had not yet received benefits. This group has advantages as the source of a comparison group, though the possibility of selection bias (that current participants are different ex ante to the current applicants) must also be considered. Current participants might have experienced larger income shocks as a result of the crisis and so were the first to join the program. Another possibility is that administrative assign- ment favored certain groups, possibly working against the program’s espoused objectives. Matching methods and longitudinal observations are used to address these concerns, comparing current circumstances for both participants and applicants with a precrisis baseline. I. THE CRISIS AND THE GOVERNMENT’S RESPONSE THROUGH THE JEFES PROGRAM Argentina fell into a severe economic crisis at the end of 2001. Widespread concerns about the impending collapse of the convertibility plan (which pegged the Argentine peso to the U.S. dollar) and possible default on external debt led to draconian measures to prevent withdrawals of bank deposits. The final collapse of the convertibility plan and the subsequent sharp devaluation and default on foreign debt, combined with a freeze on deposits, resulted in a large contraction in national output. The immediate welfare impacts were severe. McKenzie (2004) finds that three-quarters of households surveyed experienced real income declines in 2002, with the majority of them suffering a real income fall of 20 percent or more. Indicators of poverty rose sharply (Fiszbein and others 2002; World Bank 2003). The government’s statistics office estimated that the proportion of people living below the poverty line rose from 37 percent just prior to the crisis (October 2001) to 58 percent a year later (World Bank 2003). Unemployment also rose, though McKenzie’s (2004) results suggest that this contributed far less to falling living standards than did the shock to real wages.3 Widespread political and social instability ensued. As the government’s main safety net response to this crisis, Jefes provided a cash transfer of 150 pesos a month to each eligible individual, representing about half of the mean household income per capita in Argentina in 2002. Those formally deemed eligible to participate were unemployed household heads with dependents (children under age 18 or people with disabilities). 3. In particular, McKenzie (2004) finds that about three-quarters of the average fall in total real household income between October 2001 and May 2002 can be attributed to a fall in real wages for workers remaining in the same job, whereas only 10 percent is due to losses from household members exiting their jobs. 370 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 Program participation had to be requested through the local municipality or through local offices of the Ministry of Labor. Jefes replaced the smaller-scale Trabajar workfare program. Trabajar had a tightly enforced work requirement of 30–40 hours a week, with targeting criteria to help ensure that the work was of value to residents of poor commu- nities. Trabajar was found to have been effective in reaching the poorest, both as workers and residents (Ravallion 2000; Jalan and Ravallion 2003). For exam- ple, 80 percent of Trabajar workers came from the poorest 20 percent of the population (Jalan and Ravallion 2003). Because of the magnitude of the crisis, the government’s explicit aim for the Jefes program was to reach a broader segment of the population than Trabajar. At its inception Jefes was advertised as a ‘‘universal’’ program, meaning that anyone who was eligible and wanted the transfer could get it. Contrary to its predecessor, Jefes did not have an explicit poverty focus. However, genuine universality for eligible households was clearly not sustainable. In early 2002 concerns emerged about the budgetary cost of Jefes. There were signs (based largely on anecdotal evidence) that people who were not the most in need were capturing many of the program benefits. Ministry of Labor data based on registration records indicated that over half of Jefes participants were women and probably not heads of households. Administrators were not able to check whether an applicant was really a head of household. There were also anecdotal claims that to cope with the liquidity crisis municipalities and pro- vinces were signing up their employees and that local civil servants were sending their wives (who were not in the workforce) to sign up. Possibly, the program’s benefits were spilling over heavily to people who were not much affected by the crisis or who had the personal resources to cope adequately on their own. At the heart of this concern is the fact that verification of unemployment is problematic in Argentina, where over half of employment is in the informal sector. All that the administrators could reasonably verify with confidence was whether an applicant had a formal sector job, and so was registered as such. Prompted by these concerns, a work requirement was introduced in early 2002, with the aim of helping ensure that the transfers reached those in greatest need.4 The work requirement was not as demanding as that for the Trabajar program. Participants were required to do a minimum of 20 hours a week of basic community work, training activities, or school attendance. Alternatively, beneficiaries could find employment in a private company and receive a wage subsidy for six months. The municipalities (together with local nongovernmen- tal organizations) were in charge of organizing the work activities, and provin- cial offices of the Ministry of Labor and municipal and provincial councils were responsible for monitoring the work activities. 4. As a condition for financing the program, the World Bank insisted that the vast majority (90 percent was the target) of Jefes participants had to meet the work requirement. Galasso and Ravallion 371 Because poor people tend to have lower reservation wages, the work require- ment is likely to target the poor.5 But it is not clear how effective the Jefes work requirement was in practice compared with Trabajar because of the weak capacity to organize, supervise, and enforce the work requirement at the local level in such a large program. The program’s evolution (the work requirement was something of an afterthought), its rapid scaling up, and the circumstances of the crisis may have made it hard to enforce the work requirement. The work requirement is self-targeting only insofar as participants have to comply with it to obtain the transfer. The behavioral responses to such a crisis and to such a large public program as Jefes are clearly of interest. Various responses could be expected. Some have argued that all participation in Jefes should be counted as a commensurate reduction in unemployment (INDEC 2003). This clearly ignores possible beha- vioral responses to the program through other labor supply decisions, either to participate in the workforce or to change the number of hours worked. Household composition could also change as a response to such a shock, by delaying the formation of new households (Foster and Rosenzweig 2002), or as a response to the public transfers, by changing living arrangements (Duflo 2000). Splitting up households, with parents ‘‘sharing’’ children and applying to the program separately, has been reported anecdotally as a response to the Jefes program. Behavioral responses are also relevant for assessing impacts on poverty. Following common practice, INDEC (2002b) calculated the program’s poverty impact by subtracting the Jefes payment from the incomes of participants. Thus the poverty rate in the absence of the program was calculated from the simu- lated distribution of net incomes. However, this ignores the fact that partici- pants are unlikely to have remained idle in the absence of the program but would have found some sort of work, possibly doing casual odd jobs. Ignoring participants’ forgone incomes clearly leads to overestimation of the poverty reducing impact of the program. II. DATA AND DESCRIPTIVE RESULTS Data were taken from the October 2001 and 2002 rounds of the Permanent Household Survey (EPH) conducted by Argentina’s Statistical Institute (INDEC). The survey collects information on employment, incomes, education, and household demographics in large urban areas and covers about 70 percent of the population. A subset of the sample is linked as a panel, with about a third of the sampled households in 2001 reinterviewed in 2002. For this study a special survey module on Jefes participation was administered in October 2002 to adult household 5. Supportive evidence on this assumption for Argentina can be found in the results of Jalan and Ravallion (2003) on the Trabajar program. 372 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 members for whom Jefes was not the main occupation. (The existing survey was deemed adequate for those for whom Jefes was the primary occupation.) The grossed-up aggregate participation rate in Jefes taken from the EPH was compared with the administrative data on aggregate registrations (see the appendix). The comparison was complicated by the fact that the Jefes program is national in coverage, whereas the EPH sample frame excludes 30 percent of the population. A comparison based on place of residence finds that the grossed-up EPH count of Jefes participants accounts for 91 percent of the administrative aggregate. This leaves a significant discrepancy at the 5 percent level, though just barely: At the upper bound of its 95 percent confidence interval, the survey estimate accounts for 99 percent of registered participants. This suggests that there is unlikely to be any serious undercounting of Jefes participation in the EPH related to its sample frame. Another question is how Jefes eligibility should be defined in terms of the EPH data. Beneficiaries signed statements certifying that they were unemployed and a head of household. However, the only signal of unemployment status that could be reliably checked by the authorities was whether an individual was participat- ing in the formal labor market. Thus a definition of eligibility was used for this study that is close to what could be enforced by program administrators. A sampled adult was considered eligible if he or she was not employed in the formal labor market and lived in a household with a child (less than 18 years old and belonging to the head or the spouse) or a person with a disability. (Some important differences between this practical eligibility definition and the official theoretical eligibility definition are considered later in the article.) By this definition about a third of the people receiving the program were not eligible (table 1). About 80 percent of economically active individuals who were eligible (although not necessarily poor) did not get into the program. Applicants T A B L E 1 . Errors of Inclusion or Exclusion in the Jefes Program, October 2002 Ineligible Eligible Total Number Percent Number Percent Number Percent Applicants and participants Not receiving Jefes 677 14.1 824 17.1 1,500 31.2 Receiving Jefes 994 20.7 2,311 48.1 3,305 68.8 Total 1,671 34.8 3,134 65.2 4,805 100 All economically active adults Not receiving Jefes 22,285 71.0 6,763 21.6 29,047 92.6 Receiving Jefes 656 2.1 1,671 5.3 2,327 7.4 Total 22,940 73.1 8,434 26.9 31,374 100 Note: A person is deemed to be ‘‘eligible’’ if he or she lives in a household with dependents (children of the household head younger than 18 or a person with a disability) and is not in the formal labor market, as indicated by receipt of formal job benefits. Source: Authors’ calculations based on data from the October 2002 EPH. Galasso and Ravallion 373 not yet receiving the benefit were more likely to be ineligible than were current recipients. The average Jefes participant in the sample is female (69 percent of partici- pants, compared with 43 percent for all economically active adults), 36 years old, married, not a head of household (for 57 percent of participants), and has 8 years of schooling (table 2). Jefes participants are less likely to be heads of households than the sample of all economically active adults and more likely to be spouses of heads. The participants tend to come from larger than average households—5.4 people per participating household, compared with 4.2 for all economically active adults—with the difference accounted for by a greater number of children in Jefes households.6 Jefes households are poorer on aver- age, with a per capita household income of about 30 percent of the average for all economically active adults. Netting out the Jefes transfer payment reduces the per capita household income to 17 percent of the average for all economic- ally active adults. Jefes participants and applicants tend to have similar char- acteristics, a finding examined more carefully later using a multivariate model. The households of Jefes participants tend to be poorer on average than the households with eligible heads. Although there is a high incidence of ineligibility among Jefes participants and limited coverage of eligible households, ineligible Jefes participants are less poor than eligible households. A comparison of the empirical cumulative distribution function of per capita household income for eligible and ineligible participating households shows a first-order dominance— no matter what poverty line is used, eligible participants are poorer than ineligible participants (figure 1).7 Most of the eligibility violations relate to the dependency criterion. Tighter enforcement of this criterion would improve the program’s performance in reaching the poor, albeit only slightly. The precrisis baseline survey for October 2001 shows that 43 percent of Jefes participants as of October 2002 had been employed a year earlier, 38 percent were inactive, and 19 percent were unemployed (table 3). The unemployed participants were more likely to be in the bottom decile of the income distribution.8 Jefes participants and applicants have similar baseline characteristics in the panel sample. One possible source of bias in the use of Jefes applicants as a comparison group is that participants may have experienced larger income shocks in the crisis than did the applicants who had not yet joined the program. In that case 6. Note that the extent of multiple participants in the same household is limited: 13 percent of participating individuals live in household with more than one beneficiary, and 7 percent of households have more than one beneficiary. 7. This holds for a broad class of additive poverty measures (Atkinson 1987). 8. More precisely, 26 percent of previously unemployed participants are in the bottom decile of the per capita income distribution in 2001, compared with 16 percent of the previously economically inactive participants and 11 percent of the previously employed participants. T A B L E 2 . Descriptive Statistics as of October 2002, Cross-Section Jefes Participants Jefes Applicants Eligible Heads or Spouses Economically Active Adults (Age 18–65) Characteristic Mean St. Dev. Mean St. Dev. Mean St. Dev. Mean St. Dev. Individual demographics Male 0.31 0.46 0.39 0.49 0.41 0.49 0.57 0.49 Age 35.8 11.1 37.1 13.5 38.9 10.2 37.9 12.1 374 Single 0.18 0.38 0.19 0.39 0.02 0.15 0.29 0.45 Married 0.68 0.46 0.64 0.48 0.91 0.28 0.61 0.48 Head 0.43 0.49 0.44 0.49 0.49 0.50 0.49 0.49 Spouse of head 0.34 0.47 0.31 0.46 0.51 0.50 0.21 0.4 Son or daughter of head 0.16 0.36 0.17 0.37 0 0.57 0.42 Years of education 8.07 3.14 8.17 3.29 9.28 3.65 10.77 3.91 Employment status Jefes main activity 0.72 0.45 Doing work requirement (min. 20 hours) 0.83 0.37 if Jefes is main activity Jefes secondary activity 0.28 0.44 Doing work requirement (min. 20 hours) 0.16 0.36 if Jefes is secondary activity Doing work requirement (min. 20 hours) 0.64 0.47 Employed 0.84 0.36 0.31 0.46 0.52 0.50 0.17 0.38 Unemployed 0.06 0.24 0.36 0.48 0.14 0.34 0.83 0.38 Inactive 0.10 0.29 0.33 0.47 0.35 0.48 – – Total hours worked 19.8 14.2 11.6 21.5 32.5 23.7 Total hours worked = 0 0.14 0.34 0.65 0.47 0.19 0.39 Household characteristics Household size 5.42 2.42 4.89 2.4 4.83 1.83 4.23 2.05 Number children < 18 years 2.67 1.87 2.08 1.8 2.33 1.51 1.34 1.55 Total household income 420.9 302.1 350.2 323.8 647.3 917.3 985.6 1139.8 Per capita household income 84.1 59.2 77.4 71.87 150.1 226.8 271.8 378.8 Total household income net of Jefes 246 292.6 Per capita household income net of Jefes 46.8 56.1 Eligibility criteria for Jefes 1 0 0.12 0.32 0.07 0.26 Household with children of head < 18 years 0.80 0.39 0.66 0.47 1 0.53 0.49 or disabled member Household with any children < 18 or 0.95 0.22 0.84 0.36 1 0.62 0.48 disabled member Individual is formal sector worker 0.02 0.14 0.02 0.16 0 Household has at least one 0.15 0.35 0.19 0.39 0.28 0.44 0.53 0.49 formal sector worker 375 Eligible individual (2) any children, 0.93 0.25 0.82 0.38 0.80 0.40 0.41 0.49 individual not formal sector worker Eligible individual (1) children of head, 0.69 0.45 0.54 0.49 1 0.26 0.43 individual not formal sector worker Eligible household (household with 0.79 0.40 0.64 0.47 1 0.45 0.49 at least one eligible individual (1) Number of observations 3,092 1,713 13,934 31,374 Source: Authors’ calculations based on data from the October 2002 EPH. 376 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 F I G U R E 1 . Eligibility of Jefes Participants: Cumulative Distributions of Income Postintervention, October 2002 Source: Authors’ calculations based on data from the October 2002 EPH. the measured income losses for applicants during the crisis would underestimate the counterfactual income losses for participants. The likely extent of mismatch in terms of income shocks can be assessed by comparing income changes under alternative assumptions about the share of forgone income when constructing the counterfactual income of Jefes partici- pants and then calculating the corresponding income shock using the panel data. Comparing the distribution of shocks between Jefes participants and applicants gives a sense of the extent of the bias under alternative hypotheses on the net gains from the program. If the identifying assumption holds, the expected change in income in the absence of the program should be the same for participants and applicants. The results show that the expected change in income in the absence of the program is balanced across the two groups under the assumption of forgone income of about one-third to one-half, which is consistent with the estimates by Jalan and Ravallion (2003) for the Trabajar program (table 4). However, the tighter work requirements under Trabajar could mean that Jefes forgone incomes are lower. The estimated income changes based on the preferred estimates (explained later) of forgone income of Jefes participants are presented in the last column of table 4. T A B L E 3 . Descriptive Statistics as of October 2001, Panel Sample Jefes Participants Jefes Applicants Eligible Heads or Spouses Active Adults (Age 18–65) Characteristic Mean St. Dev. Mean St. Dev. Mean St. Dev. Mean St. Dev. Individual demographics: Male 0.29 0.45 0.46 0.5 0.39 0.49 0.59 0.49 Age 35.79 11.17 37.3 13.36 39.64 10.19 38.83 12.15 Single 0.2 0.4 0.24 0.43 0.02 0.15 0.28 0.45 Married 0.69 0.46 0.63 0.48 0.91 0.28 0.62 0.48 Head 0.37 0.48 0.44 0.5 0.47 0.50 0.51 0.50 Spouse of head 0.39 0.49 0.3 0.46 0.53 0.50 0.20 0.40 Son or daughter of head 0.19 0.39 0.19 0.39 0.00 0.00 0.24 0.43 Years of education 8.24 3.2 7.94 3.34 9.05 3.72 10.59 3.94 Employment status Employed 0.43 0.5 0.44 0.5 0.46 0.50 0.83 0.37 Unemployed 0.19 0.39 0.19 0.39 0.14 0.34 0.17 0.37 377 Inactive 0.38 0.48 0.36 0.48 0.41 0.49 0 Total hours worked 13.9 21.5 14.6 22.7 18.7 26.2 34.31 24.3 Hours worked = 0 0.58 0.49 0.57 0.49 0.55 0.47 0.19 0.39 Employment status * gender Male * employed 0.66 0.48 0.56 0.5 Male * unemployed 0.26 0.44 0.28 0.45 Male * inactive 0.08 0.08 0.15 0.08 Female * employed 0.36 0.48 0.33 0.36 Female * unemployed 0.16 0.32 0.11 0.32 Female * inactive 0.48 0.5 0.55 0.5 Household characteristics Household size 5.58 2.51 5.12 2.61 4.96 1.88 4.35 2.12 Nominal household income 426.4 366.7 427.2 369.1 692.8 998.3 980.2 1130.2 Nominal per capita household income 84.5 81.3 98.4 95 156.5 237.9 279.9 363.2 (Continued) T A B L E 3 . Continued Jefes Participants Jefes Applicants Eligible Heads or Spouses Active Adults (Age 18–65) Characteristic Mean St. Dev. Mean St. Dev. Mean St. Dev. Mean St. Dev. Eligibility criteria for Jefes Household with children of head < 18 years 0.80 0.40 0.67 0.46 1 0.53 0.49 or handicapped member Household with any children < 18 years or 0.94 0.22 0.79 0.40 0.61 0.48 disabled member 378 Individual is formal sector worker 0.03 0.18 0.06 0.23 0 0.39 0.48 Household with at least one formal sector worker 0.19 0.40 0.25 0.43 0.29 0.46 0.56 0.49 Eligible individual (children of head, individual not 0.67 0.47 0.52 0.50 1 0.26 0.43 formal sector worker) Eligible household (household with at least one 0.78 0.41 0.65 0.47 1 0.44 0.49 eligible individual) Number of observations 1222 679 5,273 11,401 Source: Authors’ calculations based on data from the October 2001 EPH. Galasso and Ravallion 379 T A B L E 4 . Distribution of Shocks: Actual and Simulated Changes in Real Household Income between October 2001 and October 2002 Actual Jefes, Assuming: Zero One-Third One-Half Estimated Jefes Jefes Forgone Forgone Forgone Forgone Percentile Applicants Participants Income Income Income Income Household income 1 À2,187.9 À1,688.5 À1,838.5 À1,788.5 À1,763.5 À1,759.8 5 À994.0 À1,004.6 À1,154.6 À1,104.6 À1,079.6 À1,127.9 10 À730.9 À647.7 À797.7 À747.7 À722.7 À756.7 25 À410.3 À306.7 À456.7 À406.7 À381.7 À398.2 Median À168.2 À68.2 À218.2 À168.2 À143.2 À172.1 75 0.0 103.0 À47.0 3.0 28.0 22.1 90 123.3 252.7 102.7 152.7 177.7 169.7 95 280.0 364.5 214.5 264.5 289.5 272.8 99 500.0 685.7 535.7 585.7 610.7 564.1 Mean À258.9 À151.7 À301.7 À251.7 À226.7 À242.1 St. Dev. 450.6 434.1 434.1 434.1 434.1 437.5 Individual income 1 À775.2 À825.8 À975.8 À925.8 À900.8 À1,032.2 5 À446.1 À337.9 À487.9 À437.9 À412.9 À469.8 10 À318.2 À198.5 À348.5 À298.5 À273.5 À318.0 25 À139.4 À59.1 À209.1 À159.1 À134.1 À144.8 Median 0.0 140.0 À10.0 40.0 65.0 45.8 75 0.0 150.0 0.0 50.0 75.0 77.4 90 88.5 150.0 0.0 50.0 75.0 90.1 95 150.0 190.0 40.0 90.0 115.0 109.4 99 293.0 300.0 150.0 200.0 225.0 181.8 Mean À77.3 28.8 À121.2 À71.2 À46.2 À61.2 St. Dev. 213.3 214.6 214.6 214.6 214.6 226.3 Note: The estimated forgone income is based on the single-difference estimates on individual income in October 2002; see discussion in text. Source: Authors’ calculations based on data from the October 2001 and 2002 EPH. In the precrisis period, 15 percent of participants were in the lowest decile of the income distribution, 40 percent were in the poorest 20 percent, and 90 percent of participants were among the poorest 60 percent, which was about the official poverty rate at the time (table 5).9 At the same time, unemployed heads of households with dependents were highly concentrated in the bottom 20 percent of the income distribution. Based on the eligibility criteria of the pro- gram’s official aim (recall that this study uses a weaker definition, closer to what could be implemented in practice), the theoretically eligible population turns out to be quite narrowly defined at just 5 percent of the population at the baseline. 9. The location of Jefes participants and applicants in the national distribution of income (tables 4–6) is unaffected if the tabulations are based on income adjusted for adult equivalents rather than income per capita. 380 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 T A B L E 5 . Initial Location of Participants, Applicants, and Eligible Economically Active Adults in the National Distribution of Household Income by Decile in October 2001, Panel Sample (%) Jefes Participants Jefes Applicants Eligible Eligible Theoretical Decile Households Individuals Households Individuals Households Individuals Eligibilitya 1 18.47 15.94 14.79 15.30 12.3 14.2 30.7 2 32.99 25.75 27.80 20.95 14.6 16.2 26.9 3 21.25 14.39 23.76 14.90 10.3 10.9 7.8 4 10.56 16.57 9.98 17.26 12.7 12.7 11.8 5 7.99 10.28 7.14 9.41 12.1 11.8 8.5 6 4.07 7.87 6.12 6.85 11.1 10.1 4.7 7 3.44 4.99 7.38 7.07 8.2 7.3 4.1 8 0.17 3.14 1.97 5.30 8.1 7.4 3.7 9 0.59 0.29 0.83 2.59 6.3 5.5 1.7 10 0.46 0.78 0.22 0.37 4.4 4.1 0.1 a Household with an unemployed head who has eligible dependents and who is the Jefes participant in the household. Source: Authors’ calculations based on data from the October 2001 EPH. Only 12 percent of the sample of household heads with dependents (45 percent of heads) were unemployed as of October 2001. In contrast to the theoretical target population, the practical eligibility cri- teria are quite broad and only slightly progressive (table 6). If perfectly enforced, these criteria would allow Jefes to reach about 50 percent of the poor at a poverty line of about 100 pesos. Concentration curves showing the distribution of gains from the program selected using the cross-section samples show that Jefes is clearly not as well targeted as Trabajar (figure 2), consistent with expectations that the work requirement was not as tightly enforced in Jefes. This is true both for an assumption of zero forgone income and for the preferred estimates of forgone income (see table 6). However, spending on Jefes appears to be better targeted than other categories of social spending in Argentina. Gasparini (1999, quoted in World Bank 1999) estimates concentration curves for overall social spending that indicate that the poorest 20 percent of the population receive 22 percent of outlays (30 percent for the subcomponent of social services) and the next poorest 20 percent receive 20 percent (19 percent for social services).10 10. Analogously, the concentration curve shows that targeting performance is better for Jefes than for a median transfer program by international standards and by Latin American standards, as indicated by the results of Coady and others (2002). A median social assistance program in Latin America is 60 percent more progressive than a neutral allocation (compared with 25 percent of a median transfer program in developing economies). For comparison with other programs mentioned in Coady and others the benefit shares are 0.20 for the bottom decile, 0.47 for the bottom quintile, and 0.91 for the bottom two quintiles (from table 6, including estimated forgone income). T A B L E 6 . Location of Trabajar and Jefes Participants in the Cross-Sectional Distribution of Income by Household Income by Decile (%) With Zero Forgone Income With Estimated Forgone Income Decile Trabajar Participants 1997 Jefes Participants 2002 Trabajar Participants 1997 Jefes Participants 2002 (Net of Eligible Eligible Transfer) Households Individuals Households Individuals Households Individuals Individuals Households Individuals 1 58.2 60.1 28.9 29.0 13.5 14.4 48.1 3.6 19.5 2 17.5 18.5 23.2 23.5 11.3 13.6 27.7 41.0 27.4 3 9.9 9.5 18.6 18.6 12.2 13.7 13.5 27.7 20.2 381 4 6.8 5.8 12.9 13.1 12.6 12.7 7.3 13.9 15.2 5 2.2 1.9 8.9 8.5 11.7 10.8 1.7 8.7 9.1 6 2.5 1.6 5.1 4.9 11.0 10.2 1.7 (Deciles 6–10) 3.3 5.2 7 1.7 1.6 1.5 1.6 8.9 8.2 1.3 2.3 8 0.6 0.5 0.6 0.5 7.2 6.5 0.4 0.9 9 0.4 0.3 0.3 0.3 6.8 5.9 0.1 0.2 10 0.2 0.1 0.1 0.1 4.9 4.2 0.1 0.1 Source: Authors’ calculations based on data from the 1997 Social Development Survey (EDS) for Trabajar, with zero forgone income and net gains estimates from Jalan and Ravallion (2003, tables 2 and 5); data from the October 2002 EPH for Jefes. 382 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 F I G U R E 2 . Concentration Curves for the Jefes and Trabajar Programs with No Forgone Income, October 2002 Note: Concentration curves for zero forgone individual income; see table 7. Source: Authors’ calculations based on data from the October 2002 EPH. III. METHODS OF ASSESSING IMPACTS Following common practice in the evaluation literature, impact is defined as the difference between the outcome indicator with the program and without the program.11 Also following common practice, the estimate of the counterfactual is based on a matched comparison group of nonparticipants. As with all evalua- tions the reliability of this method depends crucially on whether the comparison group is sufficiently similar to participants in the absence of the program. As a first step, individuals who have applied to the program but have not yet received assistance are selected for the comparison group. These applicants have already indicated a preference toward participation in the program (Angrist 1998). So to some extent, unobserved factors influencing participation (such as shocks associated with the crisis) are already revealed by the applicants. However, latent heterogeneity between participants and applicants that can bias impact estimates cannot be ruled out. As noted, the applicants are less likely to 11. This evaluation focuses on short-run partial equilibrium effects of the program. A referee pointed out the potential underestimation of the impact of the program due to the possibility of feedback effects on the labor market. In a situation of high unemployment and with a cash transfer of only 150 pesos a month (well below the minimum wage), however, it seems unlikely that equilibrium wages were affected much by the program. A possible indirect effect might arise through changes in the search behavior of workers in the labor market: With no time limit on participation and to the extent that the work requirement is not binding for some groups, participants might become dependent on the scheme. Galasso and Ravallion 383 satisfy the eligibility criteria than are the current participants (see table 1). To control for observable heterogeneity, propensity matching techniques were used to construct a counterfactual outcome from the sample of applicants, in which Di (an indicator of participation in Jefes) equals 1 if individual i participates and 0 otherwise. Following Rosenbaum and Rubin (1983), matching methods are used to estimate the outcome without the program by taking weighted averages over outcomes for individuals who did not participate and that are observationally similar to those of participants in terms of their propensity score, where PðXi Þ ¼ ProbðDi ¼ 1jXi Þ is the probability of participating conditional on observed (pre- determined) covariates Xi . This leaves the problem of selection bias due to unobservable characteristics. To reduce this bias, a subsample of panel households interviewed in the baseline (October 2001) and after the program (October 2002) was used to obtain a double difference (sometimes called difference-in-difference) impact estimator. This eliminates any time-invariant additive selection bias due to unobserved heterogeneity between participants and applicants. Matching in combination with double difference has been found to be effective in eliminating selection bias due to time-invariant omitted effects that might matter to participation (see, for example, Heckman and others 1997). The panel sample allows exam- ination of how impact varies according to differences in baseline characteristics. Data for October 2002 are available on N participants, indexed i ¼ 1; � � � ; N and C comparators, j ¼ 1; � � � C in the region of common support, given by the set of propensity scores for which there is positive density for both treatment and comparison groups. Imposing common support means that inferences on the impact of the program can be confined to ‘‘comparable people’’ in terms of their propensity scores.12 The smaller panel sample contains n and c individuals k in the matched treatment and comparison groups. Let Yit be the outcome of interest for individual i at time t in state k. There are two possible states for the outcome: k ¼ 1 in the presence of the program, and k ¼ 0 in its absence; there are two possible dates t ¼ 0 (October 2001) and t ¼ 1 (one year later, when program participation is observed). The evaluation problem of estimating the impact of any program stems from the impossibility of observing simultaneously both states for the same individual. Because nobody participates at the baseline, Di is used to denote Jefes participation at t ¼ 1. Note that Yi0 1 0 ¼ Yi0 for all i. The matched single-difference estimate of the mean impact is XN XC ð1Þ SD = ð1=N Þ i¼1 ðYi1 1À j=1 sd 0 Wij Yj1 Þ 12. Heckman and others (1997) show that failure to satisfy the common support condition is a major source of bias in nonexperimental evaluations. 384 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 sd where the Wij is the weight used in calculating the counterfactual for each participant. Local linear weights are used because they have been found to perform better at the boundaries of the scores, where the extent of the bias is greatest for conventional methods (Heckmanand others 1997). SD identifies the impact of the program in expectation if there is no selection bias; the condition for unbiasedness 0 0 is that (dropping i subscripts): EðY1 jPðXÞ; D ¼ 1Þ ¼ EðY1 jPðXÞ; D ¼ 0Þ, where the expectation is taken over the distribution of unobservables. The matched double difference is estimated on the matched panel sample and is given by:13 Xn Xc ð2Þ DD ¼ ð1=nÞ i¼1 ½Yi1 1 1 À Yi0 À j¼1 dd Wij ðYj0 0 1 À Yj0 ފ This yields an unbiased estimate of impact if the selection bias is time invariant 0 0 0 0 and additive: EðY1 À Y0 jPðXÞ; D ¼ 1Þ ¼ EðY1 À Y0 jPðXÞ; D ¼ 0Þ. IV. IMPACTS ON INCOMES AND EMPLOYMENT Two sets of probits were used for calibrating the propensity scores on the pooled sample of participants and current applicants, one for the October 2002 cross-section (used for the matched single-difference calculations) and one for the panel. Initial occupational status in 2001 (and type of occupation) is included in the estimation of the propensity score in the panel sample. Otherwise, the explanatory variables used are similar.14 The first thing to note is that the probits have low explanatory power for participation (table 7). The samples of participants and applicants are clearly quite similar ex ante in terms of observable characteristics. In a check of the sensitivity of the results for the panel sample to the inclusion of baseline house- hold income, the variable was not significant, and its inclusion did not affect the subsequent estimates of the net gains from the program. Given the evident similarity of the Jefes participants and current applicants, it is not surprising to find in the propensity scores for treatment and comparison units for panel and cross-sectional samples a large region of common support in both the single-difference and the double-difference matching (figure 3). Nonetheless, there are some significant covariates of participation. Jefes participation increases with age and is more likely for women, for households with a larger number of children below the age of 18, and for people who were public employees at the baseline (see table 7). Geographic effects are jointly significant. 13. Note that the set of weights in the single-difference and double-difference matching are not neces- sarily the same. In the panel sample, X also includes labor market status and occupation at baseline (October 2001). 14. The balancing of covariates in a regression framework as suggested by Smith and Todd (forthcoming) were tested by regressing each variable in the propensity score on a fourth-order polynomial of the propensity score and its interaction with D. We could not reject the null hypothesis that the covariates are balanced. Galasso and Ravallion 385 T A B L E 7 . Probits for Calibrating Propensity Scores for Jefes Participants and Applicants Cross-Section Oct 2002 Panel Oct 2001–Oct 2002 Coefficient t-Statistic Coefficient t-Statistic Ages 18–24 0.002 0.02 Ages 18–24 0.068 0.53 Ages 25–29 0.191 2.25 Ages 25–29 0.329 2.62 Ages 30–39 0.159 2.12 Ages 30–39 0.094 0.85 Ages 40–49 0.334 4.71 Ages 40–49 0.275 2.52 Male À0.371 À6.89 Male À0.544 À5.55 Head 0.012 0.17 Head À0.022 À0.19 Spouse of head À0.317 À3.8 Spouse of head À0.323 À2.45 Single À0.003 À0.04 Single À0.041 À0.32 Married 0.144 1.89 Married 0.249 1.94 Incomplete primary À0.045 À0.51 Incomplete primary 0.003 0.02 Complete primary 0.013 0.16 Complete primary À0.092 À0.67 Incomplete secondary 0.021 0.27 Incomplete secondary À0.089 À0.66 Complete secondary 0.002 0.02 Complete secondary 0.041 0.28 House, villa 0.130 1.21 House, villa À0.089 À0.51 House, apartment À0.109 À1.6 House, apartment À0.028 À0.24 1 room house À0.196 À2.07 Number of rooms À0.023 À0.82 2 rooms À0.072 À0.86 Bathroom 0.022 0.2 3 rooms À0.130 À1.62 Renting house À0.222 À1.8 4 rooms À0.117 À1.39 Free renter À0.404 À3.05 Bathroom À0.034 À0.33 Walls, masonry 0.002 0.02 Renting house À0.094 À1.34 Share of members 1.408 3.27 ages 0–5 Free renter À0.117 À1.63 Share of members 1.421 3.6 ages 6–17 Walls, masonry À0.011 À0.15 Share of members 0.468 1.29 ages 18–64 Water, drain 0.082 0.89 Household size 0.010 0.64 Water, well 0.151 1.55 Unemployed 0.103 0.9 Water, tube 0.073 0.78 Inactive À0.115 À1.05 Share of members 1.224 4.91 Public employee 0.533 2.49 ages 0–5 Share of members 0.956 4.25 Teacher 0.333 1.25 ages 6–17 Share of members 0.185 0.92 Social service 0.251 1.18 ages 18–64 Household size 0.006 0.59 Manufacturing 0.087 0.53 Northwest region À0.373 À4 Construction worker 0.218 1.49 Northeast region À0.173 À1.79 Domestic worker À0.145 À1.1 Cuyo region À0.654 À6.19 Northwest region À0.344 À2.4 Pampeana region À0.027 À0.3 Northeast region À0.185 À1.24 Patagonica region À0.094 À0.88 Cuyo region À0.615 À3.76 Pampeana region 0.134 0.91 Patagonica region À0.212 À1.18 Number of observations 4,803 Number of 1,899 observations Pseudo R2 0.060 Pseudo R2 0.0817 Note: Dependent variable = 1 if individual participated in Jefes in October 2002 and 0 otherwise. Source: Authors’ calculations based on data from the October 2001 and 2002 EPH. 386 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 F I G U R E 3 . Overlapping Support in the Distribution of the Propensity Score for Jefes Participants and Applicants Note: Histogram of propensity score distribution for Jefes participants (treated) and Jefes applicants (untreated); 28 (2 percent) of the participants are off the common support. Note: Histogram of propensity score distribution for Jefes participants (treated) and Jefes applicants (untreated); 6 (0.2%) of the participants are off the common support. Source: Authors’ calculations based on data from the October 2002 EPH. Galasso and Ravallion 387 Single difference and double difference can now be calculated as given by equations 1 and 2, using these probits to estimate the propensity scores for matching. Table 8 gives the estimates for the program’s impacts on incomes and employment, including both household and individual income gains for the Jefes participants.15 The mean impact estimates suggest that participants would have had a larger drop in real income in the absence of the program. The comparison group experi- enced a mean drop in real income of about 250 pesos per month over the year, whereas Jefes participants experienced a 150 peso decline. This suggests that Jefes acted as a partial safety net and attenuated the drop in income that would other- wise have been experienced. Net gains are on average between a half and two- thirds of the gross wage, depending on whether single-difference or double-differ- ence matching is used. The single-difference method gives lower net gains from the program. However, there is considerably greater imprecision in the double-difference estimates and in the household level single-difference estimates compared with the individual-based estimates. Indeed, for the double-difference estimate of the impact on household income, the 95 percent confidence interval includes 150, implying that the null hypothesis of zero forgone income cannot be rejected in this case. A further indication of the high variance in the double-difference estimates is found in the household and individual-level impact estimates underlying the means in table 8. Although naturally there is great imprecision in the individual estimates of impact, studying the distribution of the estimates gives a useful indication of which estimation method is most plausible. Because participation is voluntary, it is plausible that the bulk of the income gains will be found in the interval (0, 150). It cannot be ruled out that some people might have given up a job earning more than 150 pesos a month to join Jefes and therefore have negative net gains (presumably because of differences in the disutility of work), but it seems unlikely. It seems equally unlikely that the net income gain would exceed the gross transfer payment under the program. By this criterion, all but the individual single-difference estimates are implau- sible. For the double-difference estimates, 20 percent of the individual income gains are negative, and 60 percent exceed 150 pesos. For 30 percent of the sample, the double-difference estimates of household income gains are negative, whereas 54 percent exceed 150 pesos. For the single-difference estimates, half of the household income gains are negative and 30 percent are greater than 150. However, 83 percent of the individual single-difference estimates are in the interval (0, 150); only 5 percent of individual income gain estimates are negative, and 12 percent are greater than 150. The following discussion thus takes the individual single-difference results as the preferred estimates, although the 15. Real income is adjusted for regional differences in the cost of living. In the panel sample, real income figures are at base October 2002. (The annual inflation rate was 39.4 percent.) T A B L E 8 . Average Impact of Jefes Program on Incomes and Employment, October 2002 Household Individual Individual Individual Individual Total Hours Income Income Employment Unemployment Inactivity Worked/Week Cross-section (October 2002) EðY1 jD ¼ 1Þ 438.3 172.9 0.86 0.04 0.10 20.6 EðY1 jD ¼ 0Þ 357.1 83.7 0.37 0.30 0.33 11.4 Matched single difference SD ¼ EðY1 jD ¼ 1Þ À EðY1 jD ¼ 0Þ 81.19 (16.0) 89.2 (5.27) 0.49 (0.02) À0.26 (0.02) À0.23 (0.02) 9.2 (0.8) 95% confidence interval [63.8, 127.6] [81.2, 101.9] [0.45, 0.52] [À0.29, À0.22] [À0.27, À0.18] [8.0, 11.4] 388 Panel (October 2001–October 2002) EðY1 À Y0 jD ¼ 1Þ À147.2 30.2 0.42 À0.15 À0.27 6.4 EðY1 À Y0 jD ¼ 0Þ À250.6 À83.6 À0.03 0.08 À0.04 À2.34 Matched double difference DD ¼ EðY1 À Y0 jD ¼ 1Þ ¼ EðY1 À Y0 jD ¼ 0Þ 103.41 (32.27) 113.55 (15.08) 0.46 (0.04) À0.23 (0.04) À0.23 (0.04) 8.9 (1.5) 95% confidence interval [67.8, 195.9] [78.5, 138.4] [0.32, 0.49] [À0.27, À0.09] [À0.30, À0.15] [5.8, 12.1] Note: Numbers in parentheses are standard errors, bootstrapped with 100 repetitions. In the panel sample, real income figures are base 2002 (annual inflation rate of 39.4 percent). Source: Authors’ calculations based on data from the October 2001 and 2002 EPH. Galasso and Ravallion 389 double-difference results are also reported when they appear to contain insights that cannot be revealed by estimates based solely on the cross-sectional data. With the constrained individual single-difference estimates as the most plau- sible, the mean forgone income is about 50 pesos a month, or a third of the Jefes payment. Although lower than the estimated forgone income of about half the program wage for the Trabajar program in Jalan and Ravallion (2003), this result for Jefes is unsurprising given the general decline in real wages due to the crisis (World Bank 2003); the opportunity cost of participation in workfare would undoubtedly have been lower in the wake of this crisis. Although the null hypothesis of zero forgone income for the double-difference estimate of the impact on household income cannot be rejected, this result is attributable to the considerably more noise in this estimator. The extent of forgone income and displaced hours suggests that the work requirement was having an impact. How- ever, forgone income could also stem from side payments made to intermediaries (punteros) to participate. There have been anecdotal reports that such payments are typically around 50 pesos a month.16 It seems unlikely that a majority of Jefes participants were obtaining access to the program through punteros.17 So if the typical side payment is 50 pesos, then the estimate of 50 pesos a month in forgone income cannot be explained this way; the more plausible explanation is that there was an opportunity cost to the work requirement. On average, about half of the participants gained work as a result of the program: Half of these workers were drawn from the ranks of the unemployed (women and men) and half from economic inactivity (mostly women). More- over, on average Jefes participants increased their hours of work by about nine hours a week. In this respect, single-difference and double-difference results are similar. Overall, the results are suggestive of forgone income in that the net increase in hours worked is about half the Jefes stipulated work requirement of 20 hours a week.18 It is clear from these results that Jefes did not just displace unemployment. Indeed, roughly as many participants came from those who would otherwise not have been active in the workforce. This implies that assuming that all Jefes participants would have otherwise been unemployed would grossly overesti- mate the impact of the program on the rate of unemployment. 16. Such payments to intermediaries need not be interpreted as extortion. As the ethnographic literature suggests, there can be complex reciprocal links between such intermediary brokers and the poor. Auyero (2000) illustrates how the poor and marginalized members of society are drawn into problem-solving networks because of their limited access to formal sources of assistance. 17. In a random sample of Jefes beneficiaries, about 10 percent reported having registered for the program through punteros and 7 percent in similar groups (neighborhood associations or unions of unemployed people, piqueteros). 18. Some 78 percent of Jefes participants doing the work requirement reported exactly the legally required number of hours (20). It may be that municipalities, to generate work for a large number of participants, employed them for the minimum number of hours. It is also possible that some participants overreported their number of hours worked to accord with the legal requirements. 390 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 Estimates of the program’s impact on labor market status can now be used to estimate the impact on the unemployment rate. The preferred single-difference esti- mates imply that 26 percent of Jefes participants would have been unemployed if not for the program, and 23 percent would have been economically inactive (see table 8). The counterfactual unemployment rates (and activity and employment rates) were also estimated and compared with INDEC’s (2002c) estimates, which assumed that all Jefes participants would have been unemployed without the program (table 9). Allowing for the behavioral responses implied by these results gives an appreciably lower impact on the unemployment rate. Although INDEC’s calcula- tion implies a 5.8 percentage point drop in the unemployment rate due to the program, the results here show an impact of 2.5 points. In contrast to the claims by INDEC and others (including World Bank 2003), Jefes was only partially responsible for bringing down the unemployment rate in the aftermath of the crisis. The results here indicate that the unemployment rate would have fallen between May 2002 and October 2002 even without the program. Further exploration of the double-difference estimates of impacts shows that those attracted out of labor market inactivity were primarily women (table 10). There is no evidence of labor supply responses by other members of the house- hold, other than the change of labor status of the beneficiary (the net gains in T A B L E 9 . Impact of the Jefes Program on the Aggregate Unemployment Rate (%) October 2002 without Actual Jefes Program INDEC (2002c) Calculations Assuming that Calculations Participants Would Based on October May October Be Otherwise Estimated 2001 2002 2002 Unemployed Net Gainsa Activity rate 42.2 41.8 42.8 42.9 42.0 (share of total) Employment rate 34.5 32.8 35.2 32.7 33.5 (share of total) Unemployment rate 18.3 21.5 17.8 23.6 20.3 (share of economically active individuals) Note: For comparability with previous EPHs, these calculations apply to 28 urban conglomerates (excluding Viedma, Rawson, and San-Nicolas, which were added in October 2002, and new areas added in Greater Buenos Aires). INDEC’s definition of activity, employment, and unemployment rates are used. a Estimated net gains on employment, unemployment, and inactivity from table 8, single-difference estimates. Let actual number of employed individuals be Et, the number of unemployed be Ut, and J be the total number of Jefes participants in October 2002. Then the actual unemployment rate is Ut =ðUt þ Et Þ, and the counterfactual unemployment rate is ðUt þ 0:26JÞ=ðUt þ Et À 0:23JÞ. Source: Authors’ calculations based on data from the October 2001 and 2002 EPH. T A B L E 1 0 . Impact of Jefes Program on Labor Supply and Household Size, October 2002 No. Women in Household No. Men in Household Panel (October 2001– Household Number of October 2002) Employed Unemployed Inactive Employed Unemployed Inactive Size Children EðY1tþ1 À Y0t jD ¼ 1Þ 0.39 À0.08 À0.27 0.04 À0.04 0.04 0.05 0.03 EðY0tþ1 À Y0t jD ¼ 0Þ 0.04 0.07 À0.06 À0.03 0.06 0.01 0.02 À0.01 391 Matched double difference DDt;tþ1 ¼ EðY1tþ1 À Y0t j 0.35 (0.05) À0.15 (0.04) À0.21 (0.05) 0.07 (0.04) À0.10 (0.04) 0.03 (0.03) 0.03 (0.08) 0.04 (0.07) D ¼ 1Þ À EðY0tþ1 À Y0t jD ¼ 0Þ 95% confidence interval [0.17, 0.39] [ À0.20, À0.03] [À0.27, À0.06] [À0.19, À0.02] [À0.19, À0.02] [À0.03, 0.10] [À0.14, 0.18] [À0.11, 0.18] Note: Numbers in parentheses are standard errors, bootstrapped with 100 repetitions. Source: Authors’ calculations based on data from the October 2001 and 2002 EPH. 392 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 T A B L E 1 1 . Stratification of Net Gains from Jefes Program, Real Income Impacts, October 2002 Cross-Section Single Difference Panel Double Difference Household Individual Constrained Household Individual Income Income Individual Income Income Gain Gain Income Gain Gain Gain Whole sample 81.18 (6.1) 89.18 (1.5) 104.75 (0.64) 103.41 (12.5) 113.55 (6.3) Participant is Male 32.5 (11.0) 114.6 (3.4) 115.3 (1.0) 67.5 (22.9) 8.2 (15.7) Female 101.9 (7.3) 78.2 (1.5) 99.4 (0.8) 119.1 (14.9) 159.7 (5.5) Head À69.0 (5.6) 100.8 (3.0) 105.9 (1.0) 121.3 (16.6) 24.4 (13.0) Spouse of head 53.5 (8.1) 81.6 (1.8) 103.1 (1.1) 135.5 (20.2) 176.7 (7.3) Occupation at baseline Employed 11.9 (19.5) À1.12 (11.2) Unemployed or inactive 175.8 (15.7) 204.4 (4.5) Note: Numbers in parentheses are sample standard errors, not bootstrapped. Source: Authors’ calculations based on data from the October 2001 and 2002 EPH. numbers of employed, unemployed, and economically inactive people mirror the labor supply changes at the beneficiary level). Nor are there signs that households responded by changing the household composition (for example, by sharing children) to gain access to the program. Household size rose slightly more among participants than among applicants, but the difference is not statistically significant. This is not surprising, because the results suggest that the program’s eligibility criteria were not rigorously enforced. Moreover, households might not have been split up in practice, with parents claiming separate households only to gain access to the program. Stratification of the net gains shows considerable heterogeneity in impact (table 11). Those who were unemployed or economically inactive before the program had no forgone income, so their income gain from the program is the gross wage. Those who were previously employed had high forgone incomes. Spouses of the head and women averaged larger net gains—not surprising because they were more likely to be drawn to the program from unemployment or inactivity than were men, for whom the opportunity cost of Jefes participa- tion was clearly higher. Single-difference impact estimates for individual incomes constrained to be within the interval (0, 150) indicate that forgone income accounts for about one-third of the Jefes payment (see table 11).19 Using the preferred single-difference estimates, figure 4 gives the implied impacts on poverty among participants, as indicated by the cumulative density function (CDF) of income per person. The lower curve gives the observed (post- 19. In the constrained estimates, the individual net income from Jefes represents the main source of individual income for about half the sampled participants and the main source of household income for about a quarter of sampled participants. Galasso and Ravallion 393 F I G U R E 4 . Impacts on Poverty among Jefes Participants: Cumulative Distributions of Income Pre- and Postontervention Source: Authors’ calculations based on data from the October 2002 EPH. intervention) CDF, and the upper curve gives the CDF implied by the estimates of the impact of Jefes at each sample point.20 At a poverty line of around 100 pesos a month, the poverty rate among Jefes participants fell from about 82 percent to 70 percent due to the program. At a poverty line of around 50 pesos a month, the poverty rate among participants fell from about 51 percent to 29 percent. The implied impacts of Jefes on the national poverty rate are shown in table 12 for the government’s two official poverty lines. The impacts at the upper and lower poverty lines are negligible for both the official INDEC estimates of the counter- factual poverty incidence, which assume no forgone income, and the estimates using the preferred single-difference estimates of the net income gains. The decline is larger (2 percentage points) for the lower poverty line (indigence). A further perspective on the ability of Jefes to reduce poverty can be obtained using the panel data to compare the actual joint distribution of income between poor and nonpoor households over time with the estimated counterfactual distribution for Jefes participants (following the methodology in Ravallion 20. This is the same counterfactual distribution used to calculate the counterfactual income shocks for the panel sample, as in the last column of table 4. 394 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 T A B L E 1 2 . Impact of the Jefes Program on Aggregate Poverty Rates, October 2002 (% below the poverty line) Counterfactual, in Absence of the Program INDEC Calculations (Subtracting Calculations Based Actual, after Jefes Income from Total on Estimated Net the Program Household Income) Gains on Incomea Greater Buenos Aires Poverty Individuals 54.3 54.7 54.5 Households 42.3 42.6 42.5 Indigenceb Individuals 24.7 27.0 26.2 Households 16.9 18.7 18.0 Total 31 conglomerates Poverty Individuals 57.5 58.1 57.9 Households 45.7 46.2 46.1 Indigenceb Individuals 27.5 30.5 29.6 Households 19.5 21.9 21.1 Note: Income per adult equivalent is constructed using the adult equivalent scales provided by INDEC. For the analysis, new interview areas were excluded as well as households with partial income responses. a Estimated net gains on income from table 7, single-difference individual estimates. b Indigence is the food component of the poverty line. Source: INDEC (2002b, 2003). and others 1995). This distinguishes the extent to which the program prevents people from falling into poverty (‘‘protection’’) from the extent to which it helps people escape poverty (‘‘promotion’’). The actual joint distribution shows that 20 percent of participants were not poor in 2001 but became poor in 2002, whereas only 2 percent of participants who were poor in 2001 escaped poverty by the following year (table 13). Some 71 percent were poor in both periods. Under the counterfactual distribution based on the preferred single-difference individual estimates of counterfactual incomes in October 2002, without Jefes 22 percent of participants who were not poor in 2001 would have become poor in 2002 and only 1 percent would have escaped poverty. The impacts are greater at the lower poverty line. With Jefes, 30 percent were not indigent in 2001 but became so in 2002. In the counterfactual joint dis- tribution without Jefes, 40 percent would have become indigent in 2002. Again, fewer people would have escaped poverty without the program (5 percent) than with it (8 percent). The results confirm the social protection nature of the program. Jefes had a small impact in helping the participants escape poverty, but the results show an extra 10 percent of participants would have fallen into extreme poverty in the absence of the program. Galasso and Ravallion 395 T A B L E 1 3 . Measures of Protection and Promotion for Jefes Participants Nonpoor 2002 Poor 2002 Official poverty line Actual joint distribution Nonpoor 2001 0.07 0.20 0.27 Poor 2001 0.02 0.71 0.73 0.09 0.91 1 Counterfactual joint distributiona Nonpoor 2001 0.04 0.22 0.27 Poor 2001 0.01 0.72 0.73 0.06 0.94 1 Official indigence line (food poverty line) Actual joint distribution Nonpoor 2001 0.39 0.30 0.68 Poor 2001 0.08 0.24 0.32 0.47 0.53 1 Counterfactual joint distributiona Nonpoor 2001 0.28 0.40 0.68 Poor 2001 0.05 0.27 0.32 0.33 0.67 1 Note: Income per adult equivalent is constructed using the adult equivalent scales. a Estimated net gains for 2002 from table 8, single-difference individual estimates. Source: Authors’ calculations based on data from the October 2001 and 2002 EPH and equivalence scales, poverty, and indigence lines from INDEC (2002a,b, 2003). What role did the work requirement play? Two observations suggest that the work requirement had an impact. In the October 2002 cross-section, 80 percent of sampled participants reported having done work for Jefes.21 Evidence of forgone income is also consistent with a work requirement. V. CONCLUSION The Jefes program provided a basic cash transfer to all households satisfying certain eligibility criteria, and for about 80 percent of participants the transfer payment came with a work requirement. It is clear from the results, however, that the eligibility criteria were not rigorously enforced. About one-third of participants did not meet the eligibility criteria, and about three-fourths of 21. The data do not allow for a consistent definition of the type of activity undertaken by partici- pants. The definition of what represents ‘‘work’’ depends on whether Jefes participation represents the main economic activity of the participants or not. Participants for which Jefes work is the main activity are defined to be doing a controprestaction (complying with the work requirement) if they are working positive hours (among those, about half report working for the public sector, 30 percent for community service, and 8 percent for a private company). Participants for which Jefes is a secondary activity, self- report work as community service (32 percent), participation in training (41 percent), or school atten- dance (13 percent). 396 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 eligible adults were not receiving the program aid. The goal of targeting only unemployed heads of households with dependents was clearly not realized. The results suggest that a large share of participants were women who would not otherwise have been in the labor force. About half of the employment gain from the program came from the ranks of the unemployed and about half from the economically inactive population. The program reduced Argentina’s unemployment rate by an estimated 2.5 percentage points, which is less than half the size of previous estimates that assumed that all Jefes participants would have otherwise been unemployed. Jefes tended to have a positive opportunity cost for participants, consistent with the work requirement being binding for many participants. When forgone incomes are factored in, the program had a small effect on the overall poverty rate and a slightly larger impact on the incidence of extreme poverty. The program allowed an extra 2 percent of the population to afford the food component of Argentina’s poverty line. A degree of protection from extreme poverty was also achieved: An estimated 10 percent more of the participants would have fallen below the food poverty line without the program. It is not clear how much concern there should be about the extent of participation by people who were formally ineligible. There is evidence that unemployment was not the main factor bringing down living standards during the crisis—rather, it was the sharp fall in real wages (McKenzie 2004). In addition, the difficulty of effectively verifying the unemployment status of beneficiaries makes the eligibility requirement based on unemployment status unenforceable. More effective for propoor targeting were eligibility criteria correlated with structural poverty, such as having dependents or living in households with no members in the formal labor market. Thus targeting performance was relatively good despite weak enforcement of the eligibility criteria. About half of Jefes participants came from the poorest fifth of Argentine families, and all but 10 percent fell below the official poverty line. This is better targeting performance than most social spending in Argentina, though it is not as good as in the Trabajar program. Jefes participants would have suffered an appreciably larger drop in their incomes without the program. Overall, the Jefes program does appear to have contributed to social protection during the crisis, despite the fact that its implementation differed from its design. APPENDIX: COMPARISON WITH ADMINISTRATIVE RECORDS Comparing the survey aggregates on Jefes participation for the EPH with the administrative records is complicated by the fact that the (urban) sample frame for the survey does not coincide with the (national) coverage of Jefes. This can be dealt with by confining the analysis of the administrative data to areas included in the EPH sample frame. Two ways of doing this were considered. The first used the administrative data only for municipalities included in the EPH sample frame, based Galasso and Ravallion 397 on the location of participants’ Jefes registration. In other words, only participants registered in the geographical areas where there is an overlap of municipalities with the sample frame of the EPH were considered. The second method restricted the administrative data to those who have their recorded domicile in the EPH sample frame. In other words, only participants whose residence is in the same conglom- erate (according to the postal code) as where they receive their payment (boca de pago) were considered. This second method deals with cases in which people register in a nearby city in which they are not in fact resident. The grossed-up estimate of Jefes participation represents about 80 percent of the registered applicants from the administrative data (table A.1). The aggregate from the administrative data is outside the 95 percent confidence interval of the survey-based estimate. When the aggregates are broken down by urban areas, the administrative count is outside the 95 percent confidence interval for 18 areas. These are all cases in which the survey estimate of participation is lower than the administrative data suggest. T A B L E A . 1 . Comparison of Survey-Based Participation Rates and Administrative Data Grossed-up Survey Estimates Administrative of No. Participants Data Registered in Domiciled in 95% Confidence Municipalities Municipalities Conglomerates Point Estimate Interval Covered by EPH Covered by EPH Tucuma ´n 30,454 [23,451 37,457] 38,829* 29,387 Tierra del Fuego 2,341 [1,494 3,188] 2,694 2,277 Santiago del Estero 11,813 [8,828 14,798] 23,404* 22,066* Santa Cruz 1,378 [839 1,917] 1,584 1,362 San Luis 3,701 [2,496 4,906] 6,607* 6,361* San Juan 12,053 [8,988 15,118] 21,131* 16,185* Salta 23,592 [19,243 27,941] 31,948* 28,412* Rio Negro 3,049 [2,028 4,070] 2,840 3,706 Neuquen 8,411 [6,469 10,353] 8,831 8,132 Misiones 11,164 [8,302 14,026] 11,997 10,337 Mendoza 20,001 [14,460 25,542] 31,686* 21,828 La Rioja 7,014 [5,671 8,357] 9,768* 8,751* La Pampa 2,710 [1,832 3,588] 2,956 2,890 Jujuy 15,542 [11,996 19,088] 26,834* 25,718* Formosa 16,865 [14,037 19,693] 21,513* 20,431* Gran Parana 6,667 [4,872 8,462] 8,185 7,729 Concordia 5,155 [3,844 6,466] 7,861* 7,643* Corrientes 16,325 [12,840 19,810] 27,111* 15,936 Rio Cuarto 4,455 [3,093 5,817] 6,503* 5,796 Gran Cordoba 53,380 [40,058 66,702] 48,067 46,317 C.Rivadavia 1,851 [1,049 2,653] 2,988* 2,735* Rawson 4,310 [3,169 5,451] 4,467 4,018 Chaco 21,709 [16,060 27,358] 36,729* 34,082* (Continued) 398 THE WORLD BANK ECONOMIC REVIEW, VOL. 18, NO. 3 T A B L E A . 1 . Continued Grossed-up Survey Estimates Administrative of No. Participants Data Registered in Domiciled in Point 95% Confidence Municipalities Municipalities Conglomerates Estimate Interval Covered by EPH Covered by EPH Catamarca 10,955 [9,046 12,864] 14,879* 15,499* Gran Rosario 56,920 [43,906 69,934] 79,361* 75,631* Gran Santa Fe 23,628 [19,255 28,001] 29,513* 28,577* Villa Consitucion 8,224 [5,276 11,172] 6,353 7,124 Capital Federal 27,008 [10,677 43,339] 55,437* 49,421* Conurbano 379,009 [311,738 446,280] 418,018 369,349 La Plata 28,593 [21,184 36,002] 25,960 23,885 Bahı´a Blanca 6,375 [3,502 9,248] 5,244 5,367 Mar del Plata 16,754 [10,788 22,720] 16,789 15,706 Total urban areas 841,406 [767,394 915,418] 1,036,087* 922,658* *Significantly different from that implied by the survey data. Source: Calculations (kindly provided by the Ministry of Labor) are based on overlapping the database of liquidacion de beneficiarios and base de personas using data from the EPH and administrative data from the Jefes Program. Standard errors corrected for complex survey design were provided by INDEC. As would be expected, switching to the residence-based assignment of Jefes participants to urban areas reduces the discrepancy. The tighter matching by residence puts the administrative data close to the upper bound of the 95 percent confidence interval in the aggregate, but it still leaves 14 areas for which the survey gives a significantly lower count. The results suggest that the survey underrepresents Jefes participation. The source of the discrepancy is unclear. It could be respondent ignorance of Jefes participation or a desire to hide participation because of formal ineligibility. Alternatively, it might reflect overcounting in the administrative data. This could arise if there is some expropriation of the Jefes transfers for other purposes. However, once one allows for the residence-based assignment of participants, the discrepancy does not appear large enough to warrant serious concern about sampling bias in the EPH. REFERENCES Angrist, Joshua D. 1998. ‘‘Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants.’’ Econometrica 66(2):249–88. Atkinson, A. 1987. ‘‘On the Measurement of Poverty.’’ Econometrica 55(4):749–64. Auyero, Javier. 2000. ‘‘The Logic of Clientelism in Argentina: An Ethnographic Account.’’ Latin Amer- ican Research Review 35(3):55–82. Coady, David, Margaret Grosh, and John Hoddinott. 2002. ‘‘The Targeting of Transfers in Developing Countries: Review of Experience and Lessons.’’ Safety Net Primer. World Bank, Washington, D.C. Duflo, Esther. 2000. Grandmothers and Granddaughters: Old Age Pension and Intra-Household Allocation in South Africa. NBER Working Paper 8061. Cambridge, Mass.: National Bureau of Economic Research. Galasso and Ravallion 399 Fiszbein, Ariel, Paula Giovagnoli, and Isidoro Aduriz. 2002. ‘‘Argentina’s Crisis and Its Impact on Household Welfare.’’ Documento de trabajo 1/02. World Bank Office for Argentina, Paraguay, and Uruguay. Available online at www.bancomundial.org.ar/arg.htm. ERES. ´n ra 2004. ‘‘Evaluacio ´pida de la emergencia social en Argentina.’’ Documento de trabajo 1/04. World Bank Office for Argentina, Paraguay, and Uruguay. Available online at www.bancomundial. org.ar/arg.htm. Foster, Andrew, and Mark Rosenzweig. 2002. ‘‘Household Division and Rural Economic Growth.’’ Review of Economic Studies 69(4):839–69. ´blico.’’ Fundacion de Investigaciones Gasparini, Leonardo. 1999. ‘‘Incidencia Distributiva del Gasto Pu Economicas Latinoamericanas, Buenos Aires. Heckman, James., H. Ichimura, and Petra Todd. 1997. ‘‘Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Program.’’ Review of Economic Studies 64(4):605–54. INDEC ´stica y Censos). 2002a. ‘‘Incidencia de la pobreza y de la indigencia en (Instituto Nacional de Estadı los aglomerados urbanos: Octubre de 2001.’’ Informacio ´n de prensa. Available online at www. indec.gov.ar. ———. 2002b. ‘‘Incidencia de la pobreza y de la indigencia en el Gran Buenos Aires.’’ Anexo 2. ´n de prensa. Available online at www.indec.gov.ar. Incidencia del Plan Jefes/Jefas, GBA, Informacio ———. 2002c. ‘‘Mercado de trabajo: principales indicadores de los aglomerados urbanos. Octubre ´n: 2002.’’ Anexo 2. Incidencia del Plan Jefes y Jefas sobre las tasas de actividad, empleo y desocupacio ´tesis B, Informacio hipo ´n de prensa. Available online at www.indec.gov.ar. ———. 2003. ‘‘Incidencia de la pobreza y de la indigencia en los -aglomerados urbanos: Octubre de ´n de prensa. Available online at www. 2002.’’ Anexo 2. Incidencia del Plan Jefes/Jefas, Informacio indec.gov.ar. Jalan, Jyotsna, and Martin Ravallion. 2003. ‘‘Estimating the Benefit Incidence of an Anti-Poverty Program.’’ Journal of Business and Economic Statistics 21(1):19–30. McKenzie, David. 2004. ‘‘Aggregate Shocks and Labor Market Responses: Evidence from Argentina’s Financial Crisis.’’ Economic Development and Cultural Change 52(4):719–58. Ravallion, Martin. 2000. ‘‘Monitoring Targeting Performance When Decentralized Allocations to the Poor Are Unobserved.’’ World Bank Economic Review 14(2):331–46. Ravallion, Martin, Dominique van de Walle, and Madhur Gautam. 1995. ‘‘Testing a Social Safety Net.’’ Journal of Public Economics 57(2):175–99. Rosenbaum, P. R., and Donald B. Rubin. 1983. ‘‘The Central Role of the Propensity Score in Observa- tional Studies for Causal Effects.’’ Biometrika 70(1):41–55. Smith, Jeffrey, and Petra Todd. Forthcoming. ‘‘Is Matching the Answer to LaLonde’s Critique of Nonexperimental Methods?’’ Journal of Econometrics. van de Walle, Dominique. 1998. ‘‘Targeting Revisited.’’ World Bank Research Observer 13(2):231–48. World Bank. 1999. Poor People in a Rich Country: A Poverty Report for Argentina. Washington, D.C. ———. 2003. Argentina—Crisis and Poverty 2003: A Poverty Assessment. Report 26127-AR. Washington, D.C.