77234 SO CIA L D EV E L O P M E NT NOT E S COMMUNITY DRIVEN DEVELOPMENT No. 136/ February 2013 What Have Been the Impacts of World Bank CDD Programs? Operational and Research Implications Community-driven development (CDD) is an approach that emphasizes community control over planning decisions and investment resources. A rigorous evaluation process helps determine CDD’s effectiveness in various settings and highlights areas that need strengthening for second- phase programs or new projects. This note summarizes the findings of a recently conducted study, “What have been the Impacts of World Bank Community-Driven Development Programs? CDD Impact Evaluation Review and Operational & Research Implications� (Wong 2012), which synthesizes the impact evaluation results of 17 World Bank CDD programs over the past 25 years. The study finds that, on the whole, these projects achieved their stated goals of poverty welfare reduction, poverty targeting, and increased access to services. Evidence on governance, social capital, spillovers, and conflict impacts, however, is found to be limited and mixed. Introduction many national governments and international aid agencies for the delivery of services, improvement of livelihoods, and Community Driven Development (CDD) is an approach that empowerment of people. The World Bank currently supports gives communities control over the planning, investment, approximately 400 CDD projects in more than 90 countries, and management decisions for local development activities. valued at almost US$30 billion.1 Over the past decade, CDD The philosophy behind CDD is that involving communities investments have represented between 5 and 10 percent of in these decisions often leads to the better use of resources the overall annual Bank lending portfolio. Although the geared toward meeting the most pressing needs. This bottom- design of these programs has evolved over time, at their core, up approach has become a key operational strategy for most of these programs aim to improve the living conditions of poor communities through increased participation. There is a growing body of evidence documenting the impacts of CDD interventions. Several studies have tried to synthesize the findings of different aspects of the research. This “meta-analysis� summarized in this note2 focuses spe- cifically on available evaluations of World Bank-supported CDD programs to examine cumulative evidence and trends on a comparative basis. By doing so, it draws lessons about operations and results.3 The study analyzes a total of 17 World Bank-supported CDD programs with robust impact evaluations from South and East Asia, Africa, Latin America, and Central Asia, and draws implications for Bank operations (see annex 1 for a list of the programs and impact evaluations reviewed). Key Results and Areas of Interest CDD projects are, by their very nature, multisectoral, catering to several objectives. Many CDD programs reviewed for this study were initiated in response to economic and financial crises, disasters, or conflicts influencing their focus and eventual outcomes. The impact evaluations were also tailored to these focus areas. In order to provide some standardiza- tion and common framework for assessment, the meta-analysis focused on six key questions: 1. What is the welfare impact of CDD programs? 2. Who benefits from these program interventions—poorest quintiles, women, ethnic groups? Do the interventions reach impoverished areas and poor households? 3. Do the programs improve access to and use of basic services? 4. Do the programs improve social capital using the standard proxy measurements for social capital: trust, collective action, association, groups, and networks)? 5. Do the CDD programs improve local governance (participation in local meetings, satisfaction, and increased confi- dence with government officials, awareness of program activities, and so on)? 6. Do CDD operations have any impact on violent conflict in conflict-affected areas? In order to derive operational and research implications from this study, the meta-analysis also examines some of the contextual factors behind these results as well as whether and why there might be impacts along these six dimensions (box 1). Methodology Most of the programs—especially Social Funds—focus more on improving the coverage and quality of health, education, The meta-analysis was essentially a desk review that focused and other social services, such as road and water access, on: (1) World Bank-supported CDD projects and (2) their than on direct welfare or income gains. In addition, poverty “robust� impact evaluations. A robust impact evaluation has reduction and welfare improvement are generally considered a large enough sample size to allow for the claim of some long-term goals that are beyond the scope of some of these degree of internal validity, is well-constructed, and has a projects with three- to four-year phasing periods; therefore, control group. To enhance quality control, every impact they were never explicitly included in their objectives. evaluation included in the meta-analysis underwent a peer review process. Relevant information from accompanying Nevertheless, nine projects that were reviewed in the qualitative studies, project design documents, and comple- meta-analysis reported on household welfare and poverty tion reports were also drawn upon to inform the discussion. impacts as part of their evaluations (see annex 2), and Finally, the study team spoke to several CDD experts, task seven of them demonstrated positive impacts. For example, team leaders, and evaluators in order to more fully under- in the Nepal Poverty Alleviation Fund (PAF), the estimated stand the contextual and design issues leading to the stated net program impact for PAF participants on real per capita results. consumption growth was 19 percent. Other impacts included a 19 percentage point decline in the incidence of food insecu- rity (defined as self-reported food sufficiency for six months Results from Impact Evaluations or less). In the Senegal National Rural Infrastructure Program 1. Socioeconomic Welfare (PNIR), a per capita household expenditure was 65 percent higher in the treatment areas with the completed project Household economic welfare, measured in terms of compared to the control areas. This figure is extremely high, income, assets, consumption, and expenditures, can be partly because the households were particularly poor at the considered the ultimate goal of all CDD projects. However, baseline, with per capita consumption equivalent of US$0.23 only four out of the 17 projects reviewed in this study stated per household member per day. an explicit goal of improving socioeconomic conditions for the poor and vulnerable in their project development objec- Two of the nine projects—the Afghanistan National tives. These four projects focused on livelihood activities to Solidarity Program II (NSP2) and Indonesia Urban Poverty a greater degree than the other projects in the meta-analysis. Project II (UPP2)—showed no poverty impacts. For NSP2, 2 the lack of welfare impacts was not surprising because the midline survey report clearly stated that the evaluation was Box 1. Factors Contributing to Positive Impacts on Poverty conducted prior to the completion of many subprojects and Reduction and Improved Access to Services focused primarily on the creation of local councils and the socialization and planning process. In the UPP2, there was Based upon project and evaluation reports and discussions with no statistically significant impact on the welfare of the popu- project stakeholders, the meta-analysis posits several hypoth- lation living in the project areas as measured by per capita eses to explain some of the success in poverty reduction and consumption, access to credit, and assets. This evaluation improved access to services. raises questions about how to best adapt the CDD model to • The establishment of a more participatory and inclusive urban settings. model of service delivery, allowing communities to identify the In sum, CDD projects are performing well with regard to poorest among them as well as their own development needs; poverty reduction and in improving the welfare of the poor, • The provision of high-quality and adequate facilitation and but these impacts take time to become manifest and are technical assistance; dependent on whether or not specific livelihood investments • Capacity-building for communities; are made. However, there is still room for unpacking the • Utilization of poverty maps to target resources to poor areas; design of these projects based on different social and eco- • The provision of block grants of sufficient size over several nomic contexts in order to ensure positive impacts on income years that are used for economically productive purposes; and in the diverse settings in which CDD programs operate. • Flexibility in project design and implementation with an ap- proach of “growth in learning� in the medium and longer term. 2. Poverty Targeting When addressing questions about targeting, it is impor- These design features are important for any future program tant to differentiate between two levels of targeting: (1) aimed at having a greater impact on poverty and basic service inter-community—the geographic choices of communi- delivery. ties the project will engage; and (2) intra-community— the distribution and targeting of specific groups within communities. It is important to remember that most CDD districts receiving more per capita than wealthier ones. In programs invest in public goods that benefit the broader fact, the very poorest districts received shares that exceeded community (the poor and nonpoor), including infrastructure their share of the population. and services like roads, bridges, schools, and health centers. Therefore, in CDD economic infrastructure, a community In terms of second-level household targeting, much of can choose to promote economic growth for the entire area, the data indicate that poorer households are more likely to allowing everyone to benefit, as opposed to a more narrowly benefit from CDD investments than better-off households. targeted assistance program (such as individual household For example, in the Nepal PAF, where participatory well- safety nets). Nevertheless, it is important to determine if being ranking was used to identify the poor and socially CDD programs—including the investments with a broader disadvantaged, a higher decline (24 percentage points) in focus—have benefited more poor than nonpoor households food insecurity for households in disadvantaged castes or and individuals, even for these types of investments. ethnic groups was found. Positive findings were also evident in the India Andhra Pradesh District Poverty Initiative Pro- Evidence from the 14 programs examining targeting gram (APDPIP) as well as the Philippines Kapit-bisig Laban is generally positive with regard to geographical poverty Sa Kahirapan-Comprehensive and Integrated Delivery of targeting and the selection of poor areas to operate (see Social Services Program (KALAHI-CIDSS). annex 2). The geographic distribution of program funds from the central level was pro-poor, especially when those However, there is some variation in household target- programs used poverty maps and the latest survey informa- ing and types of investments based on certain socioeco- tion to target poor areas in the country. For example, in a nomic characteristics—such as urban versus rural. In the cross-country analysis of social funds in Armenia, Bolivia, Armenia Social Investment Fund (ASIF), which relied on Honduras, Nicaragua, Peru, and Zambia, Rawlings and a nationally-representative household survey, the target- others (2003)4 found that the geographic distribution of social ing of its resources was slightly progressive in urban areas fund spending was progressive in all countries, with poor and slightly regressive in rural areas. The study suggests 3 that the progressive urban targeting may have been due to UPP2 or improvement in health in the Armenia Social Invest- the fund’s focus on Yerevan, whose population was acutely ment Fund (ASIF) communities—they still impacted other suffering from economic dislocation; and the regressive types of services, such as an access to adequate sanitation in rural targeting may have resulted from the difficulties faced UPP2 and access to water in ASIF. by rural communities providing the required 10 percent Impacts on longer-term outcomes were ambiguous. community contribution. There are some reported positive impacts in a few projects Projects also must be mindful of the interplay between in areas such as child nutrition and under-age-five mortal- politics and the allocation of resources at the local level. ity: in the Nicaragua Emergency Social Investment Fund, In the Tanzania Social Action Fund (TASAF), while the cen- for example, there were improvements in health outcomes tralized funding allocation to districts was pro-poor, the because of water investments, with the incidence of stunting within-district targeting was neutral at best. Beneficiary (low height-for-age) falling from 25 to 14 percent. However, households were slightly poorer than the average eligible in education, two programs—the Bolivian Social Investment household; they were also much more likely to be civically Fund and Indonesia’s PNPM-Rural Generasi Sehat dan and politically active, related to village leaders, and benefi- Cerdas (PNPM Generasi)—measured achievement in math- ciaries of other welfare programs. These results point to the ematics and language tests and found no impact; however, importance of political involvement and widespread access this could also have been a function of the evaluations’ short to program information so that communities can mobilize timeframes. and file proposals. Marginalized or poorly-educated mem- These results may also reveal the limits of an exclusive bers are likely to have less program awareness and a more CDD approach without parallel sector investments from difficult time navigating the application process. the supply side in, for example, teacher pay, curriculum Geographic poverty targeting is an effective first step in quality, or allocation of doctors. Multisectoral approaches, ensuring that the poor benefit from project investments. especially in collaboration with line ministries and private However, several reports cited difficulties engaging specific sectors, need to be further explored. groups within communities, such as seasonal migrants, In sum, impacts on access and utilization of ser- refugees, aged persons, and the destitute. Programs that are vices are generally positive, especially when subproj- scaling up also have concerns about covering the poorest of ects are implemented in particular sectors. However, the poor or marginalized groups. Participatory poverty tar- in order to measure outcomes that require long-term geting at the household level can also be effective, but more investments—such as health status or learning achieve- attention should be paid to the socioeconomic characteristics ment—additional high-quality and timely evaluations are and local politics in which projects operate in order to avoid needed. Project teams also need to be more realistic about capture and uneven resource allocation. what could be achieved within a three-to-four year project 3. Access and Utilization of Services phase. Because more CDD projects are entering into their second or third phases, there are more opportunities to Improvements in the access to and use of services, espe- analyze longer-term outcomes. cially in health, education, and drinking water, are evident across the CDD programs reviewed (see annex 2). In Senegal 4. Social Capital PNIR, access to clean water and health services increased Social capital is broadly defined by the World Bank as “the by 22.4 percent and 24.1 percent, respectively, in the treat- norms and networks that enable collective action,� and ment areas as compared to the control areas. The Philippines it is often assumed that CDD projects contribute to this. KALAHI-CIDSS evaluation also identified an increase in By working together using a CDD approach, CDD proj- access to health facilities, secondary school, and college in ects should, in theory, build trust, networks, and collective the treatment area. A six percent increase in the proportion action. Social interaction and networks as well as trust and of households with year-round access to roads. For the most reciprocity as elements of social capital can, in turn, produce part, impacts on access to or use of services directly derive collective outcomes, both beneficial and harmful (Grootaert from the types of subprojects funded through the project and van Bastelar 2002).5 Given the link between CDD and grants. social capital, impact evaluations attempt to measure social While some projects did not observe any impact on specific capital improvements during the intervention and by deter- services—for example, access to clean water in the Indonesia mining if there are spillover effects outside of the project 4 sphere. For example, do communities take collective action Another hypothesis is that, given the amorphous, to solve other village problems, not necessarily related to the multidimensional nature of these concepts of “empower- CDD project? To measure social capital, evaluators generally ment� and “social capital,� the measurement tools or proxy examine three types of proxy indicators: indicators that were used may not be robust enough to capture them. For the impact evaluations discussed in the • Memberships in networks and associations, such as the meta-analysis, a mixture of tools was used to measure social density of networks, involvement in associations, and capital: quantitative surveys, behavioral games, and qualita- group membership; tive techniques. It would be worthwhile to further explore • Local collective action, including key questions related to how these instruments could be more effectively applied how groups of individuals work together to solve collec- when such outcomes are so heavily dependent on the social, tive action problems; economic, cultural, and political context. But despite these • Trust toward other members of the community and/or caveats, the lack of evidence on social capital impacts is different levels of government in terms of decision mak- clearly an area requiring further study and analysis because ing and the delivery of services. it is arguably at the heart of why a CDD approach should be Of the projects covered under the meta-analysis, only one— used in the first place. Sierra Leone—had an explicit goal of improving social capital. 5. Local Governance However, as a variable of high interest to development prac- titioners, social capital is still frequently measured in impact CDD programs aimed at improving local governance usu- evaluations. In the meta-analysis, eight projects looked at ally refer to changes in the way a government interacts with impacts related to greater trust, association, and collective its citizens within the project domain as well as outside of action. project parameters. This includes citizen’s participation in decision making and management, transparency of program To date, the evidence indicates that there was very little information, efficient and effective delivery of services, and positive impact on social capital. Out of the eight projects, accountability to citizens. As proxies for improved local gov- results for the Philippines KALAHI-CIDSS, Zambia Social ernance, impact evaluations often measure: Recovery Project II (SRP2), and Armenia Social Investment Fund were mixed, while the other five programs showed • People’s attitudes toward various levels of government; no impact (see annex 3). Positive stories can be found in, for • Participation in public assemblies or meetings inside and example, the Philippines KALAHI-CIDSS, which saw a 12.3 outside of the project domain; percentage point increase in the proportion of respondents who indicated that most people in their village can be trusted as compared to the control area. They also perceived that the people in their village are willing to help other people in need, with the net difference of 7.6 percent as compared to the baseline and control areas. However, even here, collec- tive action showed a decrease of 2.7 percent more in treat- ment areas compared to control areas. One possible reason for this overall lack of impact on social capital is the relatively short time span for measure- ment. Among the eight evaluations that did try to measure social capital in a rigorous way, the Indonesia’s KDP/BRA showed zero effect when the evaluation occurred after one year of implementation. The Philippines KAHAHI-CIDSS showed more positive effects and spillover impact after seven years of implementation, as described above. This could have been anticipated, because transformational soci- etal and institutional change can require longer periods of time to occur, and there can be a time lag prior to the detec- tion of these effects. 5 A number of elements might explain these positive to mixed results. Almost all the social funds started as “tem- porary� programs with semiautonomous agencies. Thus, institutional sustainability and longer-term reform were not a real focus in the earlier program phases. Many of the pro- grams were started at a time of crisis, and the fact that much- needed services and assistance were delivered at all, and that communities were satisfied with those services, may lead to a continued emphasis by governments on service delivery alone instead of on instigating real local governance reforms. It is possible that the government reform road map is not yet clearly defined or that the decentralization agenda is ambiguous when assigning roles and responsibilities to local entities. In these cases, project designs lack clarity and coherent visions. Or, for various social, political, and cultural reasons, behavioral norms encouraged or induced by the project may not spillover into other spheres of community life. Additionally, like social capital, societal and institutional • Awareness of project information and other local civic transformation may take a long time to become manifest. activities; Finally, the difficulty in defining the concept “good gover- • Spillover effects in terms of the way government officials nance� might make measurement and impact evaluations and citizens approach and manage other development difficult. programs and civic activities, such as whether or not they use a participatory approach or community-led decision- 6. Conflict making model in other programs. CDD projects exist in numerous countries affected by vio- Overall, the evidence shows positive to mixed results lent conflict. Many governments use the CDD approach to for local governance in terms of attitudes toward gover- deliver services more rapidly to their citizens and to build nance, participation in other village assemblies, awareness state-society links with a more inclusive decision-making of project information and other civic activities, and spill- process. However, given these very difficult contexts, posi- over effects in the way government officials and citizens tive results are not guaranteed. In fragile situations where approach and manage other development programs and security is problematic, goods may not be readily available, civic activities. technical expertise may be limited, and where the state’s authority may be in question, implementing any program There are only five CDD programs that explicitly can be challenging. examine local governance issues in their evaluations. Sierra Leone GoBifo demonstrated positive impacts on the The meta-analysis includes five evaluations examin- behavior of local leaders in the planning and management of ing the impact of conflict in four countries, two of them activities, increasing community confidence of local officials. fragile and conflict-affected countries—Afghanistan and The Philippines KALAHI-CIDSS also performed well, with Sierra Leone—and two dealing with more localized conflict- a five percent higher village assembly attendance outside affected situations—the Philippines and Indonesia (see of the project, a three percent increase in the awareness of annex 2). There was no impact on the macrolevels of vio- income and expense details of their local government unit, lence, except with the Moro Islamic Liberation Front rebel and improved attitudes toward government. Indonesia’s group in Mindanao, Philippines. KDP-BRA showed no improvement in local governance, but At the micro or village level, however, Indonesia dem- the project only lasted for one year, which might explain the onstrated improvements in group relations over a period lack of impact. Indonesia UPP2 was seldom recognized as of time. For example, in KDP2, there was little evidence of a a program that had the ability to solve the most important project effecting violent conflict at an aggregate level or hav- problems faced by the community. ing a direct positive impact on nonproject-related violence at the local level. However, the evaluation did find that KDP2 6 contributed significantly to better intergroup relations across a range of different identity cleavages. Improvements in the Box 2. Implementation Challenges quality of group relations grew larger over time, with vil- lages that have had KDP for four years generally showing Some project teams cited several common challenges in Imple- greater improvements than those that have had the project mentation that also affect outcomes identified in evaluations. for less time. The evaluation also showed that KDP-related Below is a snapshot of such underlining factors. conflicts were far less likely to escalate into violence than • Delayed and inadequate financial disbursements due to: cur- those related to other development projects, partly due to the rency devaluations, lateness and insufficient amount of govern- presence of effective complaints channels to defuse tensions ment counterpart financing particularly at local levels, late before they escalate. trust fund or donor, as well as community counterpart financing; In general, development programs must operate cau- • Conflict-affected situations—seven of the countries covered tiously in conflict and fragile situations. CDD programs in the meta-analysis faced varying degrees of violent conflict, are no exception. They can attract conflict by introducing making program delivery of resources and services difficult; competition for funds or by bringing development funds • Setbacks due to natural disasters such as droughts, floods, into a community, but in some cases, they can also address earthquakes, tsunami and storms—five countries were hit by longstanding grievances of exclusion and a nonresponsive such challenges, though in some cases, CDD programs allowed state or introduce community mechanisms for mediating quick and flexible response to emergency needs; burgeoning conflicts, as was seen in Indonesia. • Difficulties in coordinating with other ministries for construc- tion, supervision, O&M, and technical quality support; • Maintaining locally built infrastructure and long-term ar- Operational and Research Implications rangements for O&M; The meta-analysis points toward several important opera- • Less than optimal monitoring and evaluation systems—reli- tional and research implications as a new generation of CDD able MIS in place, need for government counterparts to use the programs emerge. available information for management and active monitoring. Evaluations need to come in time to inform the next phases of Operational Implications design or scale-up. • Be clear about project objectives and be realistic about what can be achieved, especially on governance and social capital fronts within the social, political, and eco- nomic contexts in which the projects operate. Project teams community participatory targeting when reaching more should also be ready for implementation challenges. As marginalized groups. the interviews with task team members demonstrated • In fragile and conflict-affected areas, project designers (box 2), every team faces challenges that can completely must better understand both the positive and negative alter expected impacts; these should, therefore, be fac- impacts of CDD interventions and approach the situa- tored into programs and into evaluation design. tions cautiously. More effort should be put into looking • Project teams should work with governments to for specific design features like the grievance-handling develop clearer road maps for institutional and gover- mechanisms that can effectively mitigate negative out- nance reforms. Institutional change takes time, and there comes and enhance positive impacts. is no straight, quick path to genuine reform. At the same • Invest in capacity building and training for project staff, time, CDD programs are beginning to have longer time- local government officials, and community groups. This frames. The average duration of CDD projects included in is important for long-term sustainability; it could also the meta-analysis is 11.8 years, implemented over several potentially contribute to social capital formation and phases. These longer timeframes provide an opportunity improved local governance. to discuss longer-term road maps for institutional and • Pay greater attention to the operations and maintenance social change. (O&M) arrangements for subproject investments. One • Use existing poverty maps and national statistical data possibility would be to explore piloting a variety of to improve the geographic targeting toward poor areas. options for O&M arrangements within CDD programs. At the household level, program teams must consider • Experiment in the development of stronger modes of social and political contexts in order to successfully utilize collaboration with supply-side actors, including sectoral 7 line-department and private sector agents, financial inter- • Examine why some programs are able to reach excluded mediaries, and banks for microfinance activities. This and marginalized groups better than other programs. is especially relevant for service-delivery effectiveness, This could have several design implications in terms of tar- quality improvement, and sustainability. geting criteria, training, and implementation modalities. • Unpack the black box of decision making. Very little is Monitoring, Reporting, and Evaluation Implications known or documented about how decisions are made • Monitoring, reporting, and evaluation systems need to regarding the allocation of resources. Additional qualita- be improved to provide relevant information to project tive and ethnographic work in this area would be valuable. managers overseeing programs, so that course adjust- • Analyze and build more evidence on sustainability. The ments can be made as needed. In particular, reporting and issue of sustainability came up repeatedly in the review management information systems require special atten- regarding several dimensions. CDD programs would tion. Rigorous impact evaluations of WB CDD programs benefit from additional studies on technical quality and are few and far between—only 17 rigorous evaluations O&M to sustain physical investments; institutional sus- are identified in the meta-analysis. tainability and linkages with government agencies and • Longitudinal studies are scarce. The impact evaluations the private sector that sustain investments is crucial. As included in the meta-analysis measured an average of 3.1 mentioned above, in order to examine the sustainability years of intervention despite program durations averag- of impacts, longer-term evaluations are needed to assess ing almost 12 years. There is a tension between the need whether program impacts continue or begin to yield to obtain quick information on project impacts in order diminishing returns. to inform future program directions and the time needed for some of these impacts to materialize. By their nature, Concluding Thoughts certain impacts may take longer to appear, such as gover- nance spillover effects and education learning outcomes. Depending on the country context, the emphasis by govern- Many more long-term, high-quality, and timely impact ments on achieving socioeconomic welfare goals and service evaluations are required to inform the decision-making delivery through participatory means is likely to continue. process. This may be especially true for fragile and conflict-affected countries as well as those still tackling high levels of vul- Future Research Implications nerability and pockets of poverty. However, for this next • Examine longer-term social capital and local govern- generation of longer running CDD programs, numerous ment impacts to better understand clear pathways issues emerge, including continuing poverty reduction and toward reform. Much more analytical work is warranted service-delivery efforts in an effective and sustained man- to examine pathways for longer-term local governance ner, institutional reform from the bottom-up, community reform and the enabling factors for broadening and sus- participation in governance, building in-country capacity, taining the impacts of CDD community-driven develop- enhancing community resilience and social safety nets, and ment. Local leadership transformation is also important urbanization. to sustain these changes. What the meta-analysis makes clear is that there is not one • Compare CDD approaches versus non-CDD approaches linear, straightforward path to achieving project objectives— using other government service-delivery mechanisms. or for that matter, changing social or governance norms. In None of the impact evaluations in this study compare examining some of the factors behind programs achieving “head on� CDD approaches with alternative mechanisms positive or negative results, it is evident that the undertak- like direct central- or local-government service delivery, ing of institutional reforms is a much more difficult process partly because these CDD programs grew as a response than getting roads built or children immunized. The fact that to the decades-long failure of top-down, centrally-driven many of these CDD programs arose as a direct response to service delivery. Therefore, governments do not think overly-centralized and inefficient state bureaucracies means that this hypothesis needs to be experimentally tested. that changing the way governments and bureaucracies oper- The cost for testing top-down versus bottom-up CDD ate will take time and will be a bumpy road. approaches could be both operationally and financially large. Still, more evidence along these lines would greatly inform the policy decision making. 8 Annex 1. Summary of Impact Evaluations Name of CDD Program Associated Evaluation Papers/Studies 1 Afghanistan: National Solidarity Beath, Andrew, Fotini Christia, Reuben Enikolopov, and Shahim Ahmad Kabuli. 2010. “Randomized Program II (NSP2) Impact Evaluation of Phase II of Afghanistan’s National Solidarity Programme (NSP).� Estimates of Interim Program Impact from First Follow-up Survey, NSP, Afghanistan. 2 India: Andhra Pradesh District Poverty Deininger, Klaus, and Yanyan Liu. 2009. “Longer-Term Economic Impacts of Self-Help Groups in Initiative Program (APDPIP) India.� Policy Research Working Paper No. 4886, Development Research Group, Sustainable Rural and Urban Development Team, World Bank, Washington, DC. 3 Nepal: Poverty Alleviation Fund II Parajuli, Dilip, Gayatri Acharya, Nazmul Chaudhury, and Thapa Bishnu Bahadur. 2012. Impact of (PAF2) Social Fund on the Welfare of Rural Households: Evidence from Nepal Poverty Action Fund. World Bank, Washington DC. 4 Indonesia: Kecamatan Development Voss, John. 2008. Impact Evaluation of the Second Phase of the Kecamatan Development Program. Program II (KDP2) Jakarta: World Bank. Barron, Patrick, Rachael Diprose, and Michael Woolcock. 2006. “Local Conflict and Community Development in Indonesia: Assessing the Impact of the Kecamatan Development Program.� Indonesian Social Development Paper No. 10, World Bank, Washington DC. 5 Indonesia: PNPM–Rural Generasi Sehat Olken, Benjamin A., Junko Onishi, and Susan Wong. 2010. Indonesia’s PNPM Generasi Program: dan Cerdas (PNPM Generasi) Interim Impact Evaluation Report. Jakarta: World Bank. 6 Indonesia: KDP/Badan Reintegrasi Barron, Patrick, Macartan Humphreys, Laura Paler, and Jeremy Weinstein. 2009. Aceh (KDP/BRA) “Community-Based Reintegration in Aceh: Assessing the Impacts of BRA-KDP.� Indonesian Social Development Paper No. 12, World Bank, Washington DC. 7 Indonesia: Urban Poverty Program II Pradhan, Menno, Vijayendra Rao, and Christina Rosemberg. 2010. “The Impact of the Community (UPP2) Level Activities of the Second Urban Poverty Project.� Draft. 8 Philippines: Kapit-bisig Laban Sa Asian-Pacific Policy Center. 2011. Final Survey for the Kalahi-CIDSS Impact Evaluation. Kahirapan—Comprehensive and Manila: Asia-Pacific Policy Center. Integrated Delivery of Social Services Program (KALAHI-CIDSS) Labonne, Julien. 2011. The Kalahi-CIDSS Impact Evaluation: A Synthesis Report. Discussion Paper. Sustainable Development, East Asia and Pacific Region. Washington, DC: World Bank. 9 Senegal: Programme National Arcand, Jean-Louis, and Leandre Bassole. 2007. “Does Community Driven Development Work? d’Infrastructures Rurales (PNIR) Evidence from Senegal.� Draft. CERDI-CNRS, Université d’Auvergne, and EUDN. Development Paper No. 10, World Bank, Washington DC. 10 Sierra Leone: GoBifo Casey, Katherine, Rachel Glennerster, and Edward Miguel. 2011a. “The GoBifo Project Evaluation (JSDF-funded program) Report: Assessing the Impacts of Community Driven Development in Sierra Leone.� Final Report, Brown University, Abdul Latif Jameel Poverty Action Lab, and University of California, Berkeley. _____. 2011b. “Reshaping Institutions: Evidence on Aid Impacts Using a Pre-Analysis Plan.� Draft. Brown University, Abdul Latif Jameel Poverty Action Lab, and University of California, Berkeley. 11 Tanzania Social Action Fund (TASAF) Baird, Sarah, Craig McIntosch, and Berk Özler. 2009. The Squeaky Wheels Get the Grease: Applications and Targeting in Tanzania’s Social Action Fund. The World Bank, Washington DC. 12 Zambia Social Recovery Project II Chase, Robert S., and Lynne Sherburne-Benz. 2001. Household Effects of African Community Initiatives: Evaluating the Impact of the Zambia Social Fund. Washington DC: World Bank. 13 Bolivia Social Investment Fund II Newman, John, Menno Pradhan, Laura B. Rawlings, Geert Ridder, Ramiro Coa, and Jose Luis Evia. 2000. “Impact Evaluation of Social Funds. An Impact Evaluation of Education, Health, and Water Supply Investments by the Bolivian Social Investment Fund.� The World Bank Economic Review 16 (2): 241–274. 14 Honduras Social Investment Fund III ESA Consultores International. 2005. Evaluación Ex-post Del Fondo Hondureño de Inversión Social (FHIS3) (FHIS 3). Elaborado a solicitud del FHIS. Informe Borrador. Tegucigalpa. 15 Nicaragua Emergency Social World Bank. 1998a. Nicaragua. “Second Investment Fund Project.� ICR. Report No. 18657, Human Investment Fund II (FISE2) and Social Development Department (LCHSD), Central America Country Management Unit, Latin America and the Caribbean Regional Office. 16 Peru Social Fund (FONCODES) Paxson, Christina, and Norbert R. Schady. 2002. “Impact Evaluation of Social Funds. The Allocation and Impact of Social Funds; Spending on School Infrastructure in Peru.� World Bank Economic Review 16 (2): 297–319. 17 Armenia Social Investment Fund Chase, Robert S. 2002. “Impact Evaluation of Social Funds. Supporting Communities in Transition: (ASIF) The Impact of the Armenian Social Investment Fund.� World Bank Economic Review 16 (2): 219–240. 9 Annex 2. Summary of Impacts on Poverty Reduction, Targeting and Use/Access to Services Poverty Reduction Targeted toward poor, Use/Access to Services Name of Project (consumption/expenditure) vulnerable groups (education, health, water, roads) 1 AFGHANISTAN— X No difference in household √ Yes: For female groups— √ Yes: 20 percentage point increase in water, NSP2 Interim income, per capita annual engagement in community life, strong impact on connectivity and usage in Evaluation income, or median per capita medical care, schooling, and electricity income involvement in income-generating activities X No: Transport, irrigation; weak in health, school 2 INDIA— √ Yes: 11 percentage point √ Yes: 15% nutritional gains; 50% N/A Andhra Pradesh increase in per capita increased assets for the poor, but DPIP consumption; 26% increase in not consumption assets 3 NEPAL— √ Yes: 19 percentage point √ Yes: Higher decline in food √ Yes: Education; school participation of Poverty increase in per capita insecurity; 24 percentage points for 6-15 aged children 14% (girls 21%) point net Alleviation consumption; 19% decline in disadvantaged HHs (for example, increase Fund incidence of food insecurity from disadvantaged caste/ethnic groups) X No: Health or child malnutrition 4 INDONESIA— √ Yes: 11 percentage point √ Yes: 11% higher real per capita √ Yes: Access to health case was 11.5% KDP2 increase in per capita consumption gains among poor higher in program area consumption for poor households households; 9.3% higher proportion of households moving out of X No impact on enrollment rates √ Yes: 1.5% reduction in poverty in poor districts; vulnerable unemployment households near the poverty line were less at risk of falling into poverty X No for female-headed households 5 INDONESIA— √ Yes: 11% decline in “poor� √ Yes: Larger proportion of conflict √ Yes: Conflict victims see the land they KDP-BRA reported by village head; victims (24%) than nonvictims (16%) farm double as a result of the program increase in assets and farming received support of land X No impact on education, health, or ≈ Mixed: Blunt at household (HH) infrastructure X No impact on employment level; both conflict and non-conflict- levels affected persons benefited 6 INDONESIA— Not the objective √ Yes: Twice as effective in areas √ Yes: Overall statistically significant PNPM Generasi at the lowest health and education positive impacts on 12 education and health coverage at baseline indicators ≈ Mixed: Partially at HH level, only education, not health 7 INDONESIA— X No impact on per capita X No overall: 9% improved access for √ Yes: 3% for improved access to sanitation; Urban Poverty consumption or assets sanitation among the poorest, but 9% for poorest Program II project groups consisted mostly of the more educated, affluent, and X No impact on access to clean water officially connected 8 PHILIPPINES— √ Yes: 5% increase in per capita √ Yes: The poorest villages were √ Yes: Increase in access to health facilities, KALAHI-CIDSS consumption more likely to be prioritized during secondary school (1.3 percentage points the multivillage decision-making higher in treatment); college (an increase forums. The greatest impact on per in the treatment and a decrease in the capita consumption was found in the control group with a net positive change poorest households and communities of 5.4 percentage points); 50% increase (5% increase) from baseline; 9% difference between the treatment and control group with regard to the number of financing institutions X No impact on access to water and elementary school enrollment rates 9 SENEGAL— √ Yes: 65% increase in household √ Yes: For poorer households, but √ Yes: Increased access to clean water PNIR expenditure; could be due to there is political patronage with (22.4%) and health service (24.1%); child the particularly low baseline— regard to which villages receive nutrition measured in anthropometrics US$0.23 per household member funding significantly improved despite the absence per day of targeting in the PNIR (continued) 10 Annex 2. Continued Poverty Reduction Targeted toward poor, Use/Access to Services Name of Project (consumption/expenditure) vulnerable groups (education, health, water, roads) 10 SIERRA LEONE— √ Yes: Highly significant 0.399 Not indicated in report √ Yes: Increase in community centers GoBifo standard deviation unit increase (24.1%); schools, seed bank (17.2%); in household assets; 30% increase latrines (21%); a strong positive impact in the number of petty traders on the quality of the materials and construction for primary schools, grain drying floors, water wells, and latrines 11 TANZANIA— N/A √ Yes: Mildly pro-poor overall, N/A Social Action progressive for national geographic Fund II targeting, neutral within district targeting 12 ZAMBIA— Not the objective √ Yes: In rural areas with 15% of √ Yes: 86% of children were in school in Social Recovery the resources going to the poorest treatment compared to 82% in control; Project II 10%; 25% going to the poorest 20% 49% of households reported sickness in treatment compared to 41% in control X No: In urban areas (given the same level of actual sickness, the program increased awareness) 13 BOLIVIA— Not the objective √ Yes: Progressive for poorest √ Yes: Health centers and water supply Social Investment districts; poorer households systems seem to have led to a significant Fund benefited; health and education reduction in under-age-five mortality did better at reaching the poor; (a decline from 88.5 deaths per 1,000 to sewerage fared worse; for HH 65.8 per 1,000) targeting, in all six countries, SF benefits were concentrated among X No: School infrastructure led to little the poor, with poorer households improvement in education outcomes, except more likely to benefit from a SF for a 3–3.8 % decline in dropout rates 14 HONDURAS— Not the objective investment than ones that are √ Yes: Education (15% increase in Social Investment better off age-for-grade); health (10% increase in Fund the share of sick people seeking professional medical services) 15 NICARAGUA— Not the objective √ Yes: Education (5–10% higher primary Emergency Social enrollment rates in treatment); increase Investment Fund in the share of households with access to piped water; health (incidence of stunting falling from 25–14%) 16 PERU— Not the objective √ Yes: Education (2% increase in the Social Fund probability of being in school for extreme (FONCODES) poverty households), but no impact with indigenous communities 17 ARMENIA— Not the objective ≈ Mixed: On average, beneficiary √ Yes: 87% primary enrollment in treatment Social Investment households were less well off than compared with 79% in control; 93% of Fund other households; Regressive in treatment HH have access to cold water rural, progressive in urban compared to 85% in control X No impact on health outcomes SUMMARY Out of 9 projects, 7 positive Out of 16 projects, generally Out of 15 projects, generally positive positive to mixed 11 Annex 3. Summary of Impacts on Governance, Social Capital, and Conflicts Name of Project Governance Social Capital Conflict 1 AFGHANISTAN— √ Yes: 27% in treatment compared X No: Weak or no evidence X Limited effects: No impact on National Solidarity Program to 9% in control villages; male that NSP affects trust between specific measures of community II (NSP2) Interim evaluation villagers report that village villagers, solidarity, or outbreaks trust or solidarity, on the outbreak assembly meets on regular basis; of village disputes and tribal feuds of village disputes, or tribal more awareness of CDC issues and feuds; very limited effect on more positive attitudes toward the prevalence of conflict and government; positive for women perceptions of safety and security 2 NEPAL— Not measured X No significant difference Not measured Poverty Alleviation Fund in trust, respect, relationship (PAF) between different ethnic groups, community disputes, and so on. 3 INDONESIA— Measured in next survey Measured in next survey ≈ Mixed: Little impact on Kecamatan Development conflict at an aggregate level Program II (KDP2) and little direct positive impact on nonproject-related violence at the local level; however KDP contributed significantly to improvements in intergroup relations, including ethnic, religious, and class; appears to be effectively reengineering relationship between citizens and the state at the local level 4 INDONESIA— X No: Trust in government; X No: Conflict victims in areas X No: After one year of project KDP-BRA knowledge of government affairs; receiving program were 18% implementation, no impact on attitudes about government similar less likely to say they accept social cohesion; in fact there is in treatment and control areas ex-combatants in all roles in evidence of decreased acceptance village life compared to control of ex�combatants by conflict areas; while not resulting in victims in project areas, but there increased social tensions or is no evidence that these tensions conflict, it suggests that the escalated into violence program was not effective in building trust between victims and former combatants 5 INDONESIA— X No: UPP seldom mentioned as a X No impact on community Not measured Urban Poverty Program II program that deals with the most participation, organizational (UPP2) important problems of the urban membership, or participation in village community-initiated activities 6 PHILIPPINES— √ Yes: No significant improvement √ Yes: 12.3 percentage point ≈ Mixed: 35% decrease in the KALAHI-CIDSS in individual’s trust to local increase in the proportion of number of Moro Islamic Liberation officials, but significantly higher respondents indicating that Front related conflict events; 41% trust toward national officials; most people in their village can increase in conflict events related 5% increase in attendance in be trusted; more households in to the New People’s Army village assemblies. Proportion of treatment perceived that people households aware of income and in their respective barangays are expense details of their barangay willing to help other people if local government unit increased by needed; the net difference is 7.6 3% compared to control areas percentage points X No: Collective action showed a 2.7 percentage point decrease in treatment (continued) 12 Annex 3. Continued Name of Project Governance Social Capital Conflict 7 SIERRA LEONE— X No systematic differences: X No treatment effects on the X No impact: Only 1 out of 10 GoBifo between how decisions get made standard proxy measurements for indicators related to conflict or in treatment versus control social capital—trust, collective violence considered is statistically communities action, groups and networks, significant—a reduction in inclusion, and information. Also household reports of physical X No change: in the role of women no indication of spillover of local fighting over the past one year and youth in community decision norms or institutional practices making outside the project itself outside the immediate project X No impact: on participant’s sphere. access to information about local X No impact: on crime and governance violence √ Yes: more active and improving community attitudes toward local government 8 ZAMBIA— Not measured √ Yes: In rural areas, 60% of social Not measured Social Recovery Project II investment fund communities felt school rehabilitation activity increased social capital compared to 44% in control communities X No: In Urban areas, only 25 % of households felt the urban social fund projects increased social capital, a proportion significantly less than for other projects with which they were compared 9 ARMENIA— Not measured ≈ Mixed: Communities that had Not measured Social Investment Fund completed the subproject were (ASIF) less likely than the control group to complete other local infrastructure projects, but communities that joined ASIF later than the first beneficiary group and that had not yet completed their projects reported more collective action SUMMARY Out of 5 Projects, Out of 8 Projects, Out of 5 Projects, Positive to Mixed Mixed to No Impact Mixed to Negative 13 Annex 4: World Bank CDD Programs Covered in this Study EUROPE AND CENTRAL ASIA Armenia Social Investment Fund. 1996–2012 (1996–2000). AFRICA Senegal. Programme National d’Infrastructures Rurales (PNIR). LATIN AMERICA AND THE CARIBBEAN 2000–12 (PNIR/PLDP) (2000–05). Water supply, schools, health Bolivia. Social Investment Fund II. centers or posts, livestock, women’s 1987–2004 (1993–98). Water; school activities, agriculture, youth and construction; health and sanitation. sports, commerce, and miscellaneous. Honduras. Social Investment Fund III. Sierra Leone. GoBifo. 2005–10 1990–2012 (1995–99). Small-scale (2005–09). Local public goods infrastructure; furniture and construction; agriculture and livestock equipment; and training for management; skills training and education, health, water, and income generation; and social sanitation; child and elderly care projects, such as youth sport clubs. centers and orphanages. Tanzania. Social Action Fund Nicaragua. Emergency Social Project II. Only for analysis on Investment Fund II. 1992–2006 targeting aspect. 2000–13 (2004–13). (1994–97). Social infrastructure— Education, health, water, roads, 70 percent; education; health; irrigation, markets, and support to economic infrastructure; social vulnerable groups. services; and environment. Zambia. Social Recovery Project II. Peru. Social Fund (FONCEDES). 1991–2005 (1995–2000). Education— 1993–2000 (1993–97). Social 80%; health—12%; and nutrition and infrastructure (school, water, and economic infrastructure. sanitation); social assistance; economic infrastructure (rural roads, bridges, irrigation system); and productive projects. 14 REGION Country. Project name. Total project period (period relevant to IE). Types of subprojects funded under IE phase EAST ASIA AND THE PACIFIC Indonesia. Kecamatan Development Program II (KDP2). 1998–2014 (2002–07). Rural village infrastructure—70%; revolving funds and social services—30%. PNPM-Rural Generasi Sehat dan Cerdas (PNPM Generasi). 1998–2014 (2007–10). School rehabilitation, potable water, Education (56%) and health activities (44%) to meet 12 education and maternal and and irrigation and health facilities. child healthcare targets. KDP/Badan Reintegrasi Aceh (KDP/BRA). 2006–07 (same). Economic activities such as livestock (89%) and rural infrastructure (10%). Urban Poverty Program II (UPP2). 1999–2014 (2002–08). Revolving funds for providing microcredit services, small infrastructure improvements (roads, drains, water, sanitation, and so on). Philippines. Kapit-bisig Laban Sa Kahirapan—Comprehensive and Integrated Delivery of Social Services Program (KALAHI CIDSS). 2002–14 (2003–10). Rural infrastructure (water systems, school buildings, and so on—50%; basic transport infrastructure—28%; and community enterprise facilities. SOUTH ASIA Afghanistan. National Solidarity Program II. 2003–15 (2006–11). Infrastructure (water, sanitation, rural road, irrigation, and so on) and human capital development. India: Andhra Pradesh District Poverty Initiative Program (DPIP). 2000–16 (2000–06). Microfinance activities: household dairy—30%; agriculture—29%; and nonfarm trade and sheep rearing—10%. Nepal. Poverty Alleviation Fund II (PAF 2). 2004–14 (2008–14). Income generation (livestock, agriculture)—72%; small-scale infrastructure and services (microirrigation, water supply, and so on)—17%. 15 Notes —including elections, decentralization, and school-based development approaches�­ management—it is worth fractionating out and understanding more about a specific 1. CDD Database, World Bank. http://sdweb.worldbank.org/cdd/index.cfm?Page= approach to participation from within the overall participatory approach universe, Home. “Almost US$30 billion� includes projects that adopted a CDD approach either that is World Bank-supported CDD programs. This study thus focuses much more as a whole or in specific components. narrowly on this specific subset of CDD and its impact evaluation findings. Third, this 2. Wong, Susan, 2012. “What Have Been the Impacts of World Bank Community-Driv- study attempts to focus more than past studies on contextual factors and the opera- en Development Programs? CDD Impact Evaluation Review and Operational & Re- tional implications of the findings. search Implications.� Social Development Department. World Bank, Washington DC. 4. Rawlings, Laura, L. Sherburne-Benz, and J. van Domelen. 2003. Evaluating 3. This study differs from earlier reports in several important respects. First, the study Social Funds: A Cross-Country Analysis of Community Investments. Washington DC: team has updated the data sets, focusing especially but not exclusively on the latest World Bank. generation of CDD impact evaluations from 2003 onwards. Second, whereas some 5. Grootaert, Christiaan, and Thierry van Bastelaer. 2002. Understanding and Measur- recent reports (such as Mansuri and Rao. 2012. “Localizing Development: Does Partici- ing Social Capital: A Multidisciplinary Tool for Practitioners. Washington, DC: World pation Work?�) cover a very broad range of topics under the heading of “participatory Bank. This summary note was prepared by Kaori Oshima in the Social Development Department, the World Bank, with overall guidance from Janmejay Singh and Susan Wong. Hélène Grandvoinnet and Sanjay Agarwal provided valuable comments. Laura C. Johnson provided assistance with editing and layout. The original paper: Wong, Susan, 2012. What Have Been the Impacts of World Bank Community-Driven Development Programs? CDD Impact Evaluation Review and Operational and Research Implications. Social Development Department. World Bank. http://imagebank.worldbank.org/servlet/WDS_IBank_Servlet?pcont=details&menuPK=64154159&search MenuPK=64154240&theSitePK=501889&siteName=IMAGEBANK&eid=000386194_20120614062031. The World Bank, Social Development Department Fax: 202-522-3247 1818 H Street NW Email: asksocial@worldbank.org Washington DC 20433 Web: www.worldbank.org/socialdevelopment CDD website: www.worldbank.org/cdd