THE WORLD BANK W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N T 34773 The Effectiveness of World Bank Support for Community-Based and -Driven Development OPERATIONS EVALUATION DEPARTMENT ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N T The Effectiveness of World Bank Support for Community-Based and -Driven Development An OED Evaluation 2005 The World Bank http://www.worldbank.org/oed/cbdcdd Washington, D.C. © 2005 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone 202-473-1000 Internet www.worldbank.org E-mail feedback@worldbank.org 1 2 3 4 09 08 07 06 05 All rights reserved Manufactured in the United States of America The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in the work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Cover photo: © Giacomo Pirozzi/Panos Pictures. ISBN-10: 0-8213-6390-5 ISBN-13: 978-0-8213-6390-4 e-ISBN: 0-8213-6391-3 DOI: 10.1596/978-0-8213-6390-4 World Bank InfoShop Operations Evaluation Department E-mail: pic@worldbank.org Knowledge Programs and Evaluation Capacity Telephone: 202-458-5454 Development (OEDKE) Facsimile: 202-522-1500 E-mail: eline@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 Printed on Recycled Paper i i Contents vii Acknowledgments ix Foreword xi Executive Summary xvii Acronyms and Abbreviations 1 1 Origin, Scope, Design, and Methodology of Evaluation 2 Scope 2 Design 4 Methodology 4 Some Limitations of the Study Design 7 2 The CBD/CDD Portfolio 7 Temporal, Regional, and Sectoral Distribution of the Portfolio 7 Ways That CBD/CDD Projects Differ from Those in the Non-CBD/CDD Portfolio 11 3 Outcome of Bank-Supported CBD/CDD Projects 11 Outcome Ratings 11 Relevance 15 Efficacy 22 Targeting the Poor Is Not Enough to Reach the Poor 25 Efficiency 29 4 Institutional Enhancement and Sustainability 29 Institutional Enhancement 36 Sustainability 41 5 Bank Operational Policy Requirements, Processes, and CBD/CDD Interventions 41 The Bank Has Attempted to Adapt Its Policies to Design and Implement CBD/CDD Projects 45 Bank Capacity to Ensure Effective Implementation of CBD/CDD Interventions i i i THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 49 6 Conclusions 51 7 Recommendations 53 Annexes 55 Annex A: Definition of "Community" 57 Annex B: Results Chain for World Bank CDD Projects 59 Annex C: World Bank Guidance on Key Design Principles for CDD 63 Annex D: Study Framework, Methods, and Instruments 67 Annex E: The Universe of CBD/CDD Projects and Its Distribution 71 Annex F: Sample of CBD/CDD and CDD Projects 77 Annex G: OED Ratings of Completed Projects 81 Annex H: Focus on CBD/CDD and Related Aspects in Bank and Borrower Strategy 85 Annex I: Central Government and Local Government Surveys 89 Annex J: Efficiency 95 Annex K: Nkayi District Formal and Informal Systems 97 Annex L: Results of Bank Staff Survey 103 Annex M: Methodology for Community-Level Data Collection and Analysis 119 Annex N: Enhancing Community Capacities 141 Annex O: Poverty Targeting 143 Annex P: Evidence from Fieldwork on Sustainability 147 Annex Q: Safeguard Thematic Study: A Summary 151 Annex R: Advisory Committee Comments 159 Annex S: Management Response 173 Annex T: Chairman's Summary: Committee on Development Effectiveness 177 Endnotes 191 References Boxes 3 1.1: Primary Data Collection for the Evaluation of CBD/CDD Projects 15 3.1: Change in Social Capital and Empowerment as a Means of Assessing the Community Capacity­Enhancing Impact of Bank CBD/CDD Interventions 17 3.2: Highly Satisfactory Aspects of Design and Implementation in Selected CBD/CDD Projects 18 3.3: Local Champions Can Effectively Use Bank CBD/CDD Funds: A Case from Ghana 24 3.4: Example of Sophisticated Poverty Targeting in Two Recent Vietnam CDD Projects 32 4.1: Inconsistent Strategies: The Road Sector Experience in Nepal 33 4.2: Bank CDD Projects Have Added to the Proliferation of Municipal Councils in Rio Grande do Norte (Brazil) 40 4.3: Why Formal Groups Do Not Last Long 43 5.1: The Meaning of the Environmental Categories 44 5.2: The Fiduciary Challenge: The Case of Vietnam 48 5.3: Some Monitoring and Evaluation Features to Emulate: The Case of the Indonesia Kecamatan Development Project i v C O N T E N T S Figures 2 1.1: The Universe of CBD/CDD Projects 8 2.1a: Bank Commitments to Projects with CBD/CDD Approaches Have Grown 8 2.1b: Africa Has the Largest Share of Projects with CBD/CDD Approaches 9 2.2: Communities Have More Responsibility in CDD Projects 12 3.1: Outcome Ratings of CBD/CDD Projects Have Been Better Than Those of Non-CBD/CDD Projects 13 3.2a: CBD/CDD Projects in the Latin American and Caribbean Region Have Done Better Than Projects in Other Regions 13 3.2b: CBD/CDD Projects in Education and Social Protection Do Better Than Projects in Health and Rural Development 14 3.3: CBD/CDD Is Increasingly Important in Country Programs 16 3.4: Community Priorities Are Better Met When There Is Greater Choice 20 3.5: Focus Groups Report Significant Decision Making by Local Leaders 21 3.6: Subproject Cycle Is Too Short for Meaningful Enhancement of Community Capacity 26 3.7: CBD/CDD Projects Cost the Bank More Than Non-CBD/CDD Projects 34 4.1: Low-Income Countries Work with More Partners 36 4.2: Sustainability Has Been Consistently Lower for CBD/CDD Projects but Is Improving Table xii ES.1: Overview of CBD/CDD Strengths and Weaknesses v Acknowledgments The report was prepared by a core team con- Thailand; and Professor Robert Chambers, Re- sisting of Nalini Kumar (Task Manager), Barbara search Associate at the Institute of Development Pozzoni, Anju Vajja, and George Garner Woodall. Studies at the University of Sussex (UK). Additional team members were Ewa Gradzka, Colleagues inside the Bank provided com- Kavita Mathur, Sonali Moonesinghe, Ridley Nel- ments at various stages. They include: Ruth son, Floribert Ngaruko, Camilo Gomez Osorio, Alsop, Hans Binswanger, Victor Botini, Ed Bres- and Peter Whitford. In addition to the core team, nyan, Mark Cackler, Robert Chase, Luis Coirolo, data collection was undertaken by Dr. Alberto Monica Das Gupta, Gershon Feder, Scott Coelho Gomes Costa and his team of surveyors Guggenheim, Jeffrey Gutman, Steen Jorgensen, in Brazil, the staff and students of the Centre for Christine Kessides, Ghazala Mansuri, Alexander Development Economics, Delhi School of Eco- Marc, Robin Mearns, Deepa Narayan, Daniel nomics in India, and Dr. Roch Mongbo and his Owen, Vijayendra Rao, Martin Ravallion, John team of surveyors in Benin. William Hurlbut ed- Redwood, Louise Scura, Susan Wong, Mark ited the original report and Caroline McEuen Woodward, and Michael Woolcock. edited the manuscript for publication. Tomas We would like to thank the staff of the World Caspellan, Soon-Won Pak, and Helen Phillip pro- Bank Country Offices in Benin, Brazil, Egypt, vided administrative support. Julius Gwyer pro- India, Nepal, and Vietnam for all their support. vided technical support for the maintenance of OED also gratefully acknowledges the input of the Web site. Jacintha Wijesinghe and Etsegenet numerous community members and other stake- Nigussie advised on the budget. The report holders across various countries who patiently benefited considerably from the guidance pro- participated in interviews and surveys and pro- vided by Alain Barbu, Ajay Chhibber, Nils Fos- vided valuable perspectives. tvedt, Patrick Grasso, Gregory Ingram, and Kyle We are grateful for comments from the fol- Peters. lowing OED staff: Martha Ainsworth, Bill Bat- The External Advisory Panel for the review taile, Soniya Carvalho, Victoria Elliott, Fareed consisted of Dr. Norman Uphoff, Director of the Hassan, Edoardo Masset, Janardan Prasad Singh, Cornell International Institute for Food, Agri- Denise Vaillancourt, and Howard White. culture and Development and Professor of Gov- This review was greatly enhanced by the gen- ernment at Cornell University; Mr. Paiboon erous support and input of several donors. Sub- Wattanasiritham, Chairman of the Community stantial feedback and financial support from the Organizations Development Institute (CODI) in Swiss Agency for Development and Coopera- v i i THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT tion--in particular Laurent Ruedin, Joerg Frieden, Partnership and the Norwegian Consultancy Robert Groeli, Walter Meyer, and Ursula Funk-- Trust Fund, are gratefully acknowledged. as well as contributions from the Norway MFA Acting Director-General, Operations Evaluation: Ajay Chhibber Acting Director, Operations Evaluation Department: R. Kyle Peters Manager, Sector, Thematic, and Global Evaluation: Alain Barbu Task Manager: Nalini Kumar v i i i Foreword T his report analyzes the effectiveness of the World Bank's lending sup- port for the growing area of community-based development (CBD) and community-driven development (CDD). The latter supports the em- powerment of the poor by giving communities control over subproject re- sources and decisions, while CBD gives communities less responsibility and emphasizes collaboration, consultation, or sharing information with them on project activities. Since the late 1990s, the focus of Bank-supported CBD/CDD projects has shifted toward CDD, though many CDD projects also include CBD components. The share of projects that include a CBD/CDD The study finds that Bank-supported component has grown from about 2 percent in CBD/CDD projects have typically performed bet- fiscal 1989 to 25 percent in 2003. Over the ter on capacity enhancement--an important ob- 1994­2003 period, the outcome ratings of jective of the projects--when they have CBD/CDD projects have been better than those supported indigenously matured participatory ef- for non-CBD/CDD projects. Much more success forts or when the Bank has provided sustained, has been achieved in CBD/CDD projects on long-term support to communities beyond the quantitative goals such as construction of infra- length of a single subproject. structure than on qualitative goals such as ca- The distribution of costs and benefits--to the pacity enhancement. Sustainability ratings for institution, the borrower, or the communities-- the projects have improved over time, but there related to undertaking CBD/CDD projects has is considerable room for improvement. Bank in- not been systematically assessed by the Bank. The terventions have often failed to provide the con- OED study finds that CBD/CDD projects are sistent, long-term support needed for an activity more expensive to prepare and supervise than to become sustainable (for example, in a forestry other Bank projects. Governments also spend project, support should be provided until the for- considerable resources to put a participatory est starts yielding adequate returns from tim- approach in place. While a participatory ap- ber and non-timber products). proach lowers the cost to the government for ser- i x THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT vice-delivery infrastructure, the communities Going forward, the Bank could do a number bear a significant part of this cost burden. of things to improve its effectiveness for com- CBD/CDD projects have increased access to munity-based and community-driven develop- service delivery infrastructure such as schools ment. The study makes three recommendations: and health centers for remote communities. In addition, several CBD/CDD projects in conflict · In undertaking new CBD/CDD projects, the and post-conflict countries have helped in re- Bank should analyze whether it is building on habilitating infrastructure and have provided existing local initiatives or starting a new pro- significant employment benefits to the local gram in a country and give priority to the for- population. However, as with other projects, in- mer. In the latter case, the Bank should tailor creased access to infrastructure does not always the initiative to the country and community translate into effective service delivery. Further, context and undertake selective, rigorous im- the poorest may not always benefit from these pact assessments of its projects before scaling projects. There is little hard evidence to date on up. the poverty-reducing and community capac- · The Bank needs to strengthen operational ity­enhancing impact of the projects. guidance for the application of safeguard Bank-supported CBD/CDD projects have en- policies and fiduciary oversight of CBD/CDD hanced the capacity of government institutions to projects and for cost-benefit analysis and mon- implement participatory interventions, but few itoring and evaluation (M&E) systems, and it borrower governments have adopted the ap- should commission an audit of the fiduciary proach more widely in their development pro- aspects of a representative sample of CDD grams. projects to be submitted to the Board within Adaptation of operational policies and decen- a year. tralization of the Bank to field offices have en- · CBD/CDD projects need to be integrated with hanced the Bank's capacity to implement a country's assistance strategy. Future Country CBD/CDD interventions, but additional changes Assistance Strategies (CASs) should show how are needed. Weaknesses in monitoring and eval- they have analyzed and addressed linkages uation, the need for development of adequate not only between CBD/CDD projects, but also guidelines for staff on safeguards for CDD projects, between CBD/CDD and relevant non- and the short time span of the Bank's subproject CBD/CDD projects. Such analysis should also cycle constrain the Bank's capacity to implement address whether arrangements for CBD/CDD CBD/CDD projects. These projects, most partic- project implementation are made at the ex- ularly CDD, also continue to pose a challenge for pense of local government capacity develop- both safeguard and fiduciary compliance. ment. Ajay Chhibber Acting Director-General, Operations Evaluation x Executive Summary P articipatory approaches that involve local communities in their own development have gained substantial support among international donors over the past quarter-century and have become increasingly important in the work of the World Bank. Community participation is an ap- proach to development that can be used with any Bank lending instrument and across sectors. Projects can involve communities in different ways--by shar- ing information, consulting, collaborating, or empowering them. The process of involving communities in project activities is also expected to contribute in most cases to community capacity enhancement. The World Bank's support for community cluding the World Bank, were impressed by participation has been manifested in the de- the poverty-reducing effects of local ini- sign and implementation of either com- tiatives that developed independently in munity-based development (CBD) or several countries. In these islands of success, community-driven development (CDD) local communities had taken control of their projects. Although the literature does not clearly lives as a result of independent forms of social distinguish CBD from CDD,1 there is increasing action. In its ongoing effort to reduce poverty, consensus that CDD projects give communities the Bank has emulated these local initiatives by control over resources and decisions in the de- attempting to enhance community capacity by sign and implementation of subprojects. CBD building social capital and fostering empower- projects, however, give communities compara- ment in communities through its projects. Re- tively less responsibility and emphasize collab- search into the multidimensional nature of oration, consultation, and information sharing poverty has further reinforced the importance with them. Since the late 1990s, the focus of of empowerment. Today, the Bank's Strategic such Bank-supported projects has shifted to- Framework identifies empowering poor people ward CDD, although many CDD projects also in- to participate in development as one of the two clude CBD components. priorities in the fight against poverty. Interest in community empowerment Interest in a community-based approach emerged in large part because donors, in- is also predicated on a belief that not only x i THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT would it lead to better allocation of re- where state capacity is weak or has been weak- sources to help communities, but would ened by conflict and other factors. also lead to reduced corruption and mis- use of resources, and thereby more devel- Evaluation Findings opment assistance would reach the poor. The Bank has not, until recently, system- Community involvement is expected to increase atically identified and tracked its portfolio transparency and accountability by working di- of CBD/CDD projects,2 and therefore has rectly with the ultimate beneficiaries, especially lacked a comprehensive understanding of Table ES.1: Overview of CBD/CDD Strengths and Weaknesses3 Strengths of Bank CBD/CDD projects Weaknesses of Bank CBD/CDD projects Outcome ratings are generally better for CBD/CDD than for non- The Bank's project M&E systems do not allow systematic assessment CBD/CDD projects. Much more success has been achieved in of the capacity-enhancing impact of CBD/CDD interventions. It is often CBD/CDD projects on quantitative goals, such as the construction of assumed that meeting the quantitative goals will automatically fulfill infrastructure, than on qualitative goals, such as capacity enhance- the qualitative goal--for example, holding a certain number of train- ment or quality of training. ing courses is expected to enhance capacity. Borrower officials believe that a participatory approach can con- Borrower officials do not necessarily believe that community control over tribute to poverty alleviation. decisions and resources in a subproject is the best means of engaging communities. CBD/CDD projects help lower the cost to government of delivering Communities bear an increased share of the burden for service deliv- infrastructure. ery infrastructure. They have increased access of remote communities to service-delivery But the poorest may not always benefit from CBD/CDD projects. infrastructure such as schools, health centers, and the like. Capacity-enhancement effort in a CBD/CDD project has been more The individual subproject cycle is too short to sustainably enhance successful when a Bank project supports indigenously matured ef- community capacity where it is weak or does not exist. forts or provides sustained, long-term support to communities beyond a Bank subproject cycle. Sustainability ratings have improved over time. Infrastructure and services have been difficult to sustain beyond the Bank presence because of a lack of resources from the government and com- munities to ensure their operation and maintenance. CBD/CDD projects have enhanced government capacity to implement Few governments appear to have adopted the CBD/CDD approach more participatory interventions. widely in their own development programs. Adaptation of Bank policies and decentralization to field offices have More changes are needed to improve fiduciary and safeguard compli- enhanced Bank capacity to implement CBD/CDD projects. ance in CBD/CDD projects. x i i E X E C U T I V E S U M M A R Y the evolution and scope of its work in this The borrower government officials in- field. Although the Bank has been supporting terviewed for this review were convinced such projects for more than a quarter-century, it that a participatory approach can con- has no database that has tracked these projects tribute to poverty alleviation in their coun- since their inception. It is only recently (for proj- tries. However, they did not necessarily believe ects approved from fiscal year 2000 onward) that that allowing community control over invest- the Bank has started maintaining a database. ment decisions and resources in a Bank sub- Despite a rapid increase in lending, most project is the best means of engaging Bank-supported CBD/CDD projects have communities. This appears to be partly because not yet been subjected to rigorous evalua- government officials have concerns about the ca- tion.4 The share of projects in the Bank's port- pacity of communities to manage resources, but folio that include a CBD/CDD component grew also partly because they feel threatened by de- from about 2 percent in fiscal 1989 to 25 percent volution of complete control over decisions and in 2003. However, as yet there is very little hard resources to communities. evidence on the impact of these projects in re- The Bank has not systematically and ducing poverty or enhancing community capac- realistically assessed the costs and benefits ity. This is because of: (i) the Bank's failure to of undertaking CBD/CDD projects to the include baseline surveys in most of the projects institution, the borrower, or the commu- completed thus far and (ii) inadequacies in the nities.5 CBD/CDD projects are more expensive design of the Bank's project monitoring and eval- than non-CBD/CDD projects for the Bank to uation systems to systematically assess changes prepare and supervise, and there are substantial in community capacity resulting from the Bank costs in time spent by the borrower in putting intervention. Some recent CBD/CDD projects a participatory approach in place. While are setting up baselines, but the results will not CBD/CDD projects have helped lower the cost be available for several years. To fill these gaps, to governments for delivering service delivery in- in the absence of baseline data, this evaluation frastructure, the communities now bear an in- used a pragmatic approach based on household creased part of the cost of that infrastructure. The surveys to assess the association between insufficient focus on costs and benefits in CBD/CDD projects and change in communities' CBD/CDD projects, especially on measures of capacity and drew on a variety of sources for in- poverty impact, has prevented convincing com- sights into the projects' poverty impact. parisons with non-CBD/CDD investments and policy and institutional reform programs. Outcome Much more success has been achieved Outcome ratings of Bank-supported in CBD/CDD projects on quantitative goals, CBD/CDD projects were, on average, bet- such as the construction of infrastructure, ter than those for non-CBD/CDD projects than on qualitative goals, such as capacity between 1994 and 2003. The education sec- enhancement or quality of training. How- tor has had the highest percentage of projects ever, without baseline data, and without ap- rated satisfactory on outcome, followed by proj- propriate indicators, it is often assumed that ects in the transport, urban development, and so- meeting the quantitative goals will automati- cial protection sectors. The rural development cally fulfill the qualitative goal--for example, sector, with the largest CBD/CDD portfolio, is a holding a certain number of training courses is below-average performer on outcome in the ag- expected to enhance capacity. That said, since gregate, as are projects under the water supply, CBD/CDD projects have supported construc- health, and environment sectors. Between 1999 tion of infrastructure in scattered communities, and 2003, the outcome rating for CBD/CDD proj- they have also increased access to infrastruc- ects in post-conflict countries was better than the ture for schools, health centers, and the like for outcome rating for CBD/CDD projects in non- the communities in which they intervene. Sev- conflict countries. eral of the CBD/CDD projects in conflict and x i i i THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT post-conflict countries have met quantitative decentralization under various CBD/CDD proj- targets for infrastructure rehabilitation and have ects has not always been consistent. In other also provided substantial employment benefits cases, the ad hoc parallel arrangements set up to the local population. However, as with other to implement Bank projects have hindered the Bank projects, increased access to infrastruc- long-run enhancement of local government ca- ture does not always translate into effective pacity. service delivery because of the inadequacy of complementary inputs such as teachers, doctors, The Bank's Operational Policy and medicines. Requirements and Processes Bank-supported CBD/CDD projects have It is difficult to ensure fiduciary and safe- had much more success with community guard compliance in CBD/CDD projects. capacity enhancement when they have sup- It is easier for the Bank to monitor resource use ported indigenously matured participa- and comply with safeguards in non-CBD/CDD in- tory efforts or when they have provided vestments, such as large bridges or a power sustained, long-term support to commu- plant, than where small subprojects are being im- nities. The one year of a typical subproject cycle plemented by hundreds of remote communi- is generally too short a time to bring about the ties in scattered locations. Over time, adaptation kind of enhancement of community capacity of operational policies and decentralization of the that is envisioned in Bank-supported CBD/CDD, Bank to field offices have enhanced the Bank's particularly CDD projects; it is sufficient to allow capacity to implement CBD/CDD projects, but successful subproject execution, but not to con- additional changes are needed. Among the issues sistently enhance community capacity. that need to be addressed are: weaknesses in monitoring and evaluation (particularly with re- Sustainability gard to monitoring progress on community ca- Sustainability ratings have improved over pacity enhancement), development of adequate time for both CBD/CDD and non-CBD/CDD guidelines for staff on safeguards for CDD proj- projects, although several concerns re- ects, and the short time span of the Bank's sub- main. Infrastructure and other activities sup- project cycle. ported by the Bank's CBD/CDD projects have In the end, the Bank should be aware that it been difficult to sustain beyond the Bank pres- is largely trying to use a single financial channel-- ence because of a lack of the needed resources project financing--to bring about changes in from the government and communities to ensure empowerment and social capital, which are af- their operation and maintenance. More broadly, fected by a long history of social, cultural, and Bank projects have often failed to provide con- political forces embedded in the societies in sistent long-term support for an activity to be- which the Bank is trying to support develop- come sustainable (for example, in a forestry ment. Sometimes, the Bank support works well, project, support should be provided until the for- especially when it supports existing grass roots est starts yielding adequate returns from tim- initiatives. But when the Bank tries to initiate em- ber and non-timber products). powerment and enhance social capital through CBD/CDD projects, it is often not enough--or Institutional Development Impact can even be counterproductive if the better-off Bank-supported CBD/CDD projects have sections of the community gain more than the enhanced the capacity of government less-well-off. Where the Bank supports the cre- institutions to implement participatory ation of "temporary" arrangements for the im- interventions, but so far few borrower gov- plementation of CBD/CDD projects at the local ernments appear to have adopted the ap- level, they could further undermine long-term proach more widely in their own capacity building efforts and should be carefully development programs. At the country level considered, especially if financing is provided for the Bank's approach to promoting government only a few years. x i v E X E C U T I V E S U M M A R Y Recommendations assessment of ongoing CBD/CDD projects Given the mixed and limited evidence on the im- to ensure relevance and effectiveness of the pacts of CBD/CDD projects--particularly in terms program to the country context. of poverty reduction and empowerment--and · Future CASs should show how they have ana- questions about sustainability and safeguard and lyzed and addressed linkages, not only among fiduciary compliance, the Bank should approach various CBD/CDD projects to be undertaken future CBD/CDD projects, particularly CDD, in the country, but also among CBD/CDD and with greater care. In countries where the Bank relevant non-CBD/CDD projects. In particu- is already supporting a CDD program, the insti- lar, the analysis should address whether tution needs to rigorously assess the poverty arrangements for CBD/CDD project imple- and institutional development impact of its proj- mentation come at the expense of local gov- ects before scaling them up. A cautious approach ernment capacity development. would be especially important in countries or areas where the Bank is just beginning to sup- At the project level, the Bank should give port CDD. In its future assistance to CBD/CDD, priority to helping countries build up ex- the Bank should: isting indigenously matured initiatives; where there are no such existing initia- At the corporate level, strengthen opera- tives, the Bank should tailor its project to tional guidance and management over- the country and community context, while sight. undertaking selective, rigorous impact as- · The Bank should provide operational guid- sessments to ensure learning. ance for the application of Bank safeguard · For any new CBD/CDD project, the Bank policies and fiduciary oversight of CBD/CDD should analyze (using existing processes, such projects and for the strengthening of cost-ben- as social assessments) whether it is building on efit analysis and M&E systems and should com- indigenously matured initiatives or attempting mission an audit of the fiduciary aspects of a to begin a CDD program in a country, and representative sample of CDD projects for sub- then tailor the intervention to local capacity; mission to the Board within a year. and the Bank should also selectively undertake rigorous impact assessments upon comple- At the country level, design the CBD/CDD tion of its ongoing CBD/CDD projects to learn program as an integral part of the overall for the future. assistance strategy and carry out periodic x v ACRONYMS AND ABBREVIATIONS AFR Africa Region AIDS Acquired immunodeficiency syndrome APL Adaptable Program Lending ASIP Agriculture Sector Investment Project (Ghana) CA Community association CAS Country Assistance Strategy CAE Country Assistance Evaluation CBD Community-based development CDD Community-driven development CBRIP Community-Based Rural Infrastructure Project (Vietnam) CFAA Country Financial Accountability Assessment Report CIDSS Comprehensive and Integrated Delivery of Social Services (Philippines) CPAR Country Procurement Assessment Report DFID Department for International Development (U.K.) DOLIDAR Department of Local Infrastructure Development and Agricultural Roads (Nepal) EA Environmental assessment EAP East Asia and the Pacific Region ECA Europe and Central Asia Region ERR Economic rate of return ESSD Environmentally and Socially Sustainable Network (World Bank) FI Financial intermediary FUMAC Fundo Municipal de Apoio Comunitario (municipal-community scheme, Brazil) FUMAC-P Conselho Municipal de Apoio Comunitario ­ Piloto (Pilot Municipal Community Schemes, Brazil) FY Fiscal year HIV Human immunodeficiency virus HNP Health, Nutrition, and Population (sector) IAD Internal Auditing Department ICR Implementation Completion Report ID Institutional development IDB Inter-American Development Bank IFAD International Fund for Agricultural Development JFM Joint forest management KDP Kecamatan Development Project (Indonesia) LAC Latin America and the Caribbean Region LIL Learning and Innovation Loan M&E Monitoring & evaluation MAP Multi-Country HIV/AIDS Program MDG Millennium Development Goal x v i i THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT MIS Monitoring and information system MNA Middle East and North Africa Region NGO Nongovernmental organization NMPRP Northern Mountains Poverty Reduction Project (Vietnam) O&M Operation and maintenance OED Operations Evaluation Department OD Operational Directive OP Operational Policy OPCS Operational Policy and Country Services PAC Programa de Apoio Comunitario (state-community schemes) PAD Project Appraisal Document PPAR Project Performance Assessment Report PRSP Poverty Reduction Strategy Paper PSR Project Status Report QAG Quality Assurance Group QSA Quality of Supervision Assessment RPAP Rural Poverty Alleviation Program (Brazil) RPRP Rural Poverty Reduction Project SAR South Asia Region SEWA Self-Employed Women's Association VLPA Village-Level Participatory Approach ZAMSIF Zambia Social Investment Fund x v i i i 1 Origin, Scope, Design, and Methodology of the Evaluation articipatory approaches that involve communities in their own devel- P opment have gained substantial support among international donors over the past quarter-century and have become increasingly important in the work of the World Bank.1 Community participation is an approach to development that can be used with any Bank lending instrument and across sectors. Projects can involve communities in different ways--by sharing information, consulting, collaborating, or empowering them.2 The World Bank's support for these participatory poverty-reducing effects of local initiatives that approaches has been largely manifested in the developed independently in several countries. design and implementation of either community- In these islands of success, local communities based development (CBD) or community-driven had taken control of their lives as a result of in- development (CDD) projects. Although the lit- dependent forms of social action. In its ongoing erature makes no clear distinction between the effort to reduce poverty, the Bank has emulated CBD and CDD approaches, there is increasing these local initiatives by attempting to enhance consensus that CDD projects further the Bank's community capacity by building social capital and support for empowerment by, inter alia, giving fostering empowerment in communities through communities control over resources and deci- its projects. Research into the multidimensional sions in the context of a Bank project. These proj- nature of poverty has further reinforced the im- ects are distinguished from CBD, where the portance of empower- The Bank's Strategic communities have less responsibility. Instead, ment. Today, the Bank's the emphasis is on collaboration, consultation, Strategic Framework Framework identifies and sharing information with the communities identifies empowering empowering poor about the project. Since the late 1990s, the focus poor people to partici- people to participate in of Bank-supported CBD/CDD projects has shifted pate in development by toward CDD, though many CDD projects also investing in them as one development by investing include CBD components. of the two basic priori- in them as one of the two Interest in community empowerment ties in the fight against basic priorities in the emerged in large part because donors, includ- poverty (World Bank ing the World Bank, were impressed by the 2001b). fight against poverty. 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT the Bank has been giving more emphasis to CDD Figure 1.1: The Universe of CBD/CDD Projects in recent years, a large percentage of these proj- ects continue to include both CBD and CDD components, which makes it impossible to carry out an evaluation of CDD projects only.3 Social funds, in which funds are channeled through an autonomous agency, are also a subset of CBD/CDD approaches. Figure 1.1 illustrates the relationship Community-driven between CBD, CDD, and social funds. development Community-based (CDD) Because CBD/CDD projects can involve a com- development (CBD) munity in different ways, depending on the na- ture of interaction required with that community, Social funds there is no typical CBD/CDD project, nor can a single causal chain--from inputs to outputs, out- comes, and impacts--be identified for all these Source: Study research. interventions. However, a causal chain can be identified for CDD projects. (See figure B.1 in Interest in a community-based approach is Annex B.) Support for a CDD project typically in- also predicated on a feeling that not only would cludes: (a) strengthening community groups it lead to better allocation of resources to help with training support and providing them with an communities, but also to reduced corruption opportunity to control decisions and resources and less misuse of resources, and thereby more in a project context for building small infra- development assistance would reach the poor. structure and (b) creating an enabling environ- Community involvement would increase trans- ment for these activities through appropriate parency and accountability by working directly policy and institutional reform. The underlying with the ultimate beneficiaries, especially where hypothesis in this approach is that such com- state capacity is weak or has been weakened by munity control in the preparation and imple- conflict and other factors. mentation of their donor-supported development While Bank lending plans will make interventions more suited to Bank lending for CBD for CBD and CDD ap- local needs and enhance community capacity and CDD approaches has proaches has increased for self-development. This will allow the com- significantly over the munity to hold accountable the institutions that increased significantly, years, recent reports by affect their lives. More recently, emphasis is also but there is a lack of the Bank's research and being given to decentralization reform and pro- rigorous evaluative social development de- moting partnerships between local government partments (Mansuri and institutions and communities. Strengthened local evidence on the develop- Rao 2004; Wassenich and government institutions are also expected to ment effectiveness of Whiteside 2004) note the build an enabling environment for CDD. these approaches. lack of rigorous evalua- tive evidence on the de- Design velopment effectiveness of these approaches. The Bank has no explicit benchmark, such as a policy or strategy paper, against which the design Scope and performance of CBD/CDD approaches in As noted in the Approach Paper approved by the general, and CDD projects in particular, can be Bank's Committee on Development Effectiveness evaluated.4 The CDD chapter of the PRSP Source- (CODE) in July 2003, the goal of this evaluation book (World Bank 2003b) and a Web site man- is to assess the development effectiveness of the aged by the CDD Anchor are the principal Bank's CBD/CDD projects, not that of participa- sources of guidance for Bank staff on CDD tory development projects in general. Even though (Annex C). 2 ORIGIN, SCOPE, DESIGN, AND METHODOLOGY OF THE EVALUATION Box 1.1: Primary Data Collection for the Evaluation of CBD/CDD Projects Two kinds of primary data provide perspectives from a range of compared randomly selected CBD/CDD communities with com- stakeholders on the appropriateness of the CBD/CDD approaches parator communities in the four projects (Annex M). to development: from the field and from Bank staff. Primary data were collected because (i) "projects lack careful evaluations with Comparators for the Fieldwork: The selected comparators var- good treatment and control groups and baseline and follow up ied according to project and country context. In two project data" (Mansuri and Rao 2004); and (ii) though most Bank areas (Benin and Brazil), the comparison group communities CBD/CDD projects are expected to contribute to community ca- had benefited from similar subprojects as the CBD/CDD com- pacity enhancement, project monitoring and evaluation sys- munities, but through a non-participatory approach adopted ei- tems as designed are unable to systematically assess changes ther by the government or by a religious organization. These two in that capacity because of the Bank intervention. Chapter 3 (box cases allowed the evaluation to assess whether a program that 3.1) explains how this evaluation assessed the capacity-build- involves communities is more effective than one that does not. ing impact of Bank CBD/CDD interventions. In another project area (Madhya Pradesh) the comparison-group communities benefited from a similar activity carried out through From the Field: Primary data were collected in the five case study a participatory approach, but supported by the government. Here countries and in India to assess project outcomes. All five coun- the evaluation assessed whether there is any difference in out- try studies included interviews or surveys with central or state comes because of the participatory approach pursued by the Bank government officials and bilateral and multilateral donor repre- compared with the participatory approach pursued by the gov- sentatives. Focus groups or interviews were also held with local ernment. Finally, in the fourth project area (Uttar Pradesh), com- and international nongovernmental organizations (NGOs) in parison communities did not benefit from a similar activity as each of the countries. Local government officials were inter- project communities. Here the evaluation assessed the overall viewed in the State of Rio Grande do Norte in Brazil and the Bor- outcomes of the Bank CBD/CDD project, not only that of its par- gou Region of Benin.a Extensive fieldwork was carried out in the ticipatory approach. respective areas of Benin and Brazil and in the context of two The Egypt, Nepal, and Vietnam studies also involved a limited OED project assessments (Uttar Pradesh Sodic Land Reclama- number of focus group sessions with communities. tion Project and Madhya Pradesh Forestry Project) in India. These four project areas also involved focus group sessions and Within the Bank: An electronic survey was administered to 400 key informant interviews. In the absence of baseline data, OED's Bank staff and managers familiar with CBD/CDD issues. A response fieldwork adopted a non-experimental evaluation design that rate of 38 percent resulted in 152 completed surveys (Annex L). Uttar Madhya Benin Brazil Pradesh Pradesh Number of household surveys 1,376 1,097 1,197 1,147 Number of focus groups with community members 32 56 60 58 Number of local leader interviews 32 33 29 30 Number of interviews with committee/community association members 32 28 30 30 Number of interviews with municipal council members 32 a. Henceforth, references to primary data collected in Benin and Brazil refer only to the areas studied. In assessing the relevance, efficacy, efficiency, in- 1. Are Bank-supported CBD/CDD projects relevant to stitutional development impact, and sustainability of the achievement of the Bank's poverty reduction mis- the Bank's CBD/CDD projects, the evaluation addressed sion and borrower and community priorities? (Rele- six questions: vance) 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 2. To what extent have CBD/CDD projects met economic and sector work, as well as 73 Coun- their objectives? (Efficacy) try Assistance Strategies (CASs), 29 Poverty 3. How efficient have CBD/CDD projects been Reduction Strategy Papers (PRSPs), and poverty relative to alternatives? (Efficiency) sector work for a large number of countries cov- 4. To what extent have these projects enhanced ered by the sample. the capacity of the communities and of cen- · Five country case studies--Benin, Brazil, Egypt, tral/state and local government institutions? Nepal, and Vietnam--involved desk reviews (Institutional Enhancement) of the literature and project documents plus vis- 5. What are the challenges for ensuring sustain- its to the countries. Two of the country stud- ability of the benefits from a CBD/CDD proj- ies included extensive fieldwork (box 1.1). ect? (Sustainability) · A Bank Staff Survey and interviews. 6. Do internal policies and processes position the · Two thematic studies, one a review of portfo- Bank to adequately support implementation lio projects for their compliance with safe- of CBD/CDD approaches (and CDD projects guard policies, the other an assessment of how in particular)? (Bank Policy Requirements) Bank capacity to undertake CBD/CDD, partic- ularly CDD, projects in client countries has Methodology evolved over time. The greatest challenge this evaluation faced was · A Literature Review. in identifying the portfolio of CBD/CDD projects · Nineteen project assessments, with extensive to be reviewed.5 Though fieldwork in the context of two (box 1.1). The Bank has no explicit the Bank has been sup- · A desk review of documents for six (one in each benchmark against porting CBD/CDD ap- Region) ongoing CDD projects. proaches for more than a · OED reviews of 33 Implementation Completion which the design and quarter-century, the in- Reports. performance of CBD/CDD stitution has no database · Past OED studies. approaches in general, that has tracked these projects since their in- An external Advisory Panel of three experts and CDD projects in ception. It is only very re- provided guidance to the evaluation. particular, can be cently that the CDD Since OED's rating methodology is objective- evaluated. Anchor in the Social De- based, the Portfolio Review was able to compare velopment Department the ratings for outcome, institutional develop- has started maintaining a database. Records are ment impact, and sustainability of CBD/CDD proj- being maintained on the basis of self-reporting ects with those of non-CBD/CDD projects. For the from the Regions, but only for projects approved projects covered by household surveys, country- from fiscal year 2000 forward. Hence, OED had specific comparators were selected as described to develop a methodology for identifying the in box 1.1 and Annex M. The compliance of the universe of projects approved during fiscal years CBD/CDD portfolio with the Bank's fiduciary and 1989­2003 (see Chapter 2). safeguard policies was examined to assess whether Both qualitative and quantitative evaluation internal policies and processes position the Bank tools were used to address the six evaluation to adequately support implementation of questions (Annex D). The study drew on the CBD/CDD projects, with particular attention paid following: to the cumulative impact of small subprojects and community control over resources in CDD. · A Portfolio Review included a desk review of Results of this analysis were assessed against the project documents for a proportionate and Bank's benchmark of full compliance. representative random sample of 84 projects, stratified by time and sector board, drawn from Some Limitations of the Study Design the universe of 847 projects identified by OED. Because few completed CBD/CDD projects had The Portfolio Review also reviewed relevant commissioned baseline surveys at the outset 4 ORIGIN, SCOPE, DESIGN, AND METHODOLOGY OF THE EVALUATION and project monitoring and evaluation systems ing impact of these proj- Both qualitative and are not systematically designed to assess changes ects and provide useful quantitative evaluation in community capacity, OED had to adopt a insights until sufficient pragmatic methodology based on a non-exper- baseline data are com- tools were used to imental evaluation design to assess the possible piled under Bank proj- address the evaluation impact of CBD/CDD projects on changes in ects to permit more questions. community capacity (boxes 1.1 and 3.1).6 Such comprehensive and rig- a methodology has its limitations, because there orous (longitudinal) im- are several complexities in identifying com- pact evaluations. The parators, as highlighted in Annex M, and the evidence from the fieldwork and the desk review variables for measuring change in social capital of Bank-supported participatory development and empowerment, as highlighted in Annex N. approaches was also corroborated with evi- Nevertheless, the findings of the fieldwork are dence from the literature (both Bank and non- suggestive of the community capacity­enhanc- Bank). 5 2 The CBD/CDD Portfolio A s noted in Chapter 1, the Bank has only recently developed a system for tracking its CBD/CDD projects.1 Because the Anchor's database was incomplete and could not be directly used for the evaluation's purposes, OED developed a methodology to identify all CBD/CDD projects approved between 1989 and 2003. A universe of 847 projects was identified (Annex overall CBD/CDD projects have grown at about E) this way, and a proportionate and represen- 14 percent per year, whereas CDD projects have tative random sample of 84 projects, stratified by grown at about 19 percent per year. The analy- time and sector board, was drawn for intensive sis also found that "pure" CDD projects form review.2 OED then categorized the sample of 84 about 23 percent of the CBD/CDD portfolio. projects into CBD, CDD, and mixed CDD/CBD The Latin America and Caribbean Region had interventions. Annex F explains how the largely the highest, and the Middle East and North Africa CDD interventions were separated from the Region the lowest, number of CDD projects CBD/CDD ones. (Annex F). Further, more than 75 percent of the CBD/CDD portfolio consists of projects for which Temporal, Regional, and Sectoral the majority of investment funding is for a large Distribution of the Portfolio number of small and scattered subprojects. The total number of Bank projects that include a CBD/CDD component has increased substan- Ways That CBD/CDD Projects Differ from tially over time (figure 2.1a). The largest per- Those in the Non- centage of these projects is in the Sub-Saharan CBD/CDD Portfolio The total number of Bank Africa Region (AFR), with the Latin America and projects that include a Caribbean Region (LAC) in second place (fig- A Greater Focus on CBD/CDD component has ure 2.1b). Among sectors, the rural sector has the Learning by Doing increased substantially. largest percentage of CBD/CDD projects (27 and Multisectorality percent), with health, nutrition, and population The literature on par- The largest percentage in second place (16 percent) and social protec- ticipatory development of these projects is in tion close behind (15 percent) (Annex E). highlights the impor- the Sub-Saharan Africa Analysis of the random sample of 84 projects tance of a learning-by- (Annex F) found that over the period 1989­2003, doing, flexible approach, Region. 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure 2.1a: Bank Commitments to Projects with CBD/CDD Approaches Have Grown 6,000 100 90 5,000 80 70 US$ 4,000 projects 60 of million 3,000 50 2001 40 Number In 2,000 30 20 1,000 10 0 0 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 Commitment Projects Source: World Bank database. a concept developed from the learning process as Adaptable Program Loans (APLs) and Learn- ideas of Korten (1980). Flexibility is being ing and Innovation Loans (LILs); by allowing brought to Bank CBD/CDD projects through communities greater choice in the selection of greater use of flexible lending instruments such activities, as in a multisectoral project; and by providing them the opportunity to control in- vestment decisions and resources during proj- ect implementation in the case of CDD. Other Figure 2.1b: Africa Has the Largest Share of Projects issues that are relevant for flexibility are dis- with CBD/CDD Approaches cussed in Chapter 5. The percentage of APLs and LILs in the CBD/CDD universe is more than double that in SAR the non-CBD/CDD universe (32 percent against 11% MNA AFR 13 percent for the period 1999­2003). Over time, 7% 31% research into the multidimensional nature of poverty has led to an increase in multisectoral proj- ects in the Bank's portfolio, and the percentage of these projects in the CBD/CDD universe is LAC much higher than that in the non-CBD/CDD uni- 23% verse (51 percent versus 31 percent between fis- EAP cal years 1989 and 2003; see Annex E, table E.1). 17% ECA Analysis of the sample of 84 CBD/CDD projects 11% also revealed that the CDD portfolio has a higher percentage of multisectoral projects (74 percent) Source: World Bank database. than the overall CBD/CDD portfolio (48 percent). Note: MNA = Middle East & North Africa; SAR = South Asia; AFR = Africa; However, a large number of CDD projects con- EAP = East Asia & the Pacific; ECA = Europe and Central Asia; LAC = Latin America & the Caribbean. tinue to be sectoral interventions. 8 THE CBD/CDD PORTFOLIO A Greater Focus on Giving More CBD/CDD Projects Projects have sought to Responsibility to Communities Have Been Used achieve their objectives CBD/CDD projects, particularly CDD, empha- to Promote a Variety size process issues more than the non-CBD/CDD of Objectives through (i) material projects, because they attempt to enhance social CBD/CDD projects have development and capital and further the Bank's support for com- sought to achieve several (ii) capacity-building munity empowerment. Within the CBD/CDD kinds of objectives-- support. portfolio, the CDD projects differ from the oth- poverty reduction, decen- ers in that they are designed to put greater re- tralization, employment sponsibility on communities for each phase of the generation, basic infrastructure development, ac- subproject cycle (figure 2.2). Although there is cess to health care and education, nutrition, nat- little discussion in Bank documents (at appraisal, ural resource management, private sector supervision, or completion) about the partici- development, urban upgrading, mitigation of the patory process itself,3 the importance given to socioeconomic impact of HIV/AIDS, and eco- empowerment in the Bank's Strategic Frame- nomic recovery. It is not uncommon to have sev- work, the evidence from the Egypt and Nepal eral objectives combined in a single project, country studies, and the Portfolio Review indi- without a clear hierarchy. However, more recent cate that the projects involving community par- projects generally have fewer objectives than ticipation have evolved toward allowing for a those approved in the earlier part of the period. community role in decision making, rather than The Portfolio Review found that most proj- merely "informing" them of decisions. (See fig- ects have sought to achieve their objectives ure 3.3 and Annex H.) through two broad kinds of activities: (i) ma- Figure 2.2: Communities Have More Responsibility in CDD Projects 100 75 projects of 50 Percentage 25 0 Responsible for Responsible for Responsible for Responsible for subproject design subproject implementation subproject M&E subproject O&M Community responsibility in subproject cycle CBD/CDD CDD Source: Review of 84 project appraisal documents. Note: M&E = monitoring and evaluation, O&M = operation and maintenance. 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT terial development, such as infrastructure con- CBD/CDD is Being Used in New Activities struction and (ii) and capacity-building support Such as AIDS and Post-Conflict Work for government (central, provincial, and local), The recent Africa Multi-Country HIV/AIDS Pro- communities, and other stakeholders, such as jects, aimed primarily at assisting national gov- nongovernmental organizations (NGOs). ernments in their strategies to cope with the Though the word "empower" is present in the disease, are also considered community-driven. stated objectives of only 2 of the 84 sample These projects are the subject of another inde- projects, other projects de facto seek to em- pendent evaluation in OED. Community-driven power communities by building their capac- reconstruction has also been used recently as an ity--for example, the Nepal Community School approach in the transition from war to peace. The Support Project (2003) proposes to provide portfolio of 847 projects includes 198 in conflict technical assistance and other financial support and post-conflict countries, several of which are to build the capacity of communities to take designed to provide speedy delivery of recon- over the management of government-funded struction assistance and support for infrastruc- schools. ture rehabilitation. 1 0 3 Outcome of Bank-Supported CBD/CDD Projects T his chapter first reviews the overall outcome ratings of CBD/CDD projects before going on to examine their relevance, efficacy, and effi- ciency. 1994­98 (figure 3.2a). The education sector has Outcome Ratings had the highest percentage of projects rated satisfactory on outcome, followed by projects in Outcome Ratings of CBD/CDD Projects the transport, urban development, and social Have Been Better than Those of protection sectors. The rural development sec- Non-CBD/CDD Projects tor, with the largest CBD/CDD portfolio, is a The outcome ratings (investment lending only) below-average performer on outcome in the ag- of closed CBD/CDD projects (when compared by gregate, as are projects under the water supply, number of projects), on average, have been bet- health, and environment sectors (figure 3.2b).1 ter than those of non-CBD/CDD projects over the OED's data also shows that between 1999 and period covered by the evaluation, although the 2003 the outcome rating for CBD/CDD projects gap has narrowed over time (figure 3.1). Out- in post-conflict countries was 4 percentage points come ratings of CBD/CDD projects are also bet- higher than the outcome rating for CBD/CDD ter when disbursement-weighted, with a few projects in non-conflict countries (Annex G, large projects--mostly concentrated in middle- table G.2). income countries--doing much better than a large number of smaller projects. Relevance The Latin America and Caribbean Region, with the second-largest portfolio (Chapter 2), has Increasing Importance Is Being Given to a significantly higher percentage of CBD/CDD CBD/CDD, Particularly CDD, in Recent CASs projects rated satisfactory on outcome than all A review of 62 CAS doc- The CBD/CDD approach is other Regions. Although the Africa, Middle East uments (two each from and North Africa, and Europe and Central Asia 31 countries covered by considered an important Regions have a lower percentage of CBD/CDD the sample of 84 proj- element of the Bank's projects rated satisfactory on outcome, these ects) found that the strategy in over 74 per- Regions also show an increase in satisfactory CBD/CDD approach is outcome ratings in the period 1999­2003 over considered an important cent of countries. 1 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure 3.1: Outcome Ratings of CBD/CDD Projects Have Been Better Than Those of Non-CBD/CDD Projects 100 80 74% 73% 72% 66% (%) 60 40 Satisfactory 20 0 1994­98 1999­2003 Exit fiscal year CBD/CDD Non-CBD/CDD Source: World Bank database. Note: OED ratings are based on OED reviews of Implementation Completion Reports (ICRs), 25 percent of which are subsequently revisited through OED field assessments. The outcome ratings of the closed investment projects reveal insignificant differences between CBD/CDD and non-CBD/CDD projects in the two time periods con- sidered (1994­98 and 1999­2003). The differences between CBD/CDD and non-CBD/CDD projects were also insignificant for each exit year between fiscal years 1999 and 2003. element of the Bank's strategy in over 74 percent participatory approaches is because these coun- of countries, and that the emphasis given to tries have a relatively long history of community greater community participation in decision participation in their own development. Another making and resource allocation in Bank inter- reason why governments are convinced of the ventions has increased over time (See Annex H benefits of the approach is that in an age of and figure 3.3). shrinking budgets, it allows them to do more with less. For example, the OED India country study Borrower Government Officials Interviewed on forestry (Kumar and others 2000) reported Were Convinced that a Participatory that the forest department staff valued commu- Approach Is Beneficial nity participation in forest protection. Before Surveys of government officials in the 5 case Joint Forest Management (JFM), the forest de- study countries, past OED studies, and a review partment was fighting a losing battle, and one for- of 29 PRSPs (Annex H) indicate that borrower est guard could not effectively patrol the large government officials are convinced that a par- area under his control without the participation ticipatory approach can contribute to poverty re- of the communities. duction in their countries. For example, in Vietnam, 86 percent of central government of- But They Didn't Necessarily Believe in Giving ficials interviewed reported that projects with Communities Control over Decisions and Resources beneficiary participation can address poverty re- Surveys of government officials in Benin, Nepal, duction better than those without (Annex I). and Vietnam also indicated that they did not One reason that governments in countries such necessarily believe that allowing communities as Nepal and Vietnam have accepted and adopted control over investment decisions and resources 1 2 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS Figure 3.2a: CBD/CDD Projects in the Latin America and Caribbean Region Have Done Better than Projects in Other Regions 100 80 (%) 60 40 Satisfactory 20 0 AFR EAP ECA LAC MNA SAR Region 1994­98 1999­2003 Source: World Bank database. Note: OED ratings are based on OED reviews of ICRs, 25 percent of which are subsequently revisited through OED field assessments. AFR = Africa; EAP = East Asia and the Pacific; ECA = Europe and Central Asia; LAC = Latin America & the Caribbean; MNA = Middle East & North Africa; SAR = South Asia. Figure 3.2b: CBD/CDD Projects in Education and Social Protection Do Better than Projects in Health and Rural Development 100 80 (%) 60 40 Satisfactory 20 0 Rural Social Health Education Other* development protection 1994­98 1999­2003 Source: World Bank database. *Urban, Environment, Water Supply & Sanitation, Transport. Note: OED ratings are based on OED reviews of ICRs, 25 percent of which are subsequently revisited through OED field assessments. 1 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure 3.3: CBD/CDD Is Increasingly Important percent said that more than 75 percent of the com- in Country Programs munities had the ability to identify their needs and prioritize them (Annex I). None of the local gov- ernment officials in Benin said that the commu- 0% 100 nities had the ability to manage and mobilize 21% external or internal resources. Even in Brazil, a middle-income country that has had a CDD pro- 75 gram for a decade, while over two-thirds of the 66% municipal government officials interviewed in CASs of the state of Rio Grande do Norte said that most 50 59% communities are capable of identifying and pri- oritizing their needs, the majority stated that most Percentage 7% communities are not capable of preparing de- 25 velopment plans, implementing and maintaining 7% community projects, or mobilizing resources ei- 28% 14% ther within or from outside the community. Half 0 of the municipal government officials interviewed Previous CAS Latest CAS also said that most communities are not capable Community participation in Information and coming of managing financial resources.2 decision making closer to communities and resource allocation, especially of the poor Nor Did They Have the Same Understanding Partnership (community role Top-down approach as the Bank about How Community Empowerment in decision making and (no participation in the Is To Be Promoted planning responsibilities) formal decision making) OED's field research, desk review of project doc- Source: Review of CASs (Annex H). uments, and interviews with Bank staff found that (i) within the Bank, there is a mismatch between in a Bank project context was the best means of the understanding of empowerment and the pri- engaging them (Annex I). Over 90 percent of the mary means of promoting it; and (ii) between PRSPs reviewed are also silent on such commu- the Bank and its clients, there is a consequent dif- nity control (Annex H). Central government of- ference in the understanding of how to promote ficials in Benin were asked what form of empowerment. The Bank's Web site defines em- participation is effective and efficient for devel- powerment as the process of increasing the ca- opment approaches; nearly 80 percent of the re- pacity of individuals or groups to make choices spondents said that it is where the community and to transform those choices into desired ac- is informed and consulted on the government tions and outcomes (box 3.1). However, the development plan for the area. The local gov- Bank's primary channel for promoting community ernment surveys in that country also suggest empowerment is through design and imple- that local officials feel threatened by devolution mentation of CDD projects that have a definite yet of complete control over decisions and resources narrower interpretation of how empowerment is to communities (Annex I). to be brought about in a community--that is, Central and local gov- through giving communities control over deci- There appears to ernment officials sur- sions and resources in a Bank project context. In- be a difference in veyed in case study terviews with borrower government officials in countries also appear to case study countries found that they see activities understanding between be unconvinced of the that help build the capacity of the communities the Bank and its clients ability of communities to to participate in the development process as em- on how to promote handle resources. For ex- powering activities. In Uttar Pradesh, the imple- ample, among the Viet- menting agency whose operational capacity was empowerment. namese officials, only 21 considerably enhanced with support from the 1 4 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS Sodic Land Reclamation Project did not see "em- Priority Needs Are More Likely to Be powering" communities as building toward a Addressed When Communities Are Given strategy of putting them in control. The OED as- Multiple Options from which to Choose sessment of the project notes: Household survey data collected by OED in Benin and Brazil show that where communities From the implementing agency perspec- were given a wide menu to choose from, Bank tive, the most important aspect of com- projects were more likely to meet one of the munity participation was ensuring farmer top-priority needs of the communities. The commitment to the reclamation process Madhya Pradesh Forestry and Uttar Pradesh and sharing of reclamation costs. Hence, Sodic Land Reclamation projects in India, by for them, even if they `directed' and `con- contrast, did not meet the priority needs of the trolled' the discussion in the village-level im- communities (figure 3.4). These sectoral plementing bodies, it was not a concern, as interventions were not designed to give long as farmers participated and con- communities a choice of activities. tributed and the reclamation was under- taken as per the technical standards. Efficacy The way the government officials view em- So Far, More Success Has Been Achieved powerment is in harmony with the definition of on Quantitative than on Qualitative Goals empowerment displayed on the Bank's Web site, Material development and capacity building ac- and not with the way it is interpreted in the im- tivities (see p. 9) have both quantitative and plementation of the CDD projects. The need qualitative dimensions. CBD/CDD interventions for clarity on how empowerment is to be pro- have generally been successful in achieving quan- moted by the Bank becomes even more urgent titative goals such as infrastructure built, em- when seen in the context of the discussion of the ployment generated, and number of training issue in the literature.3 courses held. Since these interventions are sup- Box 3.1: Change in Social Capital and Empowerment as a Means of Assessing the Community Capacity­Enhancing Impact of Bank CBD/CDD Interventions This evaluation assessed changes in community capacity to sociational life, participation in traditional events and in non-tra- undertake development through respondents' perception of ditional/political events, and circle of friends. changes in social capital and empowerment (as defined below) The Bank's Web site defines empowerment as the process of in CBD/CDD and comparator communities as captured through increasing the capacity of individuals or groups to make choices household surveys. The capacity assessed did not include and to transform those choices into desired actions and out- changes in a community's technical capacity. comes. The Sourcebook on Empowerment and Poverty Reduc- Social capital refers to the norms and networks that enable col- tion (World Bank 2002b) identifies four key elements of successful lective activity in a community. By drawing people in a community empowerment approaches: access to information, inclusion/par- together to collectively decide and manage project activities and ticipation of poor people, accountability, and local organizational outputs, Bank CBD/CDD projects expect to expand the depth and capacity. This understanding of empowerment has informed data range of communities' social networks. To assess the extent to collection for this study, which explores both the levels of em- which Bank-funded interventions have succeeded in enhancing powerment at the time of fieldwork and respondents' perceptions social capital at the community level, the household surveys col- of changes in empowerment before and after subproject imple- lected information on respondent perception of change in trust, as- mentation. Source: See Annex N for details and results from household surveys on these variables. 1 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure 3.4: Community Priorities Are Better Met When There Is Greater Choice Multisector Single sector 100 75 (%) 50 Communities 25 None None 0 Brazil [N = 27] Benin [N = 13] Uttar Pradesh [N = 19] Madhya Pradesh [N = 21] Top two priorities Top priority Source: Household survey data. Note: Priorities are aggregated by community. porting subprojects in scattered communities, fully met quantitative targets for infrastructure they are also likely to help in increasing access rehabilitation and reconstruction and have also to schools, health facilities, and the like for these provided substantial employment benefits to communities. On the basis of evidence from a the local population (see box 3.2 for the Eritrea limited number of evaluations, a recent Bank Community Development Fund). For example, review of CDD projects (Wassenich and White- 378,805 persons are reported to have benefited side 2004) also presents a favorable picture of directly or indirectly from infrastructure activi- CDD impacts on access to infrastructure. Find- ties under the Kosovo Community Development ings from 4 of the 5 case study countries and 13 Fund Project (2000). In post-conflict situations, of the 19 related OED project assessments sup- where a large part of the basic infrastructure port the conclusion that there has been rela- has been destroyed by war or civil strife, this is tively more success in a considerable achievement. The experience of Several of the CBD/CDD achieving quantitative the Nepal country study also found that well- interventions in conflict goals, such as infrastruc- planned participatory interventions can con- and post-conflict areas ture built and number of tribute to the mitigation of the social dissent training courses held, that fuels conflict. However, in conflict and post- have successfully met than on qualitative goals, conflict situations, where the focus is on get- quantitative targets such as community ca- ting things done quickly, it is even more difficult for infrastructure pacity enhancement and to achieve the qualitative goals. While commu- training quality.4 nities can play a major role in ensuring ac- rehabilitation and The Portfolio Review countability and proper use of donor resources reconstruction and have found that several of the in these situations, what is often lacking in post- provided substantial CBD/CDD interventions conflict communities is the ability to act to- in conflict and post-con- gether. As noted by an OED report on The World employment benefits. flict areas have success- Bank's Experience with Post-Conflict Recon- 1 6 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS struction (OED 1998) "Inherent in violent civil automatically fulfill the qualitative goal--for ex- conflict is the destruction of social capital, par- ample, holding a certain number of training ticularly institutions of governance and civil so- courses is expected to enhance capacity. ciety and such basic attitudes and behaviors as trust and participation." The Bank's CBD/CDD Projects Appear to Inadequate tracking of progress on qualitative Have Enhanced the Social Capital of aspects has been a shortcoming in most Bank in- Communities But Have Had Variable terventions. However, it becomes an even big- Success in Empowering Them ger issue with CBD/CDD projects, particularly in Wassenich and Whiteside (2004) found that CDD projects, where enhancing the capacity of only two Bank impact studies for CDD projects communities and local governments is a cen- have findings on social capital that are reliably tral objective. However, without baseline data, representative of all CDD projects, and those two and without appropriate indicators, it is often as- studies showed mixed results regarding the sumed that meeting the quantitative goals will contribution of CDD projects to the enhance- Box 3.2: Highly Satisfactory Aspects of Design and Implementation in Selected CBD/CDD Projects The Peru Rural Roads Rehabilitation and Maintenance Project The Turkey Eastern Anatolia Watershed Rehabilitation Project (1996) achieved its key project objective of providing a well- (1993) largely delivered on its objective of restoring sustainable man- integrated and reliable rural road system through rehabilitation agement of forest and farm activities in the upper watersheds of and maintenance of rural roads and key links connecting to the the three project provinces, reducing soil degradation, erosion, and primary road system. In this project community participation was sedimentation in reservoirs, as well as increasing productivity an important part of project preparation. A logical framework de- and incomes in this impoverished region of Turkey. With respect signed by the participants in the first meetings was used to to processes, important experience was gained by the public sec- structure a community's involvement in the project. (PPAR July tor with community-based participation. There was very good-- 2001) unprecedented in Turkey--coordination between ministries and The experience of the Nepal Hill Community Forestry Project departments in this project. (PPAR March 2004) (1990) shows that clear policies, laws, and procedures are cru- The Benin Borgou Pilot Project (1998) interventions took place cial to building trust between the communities and the Forest De- in 229 villages where the communities successfully completed a partment. Even though the concept of User Group Management total of 296 infrastructure subprojects. A substantial number of lit- was adopted in 1988, implementation was slow in the early years eracy and training courses were held to improve technical capacity as a supportive framework was missing. However, things changed in the communities. The project built on the Bank's experience in with the passing of the Forest Act of 1993. The procedural clarifi- Benin with the Village-Level Participatory Approach (VLPA) in the cations on the implementation of the act followed with the Rules 1990s. Under the VLPA, extension staff and other government and in 1995. As a result, identification of user groups and handing over private agencies encouraged village communities to analyze their of forests accelerated after 1995. (PPAR June 2001) situation, identify priority problems, prepare action plans to deal The Uttar Pradesh Sodic Land Reclamation Project (1993) ex- with them, and implement those action plans. (PPAR June 2003) ceeded expectations in carrying out large-scale reclamation of The Eritrea Community Development Fund (1996) financed the sodic soils. Over 68,400 hectares were reclaimed with farmer par- rehabilitation or creation of a significant amount of social and ticipation. The project also contributed to poverty reduction by help- economic infrastructure in the rural and war-devastated areas of ing increase returns to many small and marginal farmers. The Eritrea. The Eritrea Community Development Fund made an im- project effort in sequencing activities helped harness farmer portant contribution to population needs in a post-conflict situa- commitment for agricultural development on reclaimed lands. For tion. The project adopted a short-term problem-solving approach example, the transparent land titling was done before undertak- and delivered outputs under extremely difficult conditions. (PPAR ing of technical solutions to sodicity. (PPAR June 2004) June 2002) 1 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT The Bank has had the ment of social capital. where it has provided consistent, long-term ca- most success in areas Similar findings emerged pacity building support to communities. Evi- from the fieldwork car- dence from other Bank studies supports this where it has either ried out for OED's Social finding.5 The Bank provided support to India's supported indigenously Fund Evaluation (OED Operation Flood through five projects during the matured participatory 2002b). OED's analysis period 1974­87. The program is dominated by of the household data farmer-controlled, village-level dairy coopera- initiatives or where it from Benin, Brazil, and tive societies; the Bank's financial support added has provided consistent, India found the associa- value to this indigenously matured CBD/CDD ef- long-term capacity tion between CBD/CDD fort, which had already experienced a critical projects and respon- amount of learning and institution building.6 building support to dents' perceptions of The OED impact study for the program noted communities. changes in social capital that "Operation Flood is an Indian pro- to be positive in Benin, gram....when the program was massively ex- Madhya Pradesh, and Uttar Pradesh; results in panded under Operation Flood II, there was an Brazil were mixed. Household data also show already existing indigenous institution ready to that the association between CBD/CDD proj- implement the project. Bank institutional sup- ects and respondents' perceptions of changes in port thus involved genuine institution building, empowerment was variable across the four proj- as distinguished from the institution creation ect areas. Four results patterns emerged from the characteristic of many Bank projects" (Candler information gathered for this evaluation that and Kumar 1998). There are several other ex- are suggestive of the differing impacts that Bank amples (such as the Self-Employed Women's As- CBD/CDD projects can produce at the com- sociation [SEWA] in India, and the Orangi Pilot munity level. This section also attempts to ex- Project in Pakistan7) of well-known development plain these results. initiatives where the initial idea and effort started First, the Bank has had the most success in indigenously, long before the Bank or another areas where it has either supported indigenously source of external support was provided. By matured participatory initiatives, as it did with similar logic, communities that have effective the India dairy program Operation Flood, or leadership and pre-existing ability to organize for Box 3.3: Local Champions Can Effectively Use Bank CBD/CDD Funds: A Case from Ghana Although the outcome of the Ghana Agriculture Sector Invest- which provided the group with an agro-processing unit. How- ment Project (ASIP, 1994) as a whole was rated unsatisfactory ever, the success of this subproject has less to do with the proj- by OED, a few established groups were able to turn the finan- ect than with the existing group capacity and the dynamic cial opportunities offered by the ASIP to their advantage. The personality of the group leader, who is also helping other, less- Nangbanyini Nyagsa Women's Group, Savelugu Nanton Dis- effective groups in nearby villages. The agro-processing unit run trict, Northern Region, is an example. The group of 24 members by the group now consists of a grain mill, a shea nut crusher, and (2 men and 22 women) had come into existence 10 years before a rice huller. When the group heard that funds were being pro- the project and was then known as the Tiyum Taba Women's vided under the ASIP for income-generating activities, it con- Group. It had started its activity with an agro-forestry unit for tacted the local District Assemblies with a request for an which the village chief had allotted the group five acres of land. agro-processing unit. The group is now considered a success story under the project, Source: OED Assessment of the Ghana Agriculture Sector Investment Project (OED 2001). 1 8 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS collective action have been able to turn the fi- plement its own efforts When the borrower has nancial opportunities offered by a Bank-sup- to organize for collective used the opportunity ported project to the advantage of the group, as action, and the com- provided by Bank in Ghana (box 3.3). Field visits for OED's proj- munities believe in the ect assessments of the Mali Natural Resource long-term benefits of fol- financing to supplement Management Project, the West Bengal Forestry lowing the approach, in- its own efforts to Project, and the Pakistan Northern Resource terventions have met organize for collective Management Project support this finding. with reasonable success, Extending this logic further, when a Bank in- as with forestry projects action, and the tervention has been built on past experience in India, where Bank communities believe in with a similar capacity-building approach and lending helped the coun- the long-term benefits of with a focus on the process of bringing com- try bridge the financial munities together to organize for collective ac- resource gap it faced in following the approach, tion, reasonable success has also been achieved. implementing its forest interventions have met In these cases, the same communities have been strategy directed toward with reasonable success. the focus of the capacity- building effort for sev- supporting joint forest eral years. As shown by OED's evaluation of the management (Kumar and others 2000). The Aga Khan Rural Support Program in Pakistan Government of Uttar Pradesh also placed a high (OED 2002c),8 one of the basic reasons for suc- priority on reclamation of sodic lands and looked cess in this program has been the sustained to the Bank for financial support. In Madhya support for processes in the same communi- Pradesh, analysis of the household survey data ties over 20 years. In Benin, the findings from indicated that the Bank's strategy of community OED's household survey indicate that the Bank's involvement was positively associated with strategy for community participation had more change in social capital, but much less so with success in enhancing social capital and em- empowerment. In Uttar Pradesh, the house- powering communities in the Borgou Pilot proj- hold data analysis indicates that the Bank's proj- ect than in the Social Fund or the Food Security ect was positively associated with change in projects. The Borgou Pilot built on the Bank's social capital, and to a greater extent than in experience with the Village-Level Participatory Madhya Pradesh with empowerment.9 Indeed, Approach exercise in Benin, which included par- had the Bank taken steps to ensure sustain- ticipatory rural appraisal and other partnership ability in both cases (see section on Sustain- efforts that enabled communities and villages to ability in Chapter 4), even more positive results coordinate and execute their own rural devel- could have been achieved. opment, with assistance from extension agents Third, when undertaken in circumstances and financial resources from a variety of pro- where political and social settings have not his- grams. Bank staff are increasingly aware of the torically favored participation and collective need for long-term support to build commu- action, as in the state of Rio Grande do Norte in nity capacity. One of the criteria for selection of Brazil, the Bank's strategy of community partic- communities under the Indonesia Kecamatan ipation may have little influence on community Development Fund Project (KDP) 3, which be- social capital and empowerment.10 Analysis of the came effective in January 2005, is that they re- household survey data from that state indicate ceived support under KDP 1 or 2. About 52 that, at best, there is no difference in respon- percent of Bank staff surveyed said that com- dents' perceptions of change in empowerment, munity groups initially formed under a project while results for social capital have been minimal need support for at least 6­10 years to reach a (Annex N). The literature (Costa and others level of sustainability in community processes 1997; Tendler 1997) reveals that the states of (Annex L). Northeast Brazil are known for their clientelis- Second, when the borrower has used the tic ways of governing. The patron-client rela- opportunity provided by Bank financing to sup- tionships that pervade the Northeast create a 1 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure 3.5: Focus Groups Report Significant Decision interventions (Desai 1996; Gibson and Marks Making by Local Leaders 1995; van der Linden 1997; Ribot 1998). The 2004 World Development Report, Making Ser- vices Work for Poor People, also notes that "elites 100 can mobilize more quickly, master the rules of submitting applications (if they can read and 80 the majority of the community cannot), and groups present themselves to the community as an 60 effective conduit for receiving funds" (World focus of Bank 2004d, p. 73). Therefore, the report cau- 40 tions that "Rushing to create social capital where it does not exist can do more harm than good." Percentage 20 OED's focus groups in Brazil found that many communities indicated dependence on an indi- 0 vidual or a small group of community leaders to Benin Brazil Madhya Uttar bring donor-funded projects to the community. [N = 26] [N = 44] Pradesh Pradesh [N = 40] [N = 38] In Benin also, focus group sessions revealed that decision-making procedures surrounding Source: Focus group interviews. the selection of subprojects lacked the active participation of community members. Village leaders, whether traditional, administrative, or social system in which vertical ties of mutual the Groupement Villageois (which in the Borgou dependence hinder development of strong hor- region are mainly structured around the cotton izontal links of solidarity within communities.11 sector), took the lead in identifying subproject Although the Brazilian government's decentral- activities, and only later brought them to the ization policies appear to have weakened these communities for approval. In Madhya Pradesh traditional forces, they continue to exercise a and Uttar Pradesh also, a majority of villages in strong influence. An issue raised in some focus OED's focus groups indicated that most decisions group sessions with communities in Rio Grande taken by the Panchayat were largely devoid of do Norte is that communities that receive sub- any broad community participation. OED's project funds do so through political relation- household data shows that in Madhya Pradesh, ships, and not because of need. It is thus not Uttar Pradesh, and Brazil, respondents who were always a case of a community needing a moti- members of community organizations set up by vated individual, but a "connected" one. the Bank projects had a higher socioeconomic Fourth, when a Bank-supported intervention profile, including greater mobilization skills and attempts to build social capital and empower a more extensive social network, than non-mem- communities, the capacity-building benefits bers before the Bank intervention. Further, in may be cornered by the Madhya Pradesh and Uttar Pradesh, these mem- When a Bank-supported "better-off " community bers of project organizations also reported a members. Various stud- greater increase in social capital than did the intervention attempts to ies in the literature on non-members. In Madhya Pradesh, Uttar Pradesh, build social capital and participatory develop- and Brazil they also reported a greater increase empower communities, ment point out that the in empowerment (Annex N, endnote 7).12 better-educated mem- These results are not difficult to explain. Three the capacity-building bers of the community factors appear to be responsible. benefits may be cornered and the relatively better- by the "better-off" off are often the ones Projects generally do not tailor capacity build- who represent the com- ing to community capacity: The literature shows community members. munity in participatory that communities are at different stages in the evo- 2 0 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS Figure 3.6: Subproject Cycle Is Too Short for Meaningful Enhancement of Community Capacity Preparation Elect Project Management Committee Prepare/submit subproject application and budget Mobilize community contribution Plan launch ceremony Sign Financial Agreement Implementation Procure materials and services Supervise construction Manage labor contribution months Prepare progress reports Manage funds 12± Monitoring and Evaluation Develop own indicators/M&E plan Provide day-to-day oversight Conduct audits Completion Take over operation and management of asset lution of social capital and empowerment, and changed the subproject cycle at the community hence have different capacities (Greiner 1972; level. The one year of a typical subproject cycle Handy 1985; Pretty and Ward 2001). Communities (figure 3.6) is enough time to implement a sub- in Brazil, Benin, and India (and within these coun- project and, in several cases, to start a process tries) likely have different capacity levels; there- of change in a group, or move a group further fore, the change that can be expected as a result along the evolutionary process if it is at a higher of exposure to a Bank-supported project is also level of development, but in most cases is in- likely to be different. Yet project documents in- sufficient to take it far enough for sustainability dicate that Bank-supported projects do not di- to be ensured. OED's Social Fund Evaluation agnose community capacity or tailor capacity also found that the nature and extent of infor- building to existing community capacity. mation sharing and participation by community members in social fund projects was sufficient to The Bank subproject cycle is too short: The lit- allow successful subpro- erature also shows that building capacity is a ject execution, but not to The one year of a typical time-consuming process (Ostrom 1999; Pretty consistently have a sig- subproject cycle is enough and Ward 2001).13 However, Bank interventions nificant positive impact have not been designed to provide long-term on community capacity.15 time to implement a support. Village societies traditionally have been Further, a group with lit- subproject and to start a hierarchical, with the local leaders making the de- tle capacity may even process of change in a cisions. It is difficult for a Bank intervention to regress at the end of the change this with the limited money and short pe- subproject cycle.16 If a group, but in most cases is riod of support in a subproject (figure 3.5).14 group's capacity is at a insufficient to take it far The flexibility that many of the CBD/CDD proj- higher level, as was the enough for sustainability ects appear to have because of the use of an one in Ghana (box 3.3), to be ensured. APL or LIL instrument (Chapter 2) has not then it is able to use the 2 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT CBD/CDD projects can Bank financing to its ad- ergy of the communities and their leaders is enhance social capital vantage. marshaled toward ensuring the maximum re- Also, Bank projects source inflow to their village. Hence, as noted and foster empowerment typically have not gone by OED's Social Fund Evaluation, it appears that at the community level, back to the same com- the participatory interventions are "users" of but the link between munity with the same ap- existing social capital rather than "producers" of proach to enhancing it. "Using" social capital may ultimately con- CBD/CDD and social capacity to follow up tribute to its increase, but this increase does capital and community where the initial subpro- not seem to be of the kind envisioned in a Bank empowerment is weak. jects left off. Even if an- intervention. The social capital that appears to other Bank project does be strengthened is that which can ensure that go back to the same community, in most cases each community has the best opportunity to it does not build on the capacity-building effort attract the maximum external resources. While of the earlier interventions. Where it does, as in the literature confirms that social capital repre- the Borgou Pilot in Benin, positive results can be sents a potential--a propensity for collective expected. The change a year of effort can bring action (Narayan 1995; Narayan and Pritchett about in social capital and empowerment in a 1997)--and provides examples where investing given community cannot be expected to be very in it has led to desirable returns through in- dramatic. creased benefit flows (Uphoff and Wijayaratna 2000), it also notes that whether or not its po- Communities have a different understanding tential is activated and for what purpose de- of the role they are expected to play in CDD pends on several factors (Krishna 2001).18 projects than does the Bank: Focus group data In conclusion, this study finds that CBD/CDD in all three countries indicate that the commu- projects can enhance social capital and foster em- nities' understanding of participation in a sub- powerment at the community level, but the link project cycle is different from that of the Bank.17 between CBD/CDD and social capital and com- This does not appear surprising given that there munity empowerment is weak. It also finds that is also a difference in the understanding of par- the extent to which a Bank project is able to en- ticipation between the Bank and the borrowing hance the capacity of whole communities is de- government, which is technically "in charge" of termined by various local social, cultural, and implementation and sees it primarily as an op- political factors and by the very approach of the portunity to do more with less (see p. 12). The Bank's capacity building effort. The short time Bank CDD projects visualize communities taking that the Bank allows for implementation of a the lead in the choice and implementation of the subproject can lead to the benefits of the ca- subproject. However, analysis of qualitative data pacity-building efforts being cornered by the from Benin and Uttar Pradesh found that vil- better-off in the community. lagers see participation in a Bank project pri- marily as a requirement for them to meet part Targeting the Poor Is Not Enough of the subproject cost, and they see the advan- to Reach the Poor tage of meeting the 10 to 15 percent community Even strong NGO interventions, such as the Pak- contribution requirement, if that amount can istan Aga Khan Rural Support Program, most re- leverage a much larger sum of money. cently evaluated in 2001 and operating for nearly With this understanding of participation, and 20 years, have found it difficult to reach the given that a large num- poorest. The reason it is so difficult is that it in- Even strong NGO ber of communities are volves not just economic change, but also social interventions have trying to "attract" the and cultural changes. Effecting such fundamen- found it difficult to limited amount of donor tal changes requires considerable time and sus- resources, the existing tained effort of a sort that is unusual in a reach the poorest. social capital and the en- Bank-supported project of any kind. 19 2 2 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS CBD/CDD Projects Have Yet to Overcome there be representatives OED project assessments Major Obstacles to Reaching the Poor from the poorest in meet- have found instances The Bank-financed CBD/CDD projects have tried ings, their views, as well of improved living to reach the poor through targeting, but there as their priorities, are is limited evidence to show that they have done likely to remain excluded standards, but the this more successfully than any other Bank in- from collective decision- improvement was vestment. It is not surprising, therefore, that a making processes.24 greater for the better-off recent literature review (Mansuri and Rao 2004) There are even cases found that projects that rely on community par- where the position of among the communities ticipation have not been particularly effective at the poor has actually than for the poor. targeting the poor. A recent study on community- worsened in the context driven rural development projects carried out by of a Bank project. For example, in the Eastern the Inter-American Development Bank notes Anatolia Watershed Rehabilitation Project (1993) that the poorest and the most vulnerable gen- in Turkey, OED's assessment found "that there erally are not reached (Dahl-Ostergaard and oth- had been some short-term losers, in particular ers 2003). OED's evaluation of the Aga Khan landless livestock owners." While the assess- Rural Support Program in Pakistan came to a ment does acknowledge that village leaders gen- similar conclusion (OED 2002c). erally have attempted to allocate compensatory OED project assessments have found in- benefits to losers, there have been challenges in stances of improved living standards, but the doing this effectively. In India, the OED assess- improvement was greater for the better-off ment of the Andhra Pradesh Forestry Project among the communities than for the poor.20 found that livestock herders, fuelwood head- This suggests that the project may have had lit- loaders, shifting cultivators, and a dispropor- tle effect on socioeconomic factors. Specifically, tionate number of women, all within the poorest the OED assessment for the Egypt Matrouh Re- groups, may have been losers. In Nepal, the source Management Project (approved in 1993) OED Review of Community Forestry (Kumar noted that although genuine attempts to reach 2002) noted several reasons why the poorest the poorer farmers were made, the gains to the may be the losers. The OED assessment of the large and medium-size farmers were more than Borgou Pilot Project in Benin noted that the those to small farmers, since many benefits were community contribution typically required in based on land ownership. The literature also Bank interventions created hardships for the supports this finding.21 poor. It is very difficult for the poorest to make The thoroughness of the Bank's effort also their cash contribution, so they usually have to fell short in some cases. OED project assess- contribute time and labor, which takes them ments and studies found that even when sup- away from income-earning activities. In situa- porting an activity such as forestry, which can tions where the rich contribute on behalf of the benefit the poor, issues critical to their liveli- community, the position of the elite is strength- hood have not received adequate attention. For ened relative to that of the poor.25 example, in Bank-supported community forestry interventions in Nepal and India, marketing of Sophisticated Targeting Strategies May non-timber forest products has been neglected Help, but They Are Too New to Assess (Kumar and others 2000; Kumar 2002). One rea- Some more recent CDD projects incorpo- son for this is found in the literature, which notes rate quite sophisticated that the poor remain largely excluded from par- poverty targeting strate- The poorest are likely to ticipatory "spaces" created by donor-supported gies (box 3.4), and their remain excluded from CBD/CDD interventions (Kumar and Corbridge ability to achieve greater 2002; Turton and Farrington 1998).22, 23 Even success in reaching the the collective decision- where they are "formally" included in a partici- poor will need to be care- making processes. patory "space" because projects may require that fully assessed upon com- 2 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Box 3.4: Example of Sophisticated Poverty Targeting in Two Recent Vietnam CDD Projects The two CDD projects in Vietnam--the Northern Mountains which puts small budgets under the control of the communes, Poverty Reduction Project (NMPRP; fiscal 2002) and the Com- will target the needs of the very poor households and groups (so- munity Based Rural Infrastructure Project (CBRIP; fiscal 2001)-- cial targeting). include sophisticated targeting mechanisms to reach the poor. The CBRIP targets 3,600 poor people in the 540 poorest com- The NMPRP targets 368 of the poorest communes in 44 districts munes in 13 provinces (geographic targeting). Poor communes in the 6 provinces of the Northern Mountains Region using es- were selected based on the criteria established by the government tablished government criteria (geographic targeting). Using par- for its own Program 135. The project will support small-scale pub- ticipatory planning processes, the project attempts to involve the lic infrastructure works intended to increase household incomes, poor within the targeted communes. Subproject selection criteria improve living standards, and reduce poverty and vulnerability favor small-scale subprojects implemented in or giving access to (self-targeting). the poorest and most remote villages within each project commune (See Annex O for definitions for different targeting mecha- (self-targeting). The commune development budget component, nisms.) Source: Vietnam Country Study. pletion. But findings from project assessments, reliability of the methodology used by the gov- ICR Reviews, country studies, and a desk review ernment to identify poor people (Minot and of project documents for six ongoing CDD proj- Baulch 2004; Yukio 2001). In India, the lack of re- ects indicate several reasons why it may be dif- liable information on who the poor are has been ficult to realistically implement these strategies. a major constraint in the Andhra Pradesh District First, political pressures in the country may Poverty Initiatives Project (2000).26 make things difficult. For example, in Indonesia, Third, in some countries, Bank efforts to reach fieldwork done for the OED assessment of the poorer communities have been constrained by Kecamatan Development Fund Project found their lack of easy accessibility. For example, the that while poverty-related survey data were used household data for the forestry project in Mad- for ranking communities, provinces with lower hya Pradesh indicate that, on average, commu- percentages of poor still had to be included for nity members in dispersed villages are less wealthy political reasons. The early findings of the OED than those adjacent to towns that are less dis- assessment for the Pakistan North West Frontier persed. However, it appears that a relatively larger Community Infrastructure Project also reported part of the project resources were directed toward political interference in villages of the latter type. CBD/CDD projects have the selection of commu- contributed to greater nities. Substantive Participation of Women in the Second, projects Development Process Has Remained Elusive "formal" inclusion of could find it difficult to The Bank considers promoting gender inclusion women in participatory reach the poor commu- a key design principle for CBD/CDD interventions "spaces" than was nities because of data because women in most parts of the world ex- possible in the past. limitations. For example, perience significant socio-cultural constraints on in Vietnam, government their participation in development. There are However, they have had data are being used to two aspects to assessing progress on gender in- limited success in identify poor communes clusion for CBD/CDD interventions: how much promoting women's in the latest CDD proj- have women benefited from the participation ects (box 3.4), but there process, and has the infrastructure or activity "substantive" inclusion. is skepticism about the that was supported specifically benefited them? 2 4 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS The literature, OED assessments, and country versus achieving the same results through a studies show that because of specific gender-tar- non-CBD/CDD approach (Annex J). geting strategies and the support these projects have given to laws that promote gender inclusion, Less than a Fifth of Closed CBD/CDD Projects CBD/CDD projects have contributed to greater Have Calculated an Economic Rate of Return "formal" inclusion of women in participatory At the individual project level, an economic rate "spaces" than was possible in the past.27 However, of return (ERR) has not been calculated either they have had limited success in promoting ex-ante or ex-post for the majority of CBD/CDD women's "substantive" inclusion, which, ac- interventions. Of the closed projects in the pop- cording to the literature, can be largely attributed ulation of 847, only 24 percent attempted an to social norms that define gender roles (Agar- ERR at appraisal and 17 percent reported an wal 2000a, b, 2001; Cornwall 2003). OED's the- ERR at completion. This is significantly different matic review of the community forestry program from 39 percent of the non-CBD/CDD projects in Nepal noted that "the extent of women's par- at appraisal and 34 percent at completion. While ticipation and involvement in user group activi- it may be reasonable to argue that an ERR can- ties is a reflection of their position in Nepalese not be calculated in CBD/CDD projects ex-ante society. Community forestry, in so far as it em- because the subproject investments are not iden- powers women, can help improve their social sta- tified at appraisal, what reasonably can be cal- tus, but it is unreasonable to expect that it would culated is expected ERRs for typical subprojects radically transform gender relations" (Kumar that the intervention is proposing to support. Cal- 2002). OED's assessment of the Benin Borgou culation of ERRs at completion for at least a sam- Pilot Project noted the need to give careful at- ple of subprojects should be a requirement for tention to local social and cultural factors in fram- all projects, because by then all costs and many ing realistic gender-related project objectives. In benefits are known or can be estimated. In many the Matrouh Project in Egypt, where there had ICRs that do not calculate an ERR at comple- been a substantial focus on women, focus group tion, the only reason given for not doing so is that meetings with communities revealed that the it was not done at appraisal. percentage of women who believed that they In recent years, much more attention also had benefited from the project was highly vari- has been given to impact studies. In several in- able. The seven focus groups conducted found terventions, impact studies have been attempted that there were no elected women leaders or at midterm and completion. This study reviewed women's associations, and there were substan- some of these impact studies for their quality and tial concerns about marketing of products pro- found them lacking in methodological sound- duced by women. ness.28 A major reason for this seems to be the poor data and lack of a reliable baseline for mak- Efficiency ing comparisons. In the absence of baseline data, the "with-without" approach is attempted, but Efficiency Has Not Been a Primary often with limited success. Consideration in the Bank's Support for CBD/CDD Projects Bank Costs for CBD/CDD Have Been Ideally, efficiency should be addressed at two Significantly Higher than for levels in CBD/CDD projects: first, absolute Non-CBD/CDD Projects efficiency at the level of the individual project, Other than limited at- OED found the as is normally expected in all Bank projects, tempts as a part of cost- through rate of return or other measures; and, effectiveness analysis, operational costs to second, comparative efficiency as an alterna- there has been no the Bank to be higher tive approach to development, comparing the systematic analysis of for CBD/CDD than for costs and benefits of attempting to undertake costs and benefits of non-CBD/CDD projects. development through a CBD/CDD approach CBD/CDD projects in 2 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure 3.7: CBD/CDD Projects Cost the Bank More Than Non-CBD/CDD Projects Average Supervision Costs (closed investment lending) 600 R2 = 0.7894 500 N = 374 R2 = 0.6743 N = 1493 400 US$) (000 300 Costs 200 100 0 0­20 20­40 40­60 60­80 80­100 100­120 120­140 140­160 160­180 > 180 IBRD/IDA commitment (million US$) Non-CBD/CDD CBD/CDD Average Costs Prior to Board Approval (investment lending) 600 R2 = 0.7142 500 N = 839 R2 = 0.9274 N = 2361 400 US$) (000 300 Costs 200 100 0 0­20 20­40 40­60 60­80 80­100 100­120 120­140 140­160 160­180 > 180 IBRD/IDA commitment (million US$) Non-CBD/CDD CBD/CDD Source: World Bank data. Note: *Significant difference is noted between CBD/CDD and non-CBD/CDD projects for investment lending of less than $60 million for costs prior to Board approval. The mean investment lending for CBD/CDD projects is $54 million. Results on costs were aggregated across projects and rep- resent an average. Hence, individual project and country experience could vary. CBD/CDD costs are higher than non-CBD/CDD, even when costs are normalized by loan size. comparison with non-CBD/CDD projects from comparative study of costs for preparation of the Bank's perspective. The only relevant Bank participatory versus non- participatory source that OED could locate is a working projects. It found those for the former to be paper (Hentschel 1994) that undertook a higher. For this study, OED compared 2 6 OUTCOME OF BANK-SUPPORTED CBD/CDD PROJECTS operational costs to the Bank of CBD/CDD SAPROS and World Bank There is a substantial versus non-CBD/CDD projects and found them 2000) and work done for cost in time spent by to be higher for CBD/CDD by about 10 percent the Egypt and Brazil case government officials in (figure 3.7 and Annex J). Bank staff perceptions studies and an OED as- (Annex L), revealed through a staff survey, are sessment in Indonesia putting a participatory in line with these findings. No study has yet confirm this finding. approach in place. taken this issue further to explore what this higher cost to the Bank means in comparison ...Communities Bear a Part of the Cost of with benefits from a CBD/CDD approach. Service Delivery Infrastructure in For the Bank, these higher costs could be CBD/CDD Interventions justified under two conditions: first, if the higher Lower infrastructure unit costs to the govern- costs are fixed, and ultimately--as the Bank does ment are often the result of communities' shar- more CBD/CDD--its average costs for under- ing in the cost of construction and contributing taking these interventions would potentially fall; (or providing) for operation and maintenance. and, second, if the benefits to the client coun- The Bank's recent self-evaluation of social funds tries are going to be so large from the new ap- in six countries (World Bank 2003d) also noted proach that the Bank is willing to bear the that community management of investments additional costs in the interest of achieving provides a significant opportunity for cost savings poverty reduction. There is no evidence that of as much as 25­50 percent.29 However, the the first condition is going to hold. If Bank proj- merit (or its lack) of shifting part of these costs ects adhere to a true learning process approach to the communities has not been a factor in the (Korten 1980), each community intervention decision to increase support for CBD/CDD proj- must be made specific to the needs of the par- ects. As the literature shows, the cost of com- ticular community involved. Under these cir- munity participation (in cash, kind, or labor) can cumstances, the likelihood of a decline in costs be substantial, particularly under the more in- resulting from scaling up is low. tensive participatory approaches.30 If the op- portunity cost of the time that community The Cost to the Government of Introducing a members spend in meetings with donor and gov- CBD/CDD Approach Has Been Substantial... ernment officials is also considered, the costs In dealing with the second condition, the surveys are higher still. The lack of discussion of the of government officials in case study countries issue of beneficiary time in any appraisal reports (Annex I) revealed that there is a substantial suggests that the opportunity cost of time is as- cost in time spent by government officials in sumed to be low for households in poor com- putting a participatory approach in place, though munities. However, farm management studies it is reasonable to expect that these costs will de- over the years have generally shown this not to cline over time. About 75 percent of central gov- be so, and have shown high costs at peak agri- ernment officials in Benin, Nepal, and Vietnam cultural labor periods (Schultz 1964; Collinson and 75 percent of local government officials in 1982; Renata and Houston 2002). 31 Benin and 80 percent in Brazil recognized the in- Further, considering costs only in terms of in- creased time and, hence, cost implications for the frastructure construction and flow of services is government in initially putting a participatory not enough (even if it takes into account com- process in place. munity costs) because CBD/CDD projects, and particularly CDD projects, are also trying to en- Though the Actual Costs to the Government hance the capacity of the communities. It is thus for Infrastructure Have Been Lower... necessary to consider the cost for the capacity- The cost to the government for infrastructure enhancing exercises, both in resources and in through CBD/CDD projects was found to be time spent by government officials and by com- lower than that for non-CBD/CDD projects. Stud- munities. By estimating community time for ies carried out in Nepal (SAPROS & IFAD 2002; both subproject implementation and capacity 2 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT enhancement, it may be possible to approxi- pact is not noticeably better for CBD/CDD proj- mate overall costs for a CBD/CDD intervention. ects, then the extra costs of the CBD/CDD ap- (See Annex J, box J.1, for an example.) proach may not be worthwhile. Even though poverty reduction is the major objective of many Undertaking a Comparative Assessment of these projects, a full-fledged comparative as- of All Benefits Constitutes an Equally sessment of the poverty impact of different ap- Challenging Exercise proaches would require further research. The Benefits from capacity-enhancing activities are limited evidence available is in the form of the particularly difficult to assess--for instance, what beliefs of the government officials in the case should be done if the gains from the capacity- study countries and some limited references in enhancing exercise are concentrated in a small the literature.32 Government officials in four case section of the population? Is that a positive or a study countries reported that projects with par- negative? The only sure way to assess benefits ap- ticipation have better outcomes than those with- pears to be in terms of the poverty impact. If out participation (Annex I). It is also commonly CBD/CDD projects are delivering poverty-re- argued that if communities are willing to con- duction benefits, and non-CBD/CDD projects tribute for subprojects, they must perceive the are failing to do so, or doing so inadequately, then benefits of the CBD/CDD interventions to be at spending more to get the desired results may be least equal to the costs. worthwhile. However, if poverty-reduction im- 2 8 4 Institutional Enhancement and Sustainability T his chapter draws on country studies, project assessments, and the lit- erature to discuss the extent to which CBD/CDD interventions have enhanced the borrowers' capacity at the central, state, and local government levels to allow them to make effective use of their human, financial, and natural resources. Recent CBD/CDD projects have given greater Institutional Enhancement emphasis to building partnerships between community groups and local government Institutional Development Impact Has organizations, and decentralization reform. Improved over Time While it is beyond the scope of this evaluation to delve into the merits and demerits of Projects Have Not Brought about the decentralization, it does attempt to assess Radical Reorientation in Institutions whether the design and implementation of Required to Undertake CBD/CDD, but They Bank-supported CBD/CDD projects has helped Have Helped Enhance Government Institutions enhance the capacity of local governments in The literature shows that the institutionalization client countries. Capacity at the community of a CBD/CDD approach requires a radical re- level, a critical issue for CBD/CDD projects, is orientation of the way governments and bu- covered in Chapter 3. reaucracies operate (Shepard 1998; Thompson This chapter also assesses the sustainability of 1995). This requires changes in management CBD/CDD projects. Interviews with Bank staff re- and organizational procedures, as well as in the vealed that in the context of CBD/CDD inter- attitudes and behaviors of personnel, that take ventions, the understanding of sustainability time to consolidate (Pimbert and others 2000). itself varies among Bank staff. For some it implies This study found that such a radical reorientation predominantly sustainability of community has not yet come about in most of the Bank's processes, for others CBD/CDD is simply a means client countries, as seen by the experience in to an end, and sustainability is related to infra- OED case study and project assessment countries, structure investments. For still others, it is related although CBD/CDD interventions have helped to overall resource allocation, including support change the attitude of government officials and for decentralization. enabled supportive policy and legal reform.1 2 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Projects Have Supported Changes in the that officials at the central and local levels in- Attitude of Government Officials toward terpret community participation a little differently Working with Communities than does the Bank--more as a means of doing Surveys of government officials in all five case more with less, rather than actually putting com- study countries and findings of project assess- munities in control. As also noted, however, at- ments indicate greater acceptance at all levels of titudes are changing. government of the value of involving commu- But neither the policy and legal reforms nor nities in service delivery/activities and greater the change in attitude guarantee that the understanding and ability to implement a par- CBD/CDD approach will be implemented effec- ticipatory approach to development. For exam- tively. Several factors could challenge the trans- ple, the OED assessment of the Eastern Anatolia lation of reform initiatives into effective working Watershed Rehabilitation Project in Turkey notes arrangements. The following sections explore that at the central, provincial, and community lev- some of the issues around policy reform, in- els it was widely acknowledged that the Bank had cluding the connection with decentralization. introduced new ideas related to community par- ticipation processes. The OED assessment for the CBD/CDD Projects are Hampered by Weak Community Development Fund Project in Eritrea Coordination across Government Departments also notes that the project's emphasis on com- and Government Levels munity contribution was perceived as a useful Although the majority of central government lesson by line agencies in the country. staff in case study countries report an increase in the number of interministry meetings held Projects Have Supported Policy (Annex I), progress in actual coordination has and Legal Reforms been limited, at least in the case study countries. The presence of the Bank has often provided the Interdepartmental coordination problems arise incentive and opportunity for the government primarily because government ministries con- to "push" for legal and policy changes on polit- tinue to be organized sectorally, and the sectoral ically sensitive issues. OED studies indicate and culture is so firmly ingrained that it is difficult for project assessments confirm that the Bank's departments to work together in the context of presence in sectors such as forestry encour- a Bank intervention. While weak coordination aged the governments of India and Nepal to between government departments negatively af- take action on several critical policy issues that fects the implementation of both CBD/CDD and helped generate momentum for change in the non-CBD/CDD interventions, the negative im- sector. However, important overarching policy plications are greater for CBD/CDD interven- issues have occasionally been shortchanged, as tions, which also require interdepartmental in Egypt, probably because of the burden of coordination at the local level. For example, in trying to get the community participation in Egypt, most government staff interviewed be- place.2 lieved that interdepartmental coordination at It appears that most the higher levels across sectors had improved The presence of the Bank countries today subscribe somewhat over the past few years, but coordi- to an approach to partic- nation problems among government staff at the has often provided ipation that is some- community level persist. Each CBD/CDD project the incentive and where between a fully isimplementedbyadifferentdepartmentandhas opportunity for the "bottom-up" and a fully its own unique mechanism for community con- "top-down" way of doing sultation and subproject phasing. government to "push" development. As noted Further, despite progress on decentralization, for legal and policy in Chapter 3, surveys and the Benin, Nepal, and Vietnam country studies changes on politically interviews of govern- found that relations between different layers of ment officials in case government remain difficult. Capacity at the sensitive issues. study countries suggest lower levels of government in all three countries 3 0 INSTITUTIONAL ENHANCEMENT AND SUSTAINABILITY continues to be weak, and the roles of officials CBD/CDD Projects Have Increasingly Supported at various levels are not clearly defined. As in the Government Decentralization Reform case of coordination among government de- Support for decentralization reform and en- partments, weak capacity at the local govern- hancing local government capacity under ment level is more problematic in the context of CBD/CDD projects has increased significantly CBD/CDD interventions, where local officials in recent years. Of the 84 sample projects, 57 sup- are often directly involved in project imple- ported some form of government decentraliza- mentation. Only about 20 percent of Bank staff tion, and the largest percentage of these was in surveyed reported being satisfied or very satis- the later years. Analysis carried out for this study fied with coordination within the government of as a part of the Portfolio Review and country countries borrowing for CBD/CDD interventions studies revealed two fundamental factors that (Annex L). could constrain progress in decentralization, no matter how well-designed the CBD/CDD project. There Is No Evidence to Indicate That the First, success in promoting decentralization in a Participatory Approach Has Been Widely country depends on borrower commitment to Adopted beyond Bank CBD/CDD Projects the reform process. For example, in Egypt, the The evidence from country studies and project country study found that progress on decen- assessments also indicates that while a Bank tralization has been limited primarily because of project may succeed in getting a country to ex- lack of government ownership. In this context, periment with (or adopt) a demand-driven ap- the CBD/CDD projects reviewed in that country proach in a project context in a particular sector, could not accelerate decentralization and re- other government departments do not neces- sponsiveness at the local level. Second, success sarily support the approach. For example, in can also be constrained if there is lack of har- Vietnam, evidence from an internal Bank review monization between the legal framework for de- indicates that while the Coastal Wetlands Pro- centralization and the level at which the Bank's tection and Development Project (2000) is un- CBD/CDD project is implemented. For exam- dertaking mangrove plantations with beneficiary ple, in Benin, the OED assessment of the Borgou participation, the government has its own sep- Pilot Project found that the project supported ac- arate programmatic approach to replanting man- tivities at the village level, while decentraliza- grove coastlines. The report notes that it is tion stopped at the commune level.3, 4 unclear how government ownership will be achieved for the Bank project when there is a rival Individual CBD/CDD Projects Adopt government project under way, or why the Bank Varying Strategies is not financing the government program, with The evidence from four of the five country stud- whatever cost-effective improvements could ies indicates that the Bank has not followed a have been devised and negotiated. In Indonesia, consistent decentralization strategy under its while the field mission for the OED assessment CBD/CDD projects within each country. For found quite strong evidence that the processes example, in Nepal, recognition of weak capacity introduced by the Kecamatan Development Fund at the local level has led Support for decen- Project had been accepted by the government Bank-supported CBD/ at the district government level and below as the CDD projects to provide tralization reform right approach for development, sustainability re- technical assistance to and enhancing local mained difficult to predict and depended to a strengthen local govern- government capacity large extent on the attitude of the new govern- ment institutions. Yet the ment toward decentralization. Bank has also supported under CBD/CDD Finally, the Bank itself has not had a consis- creation of "temporary" projects has increased tent policy across projects in the same country, arrangements for the im- significantly in a point that is evident in its approach to decen- plementation of CBD/ recent years. tralization reform. CDD projects at the local 3 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Box 4.1: Inconsistent Strategies: The Road Sector Experience in Nepal To assist with the limited capacity of District Development Com- ernment organization at the district level has two implications. mittees, the government established the Department of Local In- First, a significant amount of resources get spent on strength- frastructure Development and Agricultural Roads (DOLIDAR) ening the temporary structures. A part of the resources from the under the Ministry of Local Development, with branches in the Rural Infrastructure Project also went toward strengthening districts. DOLIDAR's mandate, which is to provide guidance and DOLIDAR. These resources could have been spent on strength- technical support to district committees in development activi- ening the district-level bodies. The proposed Rural Access Im- ties, has often stretched to interference with the committees' au- provement Project, which is still under preparation, is expected tonomy. While the ICR for the Rural Infrastructure Project claims to continue to provide capacity-building support to DOLIDAR. that a balance between the roles of DOLIDAR and the district Second, the presence of the parallel structures sends a confusing committees in implementing the project was achieved, the very message about the authority of the district-level bodies to take presence of a temporary solution and a branch of a central gov- charge of development activities.a Source: Nepal Country Study. a. Management notes that most CDD operations in Nepal have performed well. level. These arrangements have had a negative ef- planning process of the municipal government fect on the progress of decentralization. The im- (box 4.2). By instituting ad hoc municipal councils plications are particularly worrying when formal for implementation, the project has contributed to training and other capacity-enhancing activities the proliferation of municipal councils, with little for district-level organizations fall short of plans, coordination between them. In Rio Grande do as in the Rural Infrastructure Project in Nepal. This Norte, OED found that municipal councils had perpetuates the need to continue the "tempo- weak capacity, in part because they had received rary" arrangements for a longer period, imped- little training.5 Although the project Technical Unit ing further decentralization progress (box 4.1). provides assistance to the councils, it is not enough. Perhaps it is this inconsistency in Bank strategy Most of the councilors interviewed said they that is picked up in the Central Government Sur- needed more assistance, as they often need clar- vey in Nepal, where only 19 percent of officials ifications on issues, without which they cannot interviewed said that the Bank has the capacity make progress. However, it is important to put the to enhance local government capacity to sup- low level of institutional development of the proj- port participatory interventions (Annex I). Sim- ects' municipal councils into context. Many of the ilarly, in Vietnam, while projects such as other municipal councils share similar weakness, Community-Based Rural Infrastructure (2001) and some are less effective than the Bank-sup- have provided technical support to local gov- ported councils. Indeed, it is worth asking whether ernments, their implementation has proceeded it is even beneficial for both the municipality and through establishment of temporary parallel project or program to have so many councils. structures that manage implementation at the Were the disparate efforts for capacity building to local level under central ministries. be focused on a single integrated municipal coun- Brazil, where the Bank has been supporting cil, the result would likely be a stronger, more ef- CDD projects in the Northeast for more than a fective, and efficient council.6 decade, is another case where the Bank has pro- This lack of consistency in Bank support to de- vided support for parallel centralization under CBD/CDD projects within The Bank may not have structures. The CDD proj- the same country is particularly visible in coun- supported a consistent ects in the Northeast are tries where the Bank has been supporting proj- essentially building a struc- ects under different institutional arrangements. decentralization strategy. ture that is parallel to the This has the potential to send a conflicting mes- 3 2 INSTITUTIONAL ENHANCEMENT AND SUSTAINABILITY sage to the borrower and district officials, espe- Some projects, such as The question of whether, cially when both projects are expected to result the Zambia Social In- how, and in what in similar poverty outcomes. For example, in the vestment Fund (ZAM- circumstances social Philippines, the Bank is supporting more than two SIF), approved in fiscal competing institutional arrangements.7 Piloting year 2000, are designed funds can support with different institutional arrangements makes to integrate into the decentralization is sense if the purpose is to study which would larger decentralization ef- receiving increasing work best before scaling up. But supporting dif- fort in the country. Dis- ferent arrangements side by side over large areas trict authorities are to be attention. on a long-term basis does not send the right sig- devolved increasing nals to the borrower and does not augur well for responsibility in the project cycle for community- long-term institutional development. level subprojects. But while supervision docu- ments report on the considerable amount of Recent Social Funds Have Given Much More capacity enhancement of district officials being Attention to Decentralization Issues undertaken, they also note that the actual trans- On the positive side, the question of whether, fer of social fund responsibilities to the districts how, and in what circumstances social funds can is taking much longer and is a much more diffi- support decentralization is receiving increasing cult exercise than was visualized. attention (OED 2002b). This is a notable im- provement, since older social funds frequently NGOs Have Been Development resulted in structures outside local government Partners in CBD/CDD that have had limited (perhaps even negative) im- According to Bank data, 36 percent of the pact on enhancing local government capacity. CBD/CDD portfolio, versus 8 percent of the non- Box 4.2: Bank CDD Projects Have Added to the Proliferation of Municipal Councils in Rio Grande do Norte (Brazil) Brazilian municipalities have large numbers of municipal coun- Scheme) councils, in PRONAF councils, representation of civil cils (IBGE 2003). Many federal programs require the creation of society and the government is equal. Under PRONAF, funds are an ad-hoc council to implement them at the municipal level. transferred to the municipal government, which is responsible These councils typically have representatives from both gov- for allocation. Communities do not manage funds directly, but ernment and civil society organizations. Field research in Rio receive equipment and infrastructure from the municipal gov- Grande do Norte found that many of these councils are weak ernment. Under the Bank's program, the municipal government in capacity and that there is a substantial overlap in their mem- never manages project funds, which are transferred directly, or bership. In rural agricultural development, the issue of council through the FUMAC-P (Pilot Municipal Community Scheme) proliferation is particularly evident. Municipalities receive funds council, to the communities. These differences (especially the from two main sources, the World Bank (through the Rural second) render it difficult for municipalities to argue for the fu- Poverty Alleviation Program, RPAP, and RPRP [Rural Poverty Re- sion of the two councils, even if the overlap in membership is duction Project]) and the Ministry of Agriculture (through its often significant. Only one of the 13 municipalities surveyed in PRONAF [Brazilian Federal Program to Support Family Agricul- Rio Grande do Norte was able to persuade the 2 funding bod- ture] program). Both funding bodies require that municipal ies that a single council for rural development constitutes a bet- councils be set up to implement their programs.a These two types ter institutional arrangement, and that having two parallel of councils differ in two main respects: membership structure municipal councils that work in an uncoordinated fashion on and funding modality. While the Bank requires civil society to rural development is likely lead to a suboptimal allocation of re- hold the majority of the seats in FUMAC (Municipal Community sources. Source: Brazil Country Study. a. Follow-on Bank projects use the municipal councils established under earlier projects. 3 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Middle-income countries CBD/CDD portfolio, had have also provided support for development of have a limited number of some form of NGO in- NGO capacity. volvement. NGOs have Evidence from the Bank's database (figure NGOs, and consequently traditionally provided 4.1) and the case study countries shows that less participation community development middle-income countries (Brazil, Egypt, and of NGOs in CBD/CDD support, including par- Turkey among the cases) have a limited number ticipatory diagnosis and of NGOs, and consequently less participation of projects compared with the preparation of local NGOs in CBD/CDD projects compared with low- low-income countries. development plans. In income countries. Among other reasons, this is some projects, such as because of the presence of other private and the Uttar Pradesh Rural Water Supply and Envi- public technical assistance providers in the mid- ronmental Sanitation Project (1996), however, dle-income countries. Among the low-income NGOs have also provided engineering support countries, Vietnam has active international NGOs to communities. NGOs have also played a role but few active local NGOs. In contrast, Benin in design and implementation of community and Nepal have numerous active local NGOs. subprojects, as in the HIV/AIDS projects in Africa. In Benin, particularly, the country study found that momentum created by CBD/CDD inter- CBD/CDD Projects Have Contributed ventions was largely responsible for the recent to the Development of NGO Capacity multiplication of local NGOs. Elites in the coun- The evidence from several project assessments try that traditionally were a part of the govern- and all five country studies shows that local NGO ment began forming NGOs and increasingly capacity varies widely across--and even within-- shifted their focus toward donors, where re- countries, and it is difficult to generalize. How- sources, and hence opportunities, were avail- ever, several Bank CDD projects, such as the able.8, What this means for the future is still 9 Uttar Pradesh Sodic Land Reclamation Project, unclear. However, surveys show that a largely Figure 4.1: Low-Income Countries Work with More Partners 50 40 30 projects of 20 Percentage 10 0 Donor involvement NGO partnership Low Low-middle Upper-middle income income income Source: World Bank data. 3 4 INSTITUTIONAL ENHANCEMENT AND SUSTAINABILITY shared opinion among both donor agency rep- stakeholders, including Bank staff and other resentatives and government officials in the donor agency representatives, expressed con- country is that these intermediaries have little ac- cern about the potential role of NGOs in a num- countability toward either the communities or ber of CBD/CDD projects in Benin, noting that the government. when they are paid a fixed proportion of the Partly as a result of the difference in capacity total cost of the project, NGOs tend to push the among NGOs, the degree of success of the part- choice of the subprojects that are the most costly nership between the Bank and NGOs has also but have the minimum operating cost. When varied. This was also the finding of an OED study NGOs behave this way, they may compromise of NGO involvement in all Bank projects (OED the participatory process. The OED assessment 1999a). Bank staff, when asked in the staff sur- of the Northern Resource Management Project vey whether NGO-supported interventions gen- in Pakistan found that there are both pros and erally achieve a better outcome than Bank cons to handing over all or most community mo- interventions, were fairly evenly divided: 31 per- bilization to NGOs. The assessment notes, "In cent disagreed or strongly disagreed, 26 per- favor of such an approach is the generally ac- cent agreed or strongly agreed, and 24 percent knowledged skills of NGOs. Against it, however, fell in between. Such responses may not be par- is that public technology transfer will continue to ticularly revealing because this is a complicated be needed and, in order to play a key role, pub- question, and the answer is likely to depend on lic services can benefit from the first-hand expe- the respondent's experience. NGO focus groups rience of participatory approaches. The aim in country studies, however, revealed dissatis- should be to work out effective and efficient ser- faction among NGOs with the unequal relation- vice delivery arrangements involving all actors: ship that they have with the Bank in a CDD central government, NGOs, private sector, local intervention and with Bank procedures. governments, and local communities." NGO Partnership Can Be Important Most Donors Support Community Participation in Reaching the Poor as a Strategy, but There Is Lack of Agreement The Portfolio Review, project assessment find- on Implementation Procedures ings, and country studies found that local NGOs, Bank data show that 40 percent of both because of their familiarity with local conditions, CBD/CDD and non-CBD/CDD projects have mul- have been important in helping some CBD/CDD tidonor involvement. Several evaluations of Bank projects to reach the poor and disadvantaged projects (CBD/CDD and non-CBD/CDD) have populations and as catalysts in mobilizing com- highlighted problems that arise when donor ef- munities. For example, NGOs helped mobilize forts are not well coordinated. However, the farmers and disseminate technology, and assisted case study and project assessment evidence village-level institutions in developing links with shows that the challenge of donor coordination government agencies in the Uttar Pradesh Sodic is greater in a CBD/CDD project than in a non- Land Reclamation Project. OED's project assess- CBD/CDD project. The country studies in Benin, ments also found that NGOs played an important Nepal, and Vietnam and Local NGOs have been role in enabling the Peru Rural Roads Rehabili- fieldwork for project as- tation and Maintenance Project to exceed its sessments in Benin, important in helping target number of villages benefited and in im- Ghana, and Mali confirm some CBD/CDD projects plementing the Andhra Pradesh Forestry Pro- that several donors are to reach the poor and ject. That said, the Benin country study found that often present in the same NGOs with poor qualifications handicapped proj- community, with differ- disadvantaged ect implementation in that country. In both the ent CBD/CDD strategies, populations and as Social Fund and the Food Security Projects in providing infrastructure catalysts in mobilizing Benin, a large number of NGOs had to be sus- or activity support in a pended for unacceptable performance. Many seemingly uncoordinated communities. 3 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT The challenge of manner. Each donor in- the coordination issue could be resolved by al- donor coordination is tervention at the com- locating different sectors among donors based munity level may require on their comparative advantage, with regular greater in a CBD/CDD a separate committee to meetings among them to keep the group in- project than in a meettheimplementation formed. In Vietnam, in contrast, nearly all donor non-CBD/CDD project. requirements. The sub- representatives said that stronger coordination stantial confusion that by the government could resolve the issue. this creates at the community level is a much more serious issue than the strain on institu- Coordination among Donors Is More tional capacity at the government level. Thirty- Difficult for Small, Low-Income Countries nine percent of the Bank staff surveyed agreed Than for Middle-Income Countries or strongly agreed with the observation that in- The experience of Benin, Nepal, and Vietnam also adequate donor coordination in a cofinanced indicates that the presence of numerous bilateral project is likely to have a greater negative impact and multilateral donors, each implementing on outcomes in a CBD/CDD project than in a CBD/CDD projects, but with different institu- non-CBD/CDD project. tional arrangements and procedures, has put a Interviews with Bank and other donor staff for tremendous coordination burden on these gov- three of the five country studies found that al- ernments and stretched their limited institu- though most donors endorse community par- tional capacity. This finding is supported by ticipation as a strategy, there is less agreement evidence from the literature.10 on implementation plans and procedures. For ex- In middle-income countries, however, the ample, in Nepal, the coordination efforts of in- Bank's data reveal (see figure 4.1), and the ex- dividual donor representatives are hindered by perience of the Brazil and Egypt country studies a feeling of competition among donors. Most confirms, that fewer donors are involved in donor officials interviewed in Benin said that CBD/CDD projects in those countries. In Brazil, in the state of Rio Grande do Norte, for exam- ple, no other donor is supporting CBD/CDD projects. In addition, countries such as Egypt Figure 4.2: Sustainability Has Been Consistently have better administrative and institutional ca- Lower for CBD/CDD Projects but Is Improving pacity to handle coordination. Few of the Poverty Reduction Strategy Papers 100 (PRSPs) reviewed--but most Country Assistance Strategies (CASs)--note donor coordination as an issue (Annex H). Since PRSPs are meant to be 80 the outcome of a country-driven process of dis- cussion on priorities and challenges, this finding (%) 63% 60 indicates that for the large majority of the bor- 55% rowers, this issue may not be high on their agenda. better 48% or 40 37% Sustainability Likely 20 Infrastructure and Activities Have Been Difficult to Sustain beyond the Projects The sustainability of the CBD/CDD portfolio has 0 1994­98 1999­2003 been improving, but shows considerable scope Exit fiscal year for further improvement (figure 4.2). The sus- CBD/CDD Non-CBD/CDD tainability ratings have varied by Region, with the highest rating in the Middle East and North Source: World Bank database. Africa, followed closely by Latin America and the 3 6 INSTITUTIONAL ENHANCEMENT AND SUSTAINABILITY Caribbean11 (Annex G, table G.3). As with the out- While the Portfolio Re- Social infrastructure has come ratings, some of the larger CBD/CDD proj- view shows that CDD been difficult to sustain ects show higher sustainability compared with a projects provide for a beyond the Bank large number of smaller ones. greater role for commu- About 37 percent of the OED project assess- nities in O&M of sub- intervention. ments that inform this study rate sustainability projects than CBD/CDD as unlikely, and more than 30 percent rate it ei- projects (figure 2.2), the resource constraint re- ther nonevaluable or uncertain. Even those that mains an issue even in these projects.15 rate sustainability likely raise concerns either Several Bank projects require the setting up about maintenance of supported infrastructure of maintenance funds.16 The Benin country study or activities, as in the OED assessment for the found that commitment to maintenance funds Egypt Matrouh Development Project, or lack of is significantly reduced if communities know adequate analysis of the capacity of the govern- that they can go to another donor or to the gov- ment with respect to government contribution ernment when earlier investments cease to be during the project and post-project phase, as in functional.17,18The uncoordinated presence of the OED assessment of the Pakistan Northern Re- several donors and NGOs in the same villages in source Management Project.12 support of different activities can foster de- pendence and soften the commitment of com- Scarcity of Resources for Operations and munities toward maintenance activities.19 Maintenance Has Been a Constraint Fieldwork carried out for the OED assessment CBD/CDD projects have supported a large of the Indonesia Kecamatan Development Pro- amount of social infrastructure--such as schools ject found that the community approach is gen- and heath centers--at the community level in erally to postpone maintenance until it is many countries. These have been difficult to unavoidable--for example, the road is about to sustain beyond the Bank intervention. Mansuri become impassable, or the bridge is about to col- and Rao (2004, p. 32), quoting studies on water lapse, and then to do the minimum that will get projects in Sub-Saharan Africa and South Asia, it back into usable shape. OED's project assess- found that even if communities are initially suc- ment of the Mali Natural Resource Management cessful in creating the project, they may lack Project also found that a large amount of infra- the material resources to sustain their efforts. structure was constructed at the village level, OED project assessments also show that poor but there were few arrangements made at the communities find it difficult to raise resources in- community level for its maintenance. Hence, al- ternally to provide for continuous operation and though community members were aware of the maintenance (O&M).13 Governments, too, are fis- importance of maintenance and a significant cally constrained. number of them had been trained, the lack of A previous OED study (Kumar 2003) notes available resources on a continuous basis to that Bank projects typically do not plan for si- allow for O&M was a significant constraint. multaneous investment in social and produc- tive sectors. Doing so could ensure long-term Maintaining a Quality Flow of Services sustainability, because village-level capacity to or Income Has Proven Even More Difficult provide for O&M of social investments ultimately The World Development depends on increased capacity to generate rev- Report Making Services Bank projects typically do enue at the local level. Field research in Brazil Work for Poor People not plan for simultaneous found that most of the communities in the state (World Bank 2004d) ac- investment in social and of Rio Grande do Norte had only one subproject knowledges the chal- approved, largely because project municipal lenge of ensuring service productive sectors. Doing councils could not justify a second investment delivery.20 While it is ar- so could ensure long-term in any community before all communities had gued that services can be sustainability. been covered with at least a single investment.14 improved by changing 3 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT the relationships of accountability as with CDD, been negatively affected by the severe overall there is little evidence yet. A recent Bank re- shortage of doctors in the country. In contrast, view (Wassenich and Whiteside 2004) found that flow of services from a village water supply there is little evidence regarding CDD impacts scheme can be maintained with minimum tech- on quality of service delivery. OED project as- nical support from outside the village. Ironically, sessments and a review of ICRs show that most in several of the poorest countries where the CBD/CDD projects, including CDD projects, give institutional environment is the weakest, and little thought at the design stage to the issue of coordination among various government de- maintaining a quality flow of services or income.21 partments the biggest problem, Bank projects are Hence, even though infrastructure may be stand- supporting social infrastructure such as schools ing in a village or community, it is often under- and health centers. In contrast, in Brazil, In- used or not being used for the purpose that was donesia, and the Philippines, where the level of originally intended. Further, the flow of services institutional development is higher, Bank or income from an infrastructure requires con- CBD/CDD projects have provided support for ac- sideration of issues related to coordination with tivities that often do not require the same kind different government departments. It also implies of coordination and support from higher levels increased financial responsibility for the gov- of government, such as water supply schemes ernment in the form of supplemental resources, and roads.22 Since the focus is on the number of such as to pay the salaries of teachers and doc- schools and health centers built, the actual issue tors. Annex P draws on the experience of the of how the flow of services from these facilities projects in Benin to illustrate this point in the will be maintained receives less attention. case of school subprojects. Villagers May Also Not Have the Necessary And Some Types of Services Have Been Information or Technical Knowledge Much More Difficult to Maintain Than Others Project assessment findings show that commu- OED project assessments and country study nities often may not have the information and findings show that sustaining a quality service technical expertise they need to allow for main- flow from some types of infrastructure depends tenance. The OED assessment of the Community on the scale and complexity of the service. For Development Fund Project in Eritrea notes that example, quality flow of education services from in addition to ownership and willingness of dif- a school will require coordination with the ed- ferent actors to sustain investments, sustain- ucation department to ensure the availability of ability requires that the relevant actors have the certified teachers and books, as well as adherence financial, managerial, and technical capacity to to a centrally planned curriculum, among other operate and maintain the infrastructure. Focus things. Key informant interviews in Benin re- group sessions with villagers in Benin found that vealed that 50 percent of the teachers in schools several communities had not received adequate supported under the Borgou Pilot Project and 80 training to be able to maintain their subprojects. percent of teachers in schools supported under In Uttar Pradesh, where the Bank supported the Social Fund were not sodic land reclamation, villagers are interested Since the focus is on the state certified; in the in keeping the reclaimed land from reverting to number of schools and comparator schools, its former state, but the OED fieldwork found health centers built, the fewer than a third were that they may not have the technical knowledge actual issue of how the not certified (Annex P). or capacity to do so.23 Maintenance of main In Eritrea, the OED as- drains critical to the sustainability of reclaimed flow of services from sessment of the Com- lands could only be done by the Irrigation De- these facilities will munity Development partment. The implementing agency staff be- be maintained receives Fund Project found that lieved that the political pressure from the farmers health centers supported would be sufficient to ensure that the govern- less attention. under the project have ment provided adequate resources to the Irri- 3 8 INSTITUTIONAL ENHANCEMENT AND SUSTAINABILITY gation Department to maintain the main drains. in livelihoods. Hence, The recent emphasis on However, household survey data analysis found the projects provided empowerment and social that 96 percent of the farmers in the CDD com- for complementary in- capital has focused munities were unaware that the Irrigation De- vestments in communal much more attention partment had the responsibility for maintaining infrastructure to give vil- the main drains. Further, the household survey lagers an incentive to co- on the importance of analysis also indicated that most villagers were operate. The OED study understanding the rules not even aware of the critical importance of on forestry in India and regulations that drainage for containing sodicity (Annex P). notes: "The regenerated forest area can be kept govern behavior in Lack of Clear Communication about the under tree cover only if village society. Role of Communities in a Bank Project the FPC [forest protec- Has Added to the Problem tion committee] members get enough returns to As already indicated, villagers mainly see partic- compensate for the income forgone. This would ipation in a Bank project as a requirement for mean that JFM and the Economic Development them to meet part of the subproject cost. The Program have to be part of one strategy of en- project assessment of the Uttar Pradesh Sodic suring returns in the future. Currently this is Land Reclamation Project also found that the not the case" (Kumar and others 2000). Although community members' understanding of their in several states, the projects, when appraised, contribution to the participation process has were presented as the first phase of a long-term led them to expect support from the government operation to consolidate the JFM strategy, in for maintenance of drains that were critical to sus- several states (including Madhya Pradesh) the tainability. In a situation where the implement- Bank did not commit to funding a follow-on op- ing agency expects communities to take the lead eration, which considerably jeopardized sus- and the communities are not clear about this, sus- tainability of the effort already made. tainability issues are generally neglected. The assessment noted, "The farmers continued to Formal and Informal Organizations Are think of the reclamation activity as an `outside' Both Important in Determining Collective effort brought to them by UPBSN [implement- Activity at the Community Level ing agency] rather than something that they had The literature shows that, at the community level, to carry out on their own. It is understandable both formal and informal organizational systems then that they would think that the drainage influence collective activities. Community Asso- problem too would be `taken care' of by UPBSN ciations set up in Brazil under the Rural Poverty and the Gram Pradhan." Alleviation Program (RPAP) are a good example of formal organizations, because these groups And Sometimes Bank Interventions Have Failed need to be legally constituted before they can to Provide Consistent Support Long Enough for a participate in the Bank-supported project. Village Sustained Income Flow To Be Established societies also typically Project assessments and OED country studies have informal arrange- But customs and found that where the issue is a sustained flow of ments that determine income from a particular activity, such as forestry, how groups collectively conventions that could project support is often unavailable until the re- manage resources such be specific to a particular turns from the forest could allow for a sustained as water. These informal community and are flow of income from non-timber forest prod- arrangements are not ex- important in determining ucts and timber (Annex P). For example, in plicit rules or regulations, India's forestry projects, popular support for but are based on customs collective activities have joint forestry management (JFM) was contin- and conventions or what received inadequate gent on forging a link in villagers' minds be- people consider "the gen- attention. tween protection of the forest and improvements erally accepted way of 3 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT doing things" (Cleaver 1997, 1998; Tripp 2001) are created under a Bank intervention will affect (see Annex K). the informal organizations, customs, and con- ventions of a village society and how the inter- Yet Bank Projects Have Primarily Focused action of the formal and informal rules could on Formal Organizations influence empowerment. A review of project documents and evidence Since the formal and informal systems influ- from country studies shows that in Bank projects, ence each other, it could be argued that the for- the focus is primarily on formal organizations and mal arrangements created under a Bank manifestations of collective action, such as the intervention will influence and bring about ad- creation of a user group or committees and the equate changes in the informal arrangements holding of their meetings. The recent emphasis that are in keeping with the formal systems. on empowerment and social capital has focused However, as will be seen in Chapter 5, Bank sup- much more attention on the importance of un- port is rarely provided long enough to allow the derstanding the rules and regulations that gov- new structures to become an effective part of the ern behavior in village society, but customs and way the village operates. Project assessments conventions that could be specific to a particu- and evidence from focus groups (box 4.3) show lar community and are important in determin- that new structures established to implement ing collective activities have received inadequate Bank projects tend to fade away once the proj- attention. As a result, little thought appears to ect implementation period is over. have been given to how the formal structures that Box 4.3: Why Formal Groups Do Not Last Long "Only 7 out of 12 [committee] members participated in meetings. the field drains and connecting drains. On being asked why they Today, this number has been reduced to three. The other mem- don't repair the connecting drains the reply is, that we were bers of the committee said that there is no profit in their being paid to make them, no one has paid us to repair them." Uttar on the committee and for this reason they prefer to go deal with Pradesh, India their own affairs [fieldwork]." Benin Borgou Region (AgeFIB) "Why has our community association become inactive? ...our "No committee was set in the village to monitor or manage main target was to get the water system; we got it and the peo- the project. Only the [village committee] secretary had played ple stopped mobilizing so we were benefited with no other proj- some role. There has been no discussion as to community con- ects. Our target was to have water and then people stopped tribution and participation. Financial contribution was paid on mobilizing." Rio Grande do Norte, Brazil [committee] revenue while households were requested to con- "When the Samiti was formed, it had promised to start 7 vil- tribute with free labor, especially for fetching water for the build- lage-based organizations. They could remember just 4...Once ing." Benin Borgou Region (PAMR) these Samitis were started, they operated for 4 to 5 years but later "People do not even mention collective work anymore. Not they were dissolved... None of them exist today....One man said even meetings are any longer held. There is only a meeting when that there had been no meetings for the last 2 years." Madhya money is concerned." Rio Grande do Norte, Brazil Pradesh, India "[Maintenance of the infrastructure] is the job of the district government and the project people. No one has come to repair Source: Focus groups. 4 0 5 Bank Operational Policy Requirements, Processes, and CBD/CDD Interventions T his chapter assesses the extent to which internal policy requirements and processes position the Bank to support implementation of CBD/CDD, in particular CDD, interventions. Although the Bank's mis- sion is to fight poverty and improve the living standards of people in the developing world, it is also a lending institution, and its shareholders want assurance that funds provided by their taxpayers achieve expected results and that operations are economically, socially, and environmentally sound. Consequently, in addition to meeting efficiency tered, sometimes in remote locations with poor conditions (Chapter 3), all Bank projects need to communication. They involve multiple actors, meet two basic policy requirements--fiduciary, and many communities with varying socioeco- which govern the use of project-related funds, and nomic, cultural, and political backgrounds. Un- safeguards, to prevent unintended adverse ef- like the more traditional investments, the fects on third parties and the environment. subprojects under CBD/CDD are often not even known in advance. Moreover, in the case of The Bank Has Attempted to Adapt CDD, communities are also expected to control Its Policies to Design and Implement resources and decisions and be in charge of con- CBD/CDD Projects tracting for their implementation. These signif- Both fiduciary and safeguard policies were orig- icant differences have made it difficult to ensure inally developed for non-CBD/CDD projects that the compliance of CBD/CDD projects with fi- generally involved large-scale "lumpy" invest- duciary and safeguard policies that were devel- ments at specific locations, typically implemented oped for non-CBD/CDD investments. by a central government department or agency As a result, the Bank attempted to adapt its poli- that monitored and reported on resource use. cies to be able to design and implement CBD/CDD Bank missions supervised the investment site pe- interventions, while meeting the institution's fi- riodically and reported on resource use sys- duciary and safeguard obligations.1 This chapter tematically. A typical CBD/CDD project is very examines three issues that are pertinent to as- different. Each project includes numerous small sessing Bank capacity to implement CBD/CDD subprojects that are heterogeneous and scat- projects: first, whether CBD/CDD projects can 4 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT While the environmental pose a challenge for safe- of individual subprojects may be insignificant, and social impact of guard and fiduciary com- their cumulative impact can be substantial. The pliance; second, whether study also notes that too little attention is being individual subprojects adequate changes have paid in CBD/CDD projects to the environmental may be insignificant, been made in Bank and social consequences of changes in land use, their cumulative impact policies to effectively sup- especially for livestock, irrigation, and reforesta- port implementation of tion projects. Moreover, subprojects are often can be substantial. CBD/CDD projects; third, not small and may include investments--such as whether the Bank has the wastewater treatment plants in Poland or dams capacity to ensure effective implementation of in China and Brazil--with the potential for major CBD/CDD projects. negative environmental and social impacts.2 Fur- ther, as decision making is decentralized, there CBD/CDD Projects and the Challenge for is some danger that potential impacts, particularly Safeguard and Fiduciary Compliance in resettlement cases, may not be recognized and suitably mitigated. For example, it was only Compliance with Safeguard at Entry when OED carried out an assessment of the Has Improved over the Years Andhra Pradesh Forestry Project (1994) that cases A thematic study that reviewed the sample of 84 of uncompensated land were discovered. projects for their compliance with safeguards (Annex Q) found that such compliance at entry Wrong Environmental Category Assignment has improved over the years. In terms of Re- Can Have Serious Implications gions, all projects in Europe and Central Asia Typically, projects in areas such as health, edu- were found to be satisfactory. The Middle East and cation, nutrition, institutional development, tech- North Africa recorded the next strongest record, nical assistance, and human resources are placed followed by Africa, and East Asia and the Pacific. in environmental Category C (see box 5.1) be- The study also found that despite format changes cause they are considered to be unlikely to have in the Project Status Reports that encourage de- adverse environmental impacts, or that any such tailed reporting on the implementation of safe- impact would be minimal. However, health proj- guard measures, such reporting remains sparse ects supporting immunization programs, basic and inadequate. Further, the study found that the packages of drugs and syringes, and laboratory overall quality of implementation was rated sat- services for infectious diseases (including AIDS) isfactory for fewer than 40 percent of the cases. raise concerns about the safe collection, storage, In terms of Regions, South Asia and East Asia and disposal of medical waste. The thematic re- and the Pacific score highest for quality of im- view found that 9 percent of the projects classi- plementation of safeguard issues. These findings fied as Category B and 38 percent classified as at entry and implementation are disturbing given Category C had been misclassified--that is, proj- the Bank's current emphasis on full compliance ects had been classified as C or B when they with safeguard requirements. The thematic study should have been B or A. Assignment of Category also found that 6 of the 11 projects rated unsat- B induces a B mindset, which implies concen- isfactory on overall quality were in the Bank's trating attention on documentation to be pro- largest borrower countries. duced before Board approval, rather than on appraising the capacity of the project agencies However, Cumulative Impact of Subprojects Has to screen subprojects, analyze their potential Been an Issue for Safeguard Compliance impacts, and design and implement mitigation Some have argued that individual subprojects in measures, and on specifying the needed insti- CBD/CDD interventions are so small that they tutional strengthening and monitoring systems. cannot have a substantial negative social or eco- Assignment of Category C generally means that nomic impact. However, the thematic study found no further work is done to identify and mitigate that while the environmental and social impact impacts and there is no further review by safe- 4 2 BANK OPERATIONAL POLICY REQUIREMENTS, PROCESSES, AND CBD/CDD INTERVENTIONS Box 5.1: The Meaning of the Environmental Categories The Bank classifies proposed projects into one of four categories, vironmentally important areas--including wetlands, forests, depending on the type, location, sensitivity, and scale of the grasslands, and other natural habitats--are less adverse than project and the nature and magnitude of its potential environ- those of Category A projects. These impacts are site-specific; mental impacts. few if any of them are irreversible; and in most cases mitiga- tory measures can be designed more readily than for Category Category A: A project is classified as Category A if it is likely to A projects. have significant adverse environmental impacts that are sensi- tive (a potential impact is considered sensitive if it may be irre- Category C: A project is classified as Category C if it is likely to versible), diverse, or unprecedented. These impacts may affect have minimal or no adverse environmental impacts. an area broader than the sites or facilities subject to physical works. Category FI: A proposed project is classified as Category FI if it involves investment of Bank funds through a financial interme- Category B: A project is classified as Category B if its poten- diary, in subprojects that may result in adverse environmental tial adverse environmental impacts on human populations or en- impacts. Source: Bank Operational Policy 4.01--Environmental Assessment. guard specialists. For example, OED's project cialist consultants, a practice that would be valu- assessment of the Mali Natural Resource Man- able to adopt more widely, especially where agement Project also found that since the proj- there are several CBD/CDD projects in the same ect was wrongly categorized as a C, important country. The Bank is also currently exploring environmental issues were not given attention. ways to streamline the application of safeguard policies by delegation, both within the institution Attention to Safeguards during Implementation and to national authorities. Has Also Been Inadequate While quality at entry needs improvement, safe- Although Fiduciary Compliance Is guards compliance during implementation war- More an Issue for CDD Projects rants much greater attention by the Bank and Where the handling of resources in scattered sub- borrowers, and may indicate the need for greater projects remains in the hands of a central imple- allocation of supervision resources. The low per- menting unit, the fiduciary challenge is not very centage of moderately satisfactory or better Cat- different from that for a non-CBD/CDD project-- egory A projects appears to conflict with the that is, the need to set in place within the imple- expectation that A's would receive much more menting unit a system that can adequately monitor intensive scrutiny during supervision. The in- and report on resource use. The challenge occurs adequacy of funding to address safeguard issues when control over resources, and often pro- is strongly endorsed by Bank staff in the staff sur- curement responsibility as well, is transferred to vey. If resources for safeguard supervision were communities (often remote ones), as happens in increased, it would add to the already higher CDD projects. The coun- Safeguards compliance costs of supervision for CBD/CDD projects in try studies show that this comparison with non-CBD/CDD projects, with fiduciary challenge is during implementation efficiency implications. Some of the more re- likely to be greater in warrants much greater cent projects (such as Nigeria Community-Based countries where institu- attention by the Bank Poverty Reduction) have had special assessments tional capacity is weaker, of safeguards implementation, usually by spe- such as Benin, Nepal, and and borrowers. 4 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Box 5.2: The Fiduciary Challenge: The Case of Vietnam The Country Financial Accountability Assessment for Vietnam works under the International Development Association (IDA) notes the challenge for financial management at the subna- credit to monopoly state-owned enterprises. In the view of the tional government levels created by weak capacity, especially reviewers, this represented a possible failure of the Bank's en- in budgeting, accounting, and financial reporting.a With defi- tire project implementation system in Vietnam. The report ac- ciencies and irregularities in procurement as well (Vietnam knowledges that Bank staff must be attentive to situations where CPAR 2002), it is unclear how the Bank will be able to manage the Bank's policies may not apply or where there may be more the fiduciary risk in its ongoing portfolio of CDD interventions. An economical, efficient, or transparent ways of doing things, but, internal Bank review of the Quality of Supervision for a CBD/CDD in the end, Bank management is responsible for the environment project, the Coastal Wetlands Protection and Development proj- in which projects are conceived, prepared, negotiated, and im- ect (2000), identified the serious challenge that the Bank team plemented. This should be an environment in which both parties faces.b It reports that the Bank's procurement procedures were are focused on getting results they value. almost completely rejected by the client in favor of directing a. Concerns are already being noted by Bank staff on financial management in the supervision reports for the Community-Based Rural Infrastructure Project (2001) that started disbursement to communes in fiscal 2003. b. Management notes that the operational issues highlighted are not related to the CDD aspects of the project. Vietnam (box 5.2), than in middle-income coun- still do not report on community capacity to un- tries such as Brazil, Egypt, and Indonesia, where dertake fiduciary responsibility, even though the capacity to monitor resource use at the local level project envisages communities being responsi- may be greater. The country studies also show that ble for managing resources. It is thus unclear how the understanding and interpretation of fiduciary extensively capacity at the community level is ac- responsibility and accountability can be very dif- tually assessed before a Bank CDD project is in- ferent in local communities in several countries. troduced in a particular setting. The majority of In Benin's rural communities, for example, prop- Country Financial Accountability Assessments erty rights are poorly defined and enforced. Com- (CFAAs) and Country Procurement Assessment mercial exchanges are based on trust rather than Reports (CPARs) reviewed also do not report on enforceable contracts, and fiduciary rules are on community capacity. typically informal. In this kind of environment, ap- plying fiduciary management and accountability The Readiness of Country Financial Procedures through the Bank's rules may be more difficult and and Internal Control Systems to Support CDD costly than is envisaged in CDD projects. Is Not Given Adequate Attention While CFAA and CPAR documents can indicate Most Bank Documents Have Not Reported whether the existing processes in the country at on Community Capacity to Undertake Fiduciary the central and regional levels are strong enough Management Responsibility to monitor resource use, this information does Fiduciary sector work, in- not seem to be used to assess country readi- This fiduciary challenge ternal audit reports, and ness for CDD. For example, the documents on is likely to be greater project documents for 12 Nepal, Nigeria, Pakistan, and Senegal have raised CDD projects were re- concerns about the extent of capacity in these in countries where viewed to assess fiduci- countries to monitor resource use and to re- institutional capacity is ary compliance. The port on poor compliance with financial proce- weaker than in middle- review found that the dures and internal control systems.3 However, all majority of appraisal and these countries have several CDD projects either income countries. supervision documents ongoing or in the planning stages. While some 4 4 BANK OPERATIONAL POLICY REQUIREMENTS, PROCESSES, AND CBD/CDD INTERVENTIONS have argued that it is much more likely that re- veloped, with fiduciary Policies have still not sources provided through CBD/CDD interven- issues, application of the been completely refined, tions will reach the poor than those provided guidelines is a challenge. although CBD/CDD through non-CBD/CDD work, the concern is The guidance note states that, given the level of transparency and ac- that the Bank rules and lending is growing countability in several poor countries, unless guidelines apply to CDD rapidly. adequate processes are put in place, it is quite projects in the same way possible that despite the best intentions, these that they do for any other Bank project, but that resources may not actually benefit the poor.4 In their application needs to be adapted to the ca- addition to fiduciary sector work, the weakness pacity of the project and the community. Since in these aspects of public sector governance has there are capacity differences among commu- also been noted in other Bank-IMF documents.5 nities, fiduciary requirements need to be adapted The importance of adequate follow up to such to each project--if not to each community's ca- diagnostic sector work in the design and imple- pacity. The Bank's policy on fiduciary manage- mentation of CDD projects cannot be empha- ment for CDD projects leaves the decision about sized enough.6 It is worrying that several internal what communities are required to do for each audit reports have picked up lack of fiduciary CDD project's appraisal team. As a result, compliance as an issue in several CDD projects. whether and how this policy translates into sim- It is worth mentioning that some project su- pler procedures at local level is less clear, and the pervision reports have also raised concerns about simplification, if any, may be quite variable across fiduciary issues.7 projects. In response to the questionnaire on the sub- Changes in Bank Policies to Effectively ject in the staff survey--that task managers of Support CBD/CDD Projects CBD/CDD projects can monitor fiscal account- ability as satisfactorily as can managers of more More Progress Has Been Made on Refining traditional non-CBD/CDD projects--36 percent Fiduciary than Safeguard Policies agreed or strongly agreed, 28 percent were in the While it is to the credit of the institution that the middle, and 23 percent disagreed or strongly need to adapt its policies has been recognized, disagreed (Annex L). While on other questions policies have not yet been completely refined, al- this might be considered a reasonably positive though CBD/CDD lending is growing rapidly. response, on matters related to accountability, More progress seems to have been made on re- calling for some necessary minimum standard, fining the fiduciary policies (as a fiduciary refer- having approximately one-quarter of staff ence guide at least exists and was issued in May expressing concern about the ability of task 2002) than on safeguard policies. But it is im- managers to monitor suggests a significant portant to recognize that the procedure for de- problem. veloping adequate guidance on safeguards is complicated. The Andhra Pradesh forestry expe- Bank Capacity to Ensure Effective rience, and more recently that of the Indonesia Implementation of CBD/CDD Interventions Kecamatan Development project, shows the dif- Two issues are central to assessing Bank capac- ficulty of applying the Bank's safeguard policies, ity to ensure effective implementation of particularly resettlement policies, to situations CBD/CDD interventions. These are: the Bank's where it is not easy to determine what is a "loss" mode of operation and institutional organization for which a displaced person needs to be com- and the Bank's capacity pensated.8 The challenge for the Bank today is to to undertake adequate With fiduciary issues, ensure that safeguard violations are minimized monitoring and evalua- application of the while the new guidelines are being developed. tion of its operations. guidelines is a Although the thematic study found that guid- The following sections challenge. ance to staff on safeguard issues is still being de- deal with them in turn. 4 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Mode of Operation and · Fifth, the Bank's own supervision resources Institutional Organization allow missions to "inspect" only a very limited number of subprojects, and internal audit re- The Information Access Chain is Much Longer ports show that these are mostly in sites that and Has Greater Gaps in CBD/CDD Interventions can be easily visited. than in Non-CBD/CDD Interventions Regular "supervision" Bank Decentralization Has Increased the Capacity Borrowers almost missions were largely of the Institution to Track CBD/CDD Interventions universally said that adequate to ensure that Nearly 3,000 of the 10,000 Bank staff now live supervision by field implementation was pro- and work in client countries.9 With this decen- ceeding as planned in tralization to the field, the Bank is today better con- offices has helped them. non-CBD/CDD opera- nected with its borrowers than it in the past. tions. In CBD/CDD inter- During OED project assessment field missions, ventions, and particularly CDD, where the borrowers almost universally said that supervision implementation is being undertaken by commu- by field offices has helped them. Sixty-five percent nities, the information access chain for the Bank of Bank staff were somewhat satisfied or better is much longer and has larger gaps: with the impact of Bank decentralization to field offices on the efficacy of Bank support for par- · First, while the Bank relies on the borrower ticipatory projects. Only 13 percent were not sat- to provide reports, several layers of government isfied, suggesting fairly widespread satisfaction are involved. The extent to which the bor- with Bank decentralization. While decentralization rower is able to maintain accurate records de- has undoubtedly brought the Bank closer to the pends on its ability to coordinate among borrower, it can do little about the government different layers and get information from the and community side of the information chain community level. and about the Bank's own ability to monitor what · Second, the reliability and accuracy of the in- is happening in thousands of remote communi- formation from the community level depends ties. The staff survey raised concerns about the on the capacity of the communities to main- availability of adequate supervision resources to tain the records and of the local government be able to monitor compliance with safeguards. agencies or facilitators to monitor community As Chapter 3 indicates, the Bank's preparation and record keeping. supervision costs for CBD/CDD projects are al- · Third, it is almost impossible to expect the bor- ready higher than for non-CBD/CDD project, and rower to "supervise" and cross-check for ac- there are no additional incentives for country di- curacy and consistency the information that is rectors to provide the additional resources re- coming from thousands of remote communi- quired to prepare and supervise these operations. ties. The government has to rely for this ac- curacy on facilitators that may have a vested But the Sectoral Organization of the Bank interest in reporting that implementation is Continues to Handicap the Design and going well, since the facilitator's survival often Implementation of CBD/CDD Interventions depends on the availability of donor resources. Further, despite the attempt by Regions to · Fourth, given that there is no systematic way create multisectoral teams to coordinate of checking for cumula- CBD/CDD, the Bank itself is compartmental- The Bank itself is tive impact, and that base- ized, so integrated approaches across several line data are often not sectors have remained limited.10 Only 9 compartmentalized, so available, it is almost im- percent of Bank staff surveyed reported being integrated approaches possible to tell how, say, satisfied or very satisfied with coordination across several sectors the simultaneous digging within the Bank across sectors in CBD/CDD of 500 wells will affect the interventions. It is striking that this concern have remained limited. water table in a given area. about Bank coordination was actually greater 4 6 BANK OPERATIONAL POLICY REQUIREMENTS, PROCESSES, AND CBD/CDD INTERVENTIONS than staff concern about borrower coordina- For example, the OED as- Most projects, including tion. Only about 14 percent reported being sessment of the Borgou CBD/CDD, have given fully satisfied with the support from the current Pilot Project, a fiscal 1998 more attention to M&E matrix-management organizational structure intervention, shows that for CBD/CDD projects and about 27 percent monitoring indicators still capacity and indicators, were not satisfied (Annex L). A key finding of a track quantitative input but most indicators recent Regional review of Bank supervision of and output achievements continue to focus on CDD projects in the East Asia Region was that (such as days of training there was little collaboration between the provided, number of vil- quantity rather than Region's sector units in supervising CDD lages covered, and the quality. projects despite the multisectoral scope of like) rather than qualita- these operations (World Bank 2003c). tive progress toward achieving the primary proj- ect objective of improving the capacity of village Monitoring and Evaluation communities to better manage their socioeco- nomic environment. Similarly, the Yemen Third Monitoring and Evaluation Have Improved Social Fund and the recently approved Cameroon Over the Years Community Development Program APL have Since an Operational Memo in January 1996 large capacity building components and progress from the Operations Policy and Country Ser- toward the objectives should thus be measured vices (OPCS) unit provided guidance to staff on by qualitative and process-oriented indicators. preparing indicators, most projects, including Yet most of the outcome and impact indicators CBD/CDD, have given more attention to M&E noted in the report continue to be quantitative capacity and indicators, though there is variation and will be able to say little about the quality in quality across projects. A review of the sam- and impact of the capacity-building effort. How- ple of 84 projects found that M&E was very ever, on the positive side, there are examples of weak in most of the early projects (pre-1994), some projects, such as the India Andhra Pradesh and most of the indicators were output-related District Poverty Initiatives Project (2000), which rather than outcome-related. Some projects had make some provision for process monitoring in very little monitoring of any kind. For example, design and may be able to indicate improvement the OED assessment for the Turkey Eastern Ana- in capacity assuming adequate follow up. tolia Project, a 1993 intervention, notes the weak- Several projects, such as the CWPII in Albania, ness of M&E and the fact that the first baseline Kalahi CIDSS in the Philippines, Kecamatan De- was not done until 1998. In Egypt, the OED as- velopment Project in Indonesia, and the Third sessment of the Matrouh Resource Management Social Fund for Development in Yemen, are em- Project (1993) also noted that M&E started too phasizing participatory M&E to involve commu- late and analysis focused largely on number of nities in tracking progress on activities. This is adoptions rather than quantified impacts, so not not only likely to support the Bank capacity-en- much could be said about gains in productivity. hancing effort at the community level, but is also likely to help build greater ownership of Bank ac- But Little Has Been Done about tivities in communities, with positive implications Monitoring Capacity Enhancement for sustainability (Estrella and Gaventa 1998). There has been progressive improvement, and Some projects have attempted to combine par- more projects approved in later years (after fis- ticipatory M&E with other measures in a pluralis- cal 1995) have outcome and impact indicators. tic approach that could prove invaluable in tracking The Portfolio Review found that 95 percent of progress (box 5.3). But, given the difference in the these projects have indicators to monitor progress understanding of participation between the Bank and impact, compared with 50 percent in the and the borrower/community noted earlier, it re- earlier period. However, most indicators con- mains to be seen how effective participatory M&E tinue to focus on quantity rather than quality. will be in tracking progress on process issues. 4 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Box 5.3: Some Monitoring and Evaluation Features to Emulate: The Case of the Indonesia Kecamatan Development Project (KDP) The KDP project illustrates some features that an M&E system · Contracted, but independent, journalists to be another eye in for CDD should emulate: monitoring, which enabled the publicizing of corruption cases. · Contracted NGOs to monitor performance at the community · Involvement of beneficiaries in monitoring performance (for and local government levels (in most CDD projects NGOs are example, measuring quantities of materials delivered by con- the facilitators, in KDP they were only monitors). tractors). · A project-run public grievance system that generally re- · Public display of financial data at the village level on notice sponded quite quickly. boards so that all beneficiaries could see, monitor, and ques- · A series of related studies, such as an innovative quantitative tion. assessment of corruption in infrastructure (for example, · The establishment of relatively strong record-keeping sys- through core sampling of roads), a study of microfinance per- tems and bookkeeping skills at the village level (although formance, and studies of conflict problems. skills in the wrong hands can make corruption easier to · A general readiness to respond with new, quick studies as new hide also). performance issues arose. · A quite strong central government monitoring system. · Baseline studies, impact studies, and other studies on par- This said, the KDP system also has some weaknesses (for ex- ticular issues that were contracted out (though not all these ample, related to community process measurement, insufficient reports have been assessed for their quality). measurement of poverty and gender impact, and methodological · Qualitative and quantitative indicators measure physical problems with measuring impacts), which are planned to be ad- achievement, corruption, and conflict. dressed under KDP3, which became effective in January 2005. Source: Portfolio Review. Further, the preparation for most Bank projects be little, if any, systematic relationship between does not include establishing a baseline. The Port- evaluation capacity development activities and folio Review found that fewer than 10 percent of individual project-level M&E. the projects involve establishing a baseline against An ideal M&E system for CBD/CDD should be which to assess the impact of Bank intervention, able to do at least five things: instead relying largely on with-without compari- son with weak counterfactuals. Hence, even if · Tell whether adequate qualitative and quanti- studies are carried out, it is very difficult to tell tative progress is being made toward meeting whether there are any achievements. the project objectives. Furthermore, well-designed M&E is pointless · Tell whether the Bank resources are being if it is not effectively used. For example, the OED used effectively and efficiently. assessment of the Uttar Pradesh Sodic Land · Give some indication of whether progress is Reclamation Project notes that the latest moni- being made in reaching the poor and the poor- toring technology is available to the Remote est. Sensing Application Center in Uttar Pradesh, but · Provide information on safeguard and fiduci- the large amount of data being generated is not ary compliance. being used effectively. This is also because proj- · Give an indication of whether sustainability ect-related M&E procedures contribute little to can be ensured. systematically building · If the first five are not happening, it should pro- The preparation for evaluation capacity in the vide flags for mid-course corrections. most Bank projects country. Most project-re- does not include lated M&E effort comes Existing M&E systems in Bank CBD/CDD in- to an end when projects terventions are a long way from meeting these establishing a baseline. close. There appears to criteria. 4 8 6 Conclusions T his evaluation of the Bank's support for CBD/CDD interventions in client countries supports four broad findings. The Bank has not, until recently, system- of CDD projects, is not uniformly understood atically identified and tracked its portfolio within the Bank or, even more important, be- of CBD/CDD projects, and therefore has tween the Bank and its clients. Regarding com- lacked a comprehensive understanding of munity participation, the surveyed beneficiaries the evolution and scope of its work in com- appear to have a very limited, pragmatic under- munity development. It also has not been standing of the concept that differs significantly sufficiently clear about the objectives of from the Bank's intent. using CBD/CDD approaches, criteria for choosing among different community de- The Bank's structure and mode of opera- velopment approaches, or about how to tion limit its ability to ensure sustainable measure the results. outcomes from CBD/CDD projects. This Although the Bank has been involved in limitation has become much more appar- CBD/CDD for a long time, a database of projects ent since the institution began emphasiz- using community development approaches was ing CDD in the late 1990s. only established for projects approved from 2000 It is easier for the Bank to monitor resource onward. Overall development effectiveness can use and be in compliance with safeguards in be assessed only on a clearly defined portfolio. non-CBD/CDD investments such as bridges or a Furthermore, effectiveness can only be assessed power plant than where small subprojects are against clear objectives, preferably with clear in- being implemented by hundreds of remote com- dicators against which to judge success. Differ- munities in scattered locations. In CBD/CDD ent community development approaches have projects, and more so in CDD ones, the critical varying community capacity requirements, so it challenge that the Bank faces is that the process is important to ensure that Bank staff, as well as must be managed "close to the ground," but the Bank's clients and the ultimate beneficiaries, normally without direct Bank involvement at understand the expectations under the project. the local level. As a result, with its mode of op- For example, the concept of empowerment, a eration, distance from implementation, and its major justification for the most recent generation current monitoring and evaluation system, the 4 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Bank has found it difficult to ensure safeguard activities, during project implementation rel- compliance and sustainability of development atively greater importance has tended to be outcomes from its CBD/CDD projects. given to achievement of material development goals. This raises concerns about whether the The Bank's support for CBD/CDD has pro- Bank is using CBD/CDD as a means for facili- duced different, though systematic, result tating an investment program rather than for patterns depending on local political and sustainably improving community decision social conditions, government commit- processes. ment, and community capacity. · Calculation of the costs and benefits, including Bank-supported CBD/CDD projects have had the long-term poverty impact, of undertaking much more success, particularly regarding ca- the CBD/CDD approach as a basis for com- pacity enhancement, in supporting indigenously parison with alternatives. The Bank has not matured participatory efforts or where it has systematically and realistically assessed the dis- provided consistent long-term capacity-build- tribution of costs and benefits of undertaking ing support to communities over time. How- CBD/CDD projects to the institution, the bor- ever, most projects make little effort to tailor rower, and the communities. The insufficient capacity building to community capacity or to go focus on costs and benefits, especially meas- back to the same communities with a consis- ures of poverty impact, in CBD/CDD projects tent capacity-building strategy; the one year of has prevented convincing comparisons with a typical subproject cycle is sufficient to allow suc- more traditional investments and policy and in- cessful subproject execution, but not to consis- stitutional reform programs. tently have a significant positive impact on · Focus on sustainability and long-term devel- community capacity; and communities do not opment. Project experience indicates that in appear to have understood that their participa- a number of cases there has been a lack of ad- tion is meant to drive the development process, equate follow through of activities supported and see participation in a Bank project primarily by Bank projects in order to address and min- as a requirement for them to meet part of the imize risks to long-term outcomes. In other subproject cost. cases, the ad hoc parallel arrangements made to implement Bank projects have hindered To effectively support CBD/CDD projects, the long-run enhancement of local govern- and especially CDD ones, the Bank will ment capacity. not only need to carefully consider its own · Addressing constraints related to the Bank's capacity but also to assess borrower com- mode of operation, its operational policies, mitment, community capacity, and the and its monitoring and evaluation systems. costs and benefits of the alternatives avail- For individual communities, the Bank's sub- able. Four issues need special attention project cycle is generally too short to bring when future CBD/CDD projects are con- about the kind of enhancement of commu- sidered: nity capacity that is visualized in Bank-sup- ported CBD/CDD, particularly CDD projects. · Clear articulation of expected achievements Further, Bank processes and systems have not of CBD/CDD interventions. While the design been geared toward supporting long-term of CBD/CDD projects has emphasized both processes such as empowerment and social material development and capacity building capital enhancement. 5 0 7 Recommendations iven the mixed and limited evidence on the impacts of CBD/CDD proj- G ects--particularly in poverty reduction and empowerment--and questions about sustainability and safeguard and fiduciary compliance, the Bank should approach future CBD/CDD projects, particularly CDD ini- tiatives, with greater care. In countries where the Bank is already supporting a CDD program, the institution needs to rigorously assess the poverty and in- stitutional development impact of its projects before scaling them up. A cau- tious approach would be especially important in countries or areas where the Bank is just beginning to support CDD. In its future assistance to CBD/CDD, the Bank should: At the corporate level, strengthen opera- · Future CASs should show how they have ana- tional guidance and management over- lyzed and addressed linkages not only between sight. various CBD/CDD projects to be undertaken · The Bank should provide operational guid- in the country but also between CBD/CDD ance for the application of Bank safeguard and relevant non-CBD/CDD projects. In par- policies and fiduciary oversight of CBD/CDD ticular, the analysis should address whether projects and for the strengthening of cost-ben- arrangements for CBD/CDD project imple- efit analysis and M&E systems; and should mentation come at the expense of local gov- commission an audit of the fiduciary aspects ernment capacity development. of a representative sample of CDD projects for submission to the Board within a year. At the project level, the Bank should give priority to helping countries build up ex- At the country level, design the CBD/CDD isting indigenously matured initiatives; program as an integral part of the overall where there are no such existing initia- assistance strategy and carry out periodic tives, the Bank should tailor its project to assessment of its ongoing CBD/CDD proj- the country and community context, while ects to ensure relevance and effectiveness undertaking selective rigorous impact as- of the program to the country context. sessments to ensure learning. 5 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT · For any new CBD/CDD project, the Bank tailor the intervention to local capacity; and the should analyze (using existing processes, such Bank should also selectively undertake rigor- as social assessments) whether it is building on ous impact assessments upon completion of its indigenously matured initiatives or attempting ongoing CBD/CDD projects to learn for the fu- to begin a CDD program in a country and then ture. 5 2 ANNEXES ANNEX A: DEFINITION OF "COMMUNITY" The Community tiveness of participatory approaches. The poor themselves are rarely a homogenous group; they All Bank participatory projects exhibit three live in different geographic areas and face dif- basic assumptions about communities (from ferent kinds of deprivations, and each seeks a OED 2003): personalized way of reducing poverty (Schnei- der 1999). · They comprise a group of people who share The shared norms that are expected to unify broad development goals. the community can themselves hinder commu- · Their social behavior and relationships are gov- nity action (Western and Wright in Agarwal and erned by social norms that are expected to Gibson 1999). Such norms may dictate patterns provide solidarity. of behavior, such as deference to the elite, which · By extension, those who do not belong to that do not allow the poorest and the marginal to ef- community are "excluded." fectively demonstrate their choice. Moreover, participation may lead to significant psycholog- The "community" in this approach is often ical and even physical duress for the most socially considered a "unified, organic whole" (Agrawal and economically disadvantaged, typically the and Gibson 1999). Since the group of people in prime potential beneficiaries of CDD projects, a "community" live in a particular area, share a since genuine participation may require them to common interest (water users associations, take positions that are contrary to the interests herders, and the like), and are governed by a set of more powerful groups (Mansuri and Rao of norms, its members are assumed to be in the 2004). best position to identify their most pressing Participation starts a process of institutional needs and problems. change in communities. A distinction between The latter idea suggests that there are com- formal and informal "rules of the game" and or- mon problems that can be solved through com- ganizations is essential to understanding this munity consensus. While this may be true, it change process. Many crucial decisions in a vil- neglects community members' differences and lage community are made not through formal power relationships, the conflicts, and the di- committees and groups, but through informal or- versity of interests that determine day-to-day ganizations that vary from community to com- behavior and that have an impact on the effec- munity. 5 5 ANNEX B: RESULTS CHAIN FOR WORLD BANK CDD PROJECTS1 How are CDD interventions expected to work that links inputs to impacts through outputs in a Bank project? Since the 1992 Wapenhans and outcomes. report, the World Bank has tried to increase the Within this framework, the principal impact results focus of its operations to track the of a CDD approach is expected to be sustained progress of Bank interventions, including CDD. development and positive impact on the lives of The relationship among Bank inputs, outputs, the poor. Underlying this is a hypothesis that em- expected outcomes, and impacts of CDD oper- powered communities (outcome) can partici- ations are shown in figure B.1. The arrows in- pate in decision making, create and implement dicate the direction of the results-based chain their own development plans, and hold ac- Figure B.1: The Results Chain in a Bank-Supported CDD Intervention Output Outcome Increased access of Empowered communities communities to basic infrastructure, services, and Community control and income-generating activities management of decisions and resources Favorable policy and legal environment and strong local government institutions Input Impact Resources for financing Sustained development capacity building at the and positive impact on community level, for lives of poor preparing and implementing development plans according to community priorities Support for strengthening legal and policy environment and local government institutions 5 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT countable the institutions that affect their lives. services and income-generating activities, a fa- This is expected to allow for improved effec- vorable policy and legal environment, and tiveness and targeting of development inter- stronger local government institutions. Bank in- ventions, which in turn is expected to promote terventions--through resources for financing sustainable development. However, this can hap- capacity-enhancing efforts at the community pen only if benefits are not captured by the elite level, resources for preparation and implemen- and donor support is available over a defined pe- tation of development plans, support to the riod to allow elements of sustainability to be country for improving the legal and policy en- built. The major outputs would be increased ac- vironment and for strengthening local govern- cess of communities to basic infrastructure and ment institutions are the inputs. 5 8 ANNEX C: WORLD BANK GUIDANCE ON KEY DESIGN PRINCIPLES FOR CDD The CDD Anchor has identified 10 principles to meet their needs and fit local conditions, cul- guide policy formulation and program design ture, values, and available operation and main- to enhance the effectiveness and sustainability tenance capacity. of support to CDD.1 · Community contributions to investment and recurrent costs. Community co-financing has 1. Establish an enabling environment been shown to be an important factor in build- through relevant institutional and policy ing ownership and in helping to ensure that ap- reform. CDD involves more than strengthening propriate choices are made and that community-based organizations (CBOs) and fund- investments are sustainable. People seem to ing their projects--it also requires active measures make better choices when they have their own to establish an appropriate enabling environment. resources at stake and when opting for a more Large programs of support to CDD will not be sus- expensive option implies a proportionally tainable without the policies, laws, systems, and higher cost. governance processes that encourage effective collaboration among local governments, central 3. Build participatory mechanisms for com- governments, civil society, service providers, and munity control and stakeholder involve- CBOs. Specifically, such an environment should ment. Communities that have ownership of a include: (a) elected local governments that are re- project or program are more likely to sustain sponsive to constituents and are empowered to outcomes. This implies providing inclusive com- serve them; (b) intergovernmental arrangements munity groups with knowledge, control, and au- for fiscal flows to local governments and CBOs; thority over decisions and resources throughout (c) a conducive legal and regulatory framework all phases from program inception. Programs that supports community action; and (d) clear sec- should be designed to engage relevant stake- tor policies with well-defined financing rules and holders (government, local leaders, NGOs, civil defined roles and responsibilities of key players society, the community) at the earliest opportu- in each sector. nity and dynamically over time. Political will--gar- nered through broad-based support and/or 2. Make investments responsive to informed "political champions" to drive necessary re- demand. Enabling communities to be involved forms--have played critical roles in the scaling up in decision making is not sufficient to achieve of many existing CDD programs. Broad stake- sustainable outcomes. Decisions need to be based holder participation helps tap into local techni- on accurate information about the costs and ben- cal and financial resources in support of efits of various options, and communities need to community initiatives. It also ensures that local have some of their own resources invested. knowledge and preferences are incorporated into the project design. · Informed, meaningful choice. Communities and stakeholders should have access to suffi- 4. Ensure social and gender inclusion. Com- cient information to weigh tradeoffs and make munity-driven development has the potential to realistic choices from a range of options that increase the power of poor communities to ne- 5 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT gotiate with government, the private sector, and 7. Develop simple rules and strong incen- civil society. But to fulfill this potential, CDD tives, supported by monitoring and evalu- needs to be responsive to the priorities of all ation. Experience indicates that sustainability poor groups. Communities are not homoge- and effectiveness of CDD is enhanced when neous; thus CDD needs to be designed to be processes are simple and transparent and when socially inclusive, giving voice and decision-mak- actors have strong and consistent incentives for ing responsibility to women, the elderly, youth, performance. Regular monitoring and evaluation religious and cultural minorities, indigenous and then provides the necessary information to ensure other ethnic groups, those with HIV/AIDS, and that the integrity of the system is maintained. the disabled. When community-driven develop- ment does not pay attention to issues of social · Simple rules. Community access to resources inclusion, groups of poor people may be ex- needs to be governed by simple rules that are cluded, investment choices may not reflect the easy for participating communities to inter- true needs of the poor, and impacts may be sig- pret and apply. To maintain the credibility of the nificantly compromised. system, these rules should be monitored and transparently enforced. 5. Invest in capacity-building of CBOs. The · Strong performance incentives. Key ac- lasting impact of CDD programs depends on the tors at all levels should be rewarded for per- capacity of CBOs to provide services and goods formance through objective evaluation based on a sustainable basis, often in partnership with on clear criteria. responsive formal institutions. Capacity building · Regular monitoring and evaluation. Sys- of CBOs, and strengthening linkages with formal tematic monitoring and evaluation of program institutions, is a critical area for investment. The processes and outcomes is critical for ensuring impact of CDD programs is directly related to that programs continue to grow and adapt to the strength of the CBOs driving the process. Ex- changing conditions. perience and studies have shown that those CBOs with clear lines of responsibility, open decision- 8. Maintain flexibility in design of arrange- making processes, and direct accountability to ments. Flexibility in design, often through pi- the community improve service provision, make loting, is essential to allow systems to evolve and more effective use of resources, and are more sus- better adapt to local demand and capabilities. tainable. Flexible program planning and decentralized de- cision-making mechanisms, situated as close to the 6. Facilitate community access to informa- community as possible, facilitate quick response tion. Support to CDD is as much about facilitat- to change. For example, in Zambia, the Social Re- ing flows of information among all groups in a covery Program is experimenting with more direct community as it is about facilitating flows of funds. capacity building and integration of local gov- The lack of information is often the most signif- ernments into the project cycle. In both the icant limitation on CBOs' capacity to play a part Moldova and Albania Social Investment Funds, in the development enterprise--community or- the initial pilot phase was extremely important to ganizations need information on market oppor- work out operational procedures before the pro- tunities, on what support resources are available, gram was offered nationwide. As part of this learn- and on how to use these resources productively ing process, direct feedback from the community and efficiently. A variety of media may be used to on program performance is essential. facilitate access to and stimulate flows of infor- mation. Information technology and the inter- 9. Design for scaling-up. Despite the many net, adapted to community needs, are playing a islands of success in community-driven devel- growing role in this process and can dramatically opment, most countries still have significant op- accelerate local learning and connections with a portunities for scaling up CDD. To have a material wide range of opportunities. impact on macro indicators of poverty, CDD 6 0 A N N E X C needs to take place in many communities si- CDD interventions. A clear distinction must be multaneously. It is no longer acceptable to design made between support services that are recurrent CDD as small, non-replicable, isolated interven- or permanent in nature and those that are tem- tions. However, the challenge of scaling up is porary. For recurrent services, sustainability re- not about bigger projects or bigger organiza- quires putting in place permanent institutional tions, but rather about achieving sustainable re- and financing arrangements at a cost that can sults in a large number of communities. The be supported over the medium- and long-term. section entitled "Scaling Up" provides more de- Temporary services, such as initial intensive ca- tail and links to documents and sites with more pacity-building support to community-based or- information. ganizations, may, however, not require sustainable financing or permanent institutional structures. 10. Invest in an exit strategy. An exit strategy For such temporary services, explicit exit strate- for external support is a critical component of all gies need to be designed and implemented. 6 1 ANNEX D: STUDY FRAMEWORK, METHODS, AND INSTRUMENTS Study Methods and Instruments · Reviewed 33 OED ICR Reviews.2 The study had four components: a Portfolio Re- · Reviewed 19 OED Assessments. view, country case studies, a Literature Review, and thematic studies. Country Case Studies Portfolio Review The country case studies were undertaken to The Portfolio Review was a desk study of complement the portfolio review. The five coun- CBD/CDD projects (and project components) try studies include two middle-income coun- supported by the Bank between fiscal 1989 and tries, Brazil and Egypt, and three low-income fiscal 2004. The study: countries, Benin, Nepal, and Vietnam. The se- lection of case study countries was done to pro- · Identified all International Bank for Recon- vide an opportunity for pairing of middle- and struction and Development (IBRD), IDA, and low-income countries. This provided a basis for Special Financing CBD/CDD and non- comparing the performance of CBD/CDD in- CBD/CDD lending approved Bankwide be- terventions in countries where institutions are tween fiscal 1989 and fiscal 2003 using the relatively more developed, where literacy levels Bank Business Warehouse database. (Annex E are relatively higher, and where the policy and notes the methodology used to identify the 847 legal environment is stronger with countries CBD/CDD projects.) that have less developed institutions, lower lev- · Examined CBD/CDD-related nonlending ac- els of literacy, and a weaker enabling environ- tivities--economic and sector work in each ment for CBD/CDD interventions. These country case study country, relevant participatory case studies also provided an opportunity to poverty assessments, beneficiary assessments, gain in-depth understanding of participatory local-level studies of institutions, and other approaches supported by the Bank in client formal and informal sector work. countries and to provide national perspectives · Reviewed project documents--appraisal doc- from a range of stakeholders on the appropri- uments (PAD, SAR), Project Status Reports ateness of the approach to development. All (PSRs), Aide Memoires, Implementation five case studies--Benin, Brazil, Egypt, Nepal, Completion Reports (ICRs), and Operational and Vietnam--involved desk reviews plus visits Manuals--for a sample of 84 CBD/CDD to the country, interviews and surveys of central projects. government officials and other international · Reviewed 73 CASs and 29 PRSPs and poverty donors, and focus group sessions with NGOs. sector work for a number of countries cov- The Egypt, Nepal, and Vietnam case studies in- ered by the sample of 84 projects. (See Annex volved limited visits to relevant project sites to H for details.) meet with communities and hold focus group · Reviewed six recent CDD projects (one in each sessions. In Benin and Brazil, extensive house- Bank Region) to better assess the attributes of hold-level fieldwork was undertaken in approx- the most current CDD projects under imple- imately 30 communities involving 1,200 mentation in the Bank.1 household surveys, 60 focus group sessions, 6 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure D.1: Study Framework Community-Level Bank Project-Level Key Evaluation Key Evaluation Concerns Concerns Have Bank-supported subproject Input How well are objectives of interventions been relevant to Resources for CBD/CDD interventions derived community priorities? capacity building, from the country's overall development priorities? basic infrastructure, and services To what extent are CBD/CDD interventions relevant to borrower priorities? Does the participatory process ensure the representation of the whole community? Output Basic infrastructure and services created, more Is the CBD/CDD approach the most To what extent are communities income-generating activities, efficient way to create this satisfied with the nature of the favorable enabling environment infrastructure for the country? subprojects financed? and local institutions strengthened Are the subprojects being Have adequate forward and maintained and are they being backward linkages been adequately utilized? established for income-generating activities? Is there sufficient ownership of Has CBD/CDD built capacity & social the CBD/CDD intervention in capital at the community level? Outcome the government? Empowered communities, community control and Is there harmonization of donor Do communities have greater management of approaches? access to information and are they decisions and resources aware of development activities? Have sustainability issues been addressed during project design and implementation? Is there sufficient ownership of the CBD/CDD intervention in the community? How far have CBD/CDD interventions Is CBD/CDD seen as an improved the institutional capacity important poverty alleviation tool Impact of the communities to take charge by the borrower? of their own development? Sustainable development and poverty alleviation How far have the stated goals of CBD/CDD projects been directed Have Bank-supported at poverty alleviation? interventions had an impact on the living standards of the poor? Have the poor been specifically targeted? Source: From the "CDD Evaluation Design Paper": http://www.worldbank.org/oed/cbdcdd/documents/discussion_paper.pdf. 6 4 A N N E X D and 60 key informant interviews with local gov- (iii) to explore the evidence on factors that ernment officials. (See box 1.1 of the main re- have a bearing on the development effectiveness port and Annex M for details.) of CBD/CDD­type interventions; (iv) to pro- Similar household-level fieldwork was un- vide a means for "testing" the validity of findings dertaken in two states in India--Uttar Pradesh emerging from other study components, par- and Madhya Pradesh. (See box 1.1 in the main ticularly case study countries and the Portfolio report and Annex M for details.) The two proj- Review. ects covered were Uttar Pradesh Sodic Land OED also participated in or attended brown Reclamation project and Madhya Pradesh bags, seminars, and other training events around Forestry. the Bank on issues relevant to CBD/CDD and drew on relevant information disseminated at these events. Thematic Studies Two thematic studies were undertaken to in- vestigate issues that could not be adequately Project Assessments of Participatory addressed in either the Portfolio Review or the Assessments case studies: Nineteen project assessments informed the study: Uttar Pradesh Sodic Lands Reclamation · The Bank Capacity study reviewed Bank doc- Project (India); Borgou Pilot Project (Benin); uments and interviewed and surveyed Bank Household Energy Project (Mali); Natural Re- staff (152 completed surveys analyzed of 400 source Management Project (Mali); Eastern Ana- mailed to the relevant group of staff). Annex tolia Watershed Rehabilitation Project (Turkey); L presents the main results of the survey. Matrouh Resource Management Project (Egypt); · The Safeguards study reviewed project ap- Nepal Hill Community Forestry Project; Nepal praisal documents, ICRs, and a limited number Second Forestry Project; West Bengal Forestry of supervision reports for the 84 sample proj- Project (India); Kerala Social Forestry Project ects to gather information related to safeguard (India); Ghana Agricultural Sector Investment compliance. Relevant safeguard literature was Project; Second Village Infrastructure Project also reviewed, as were a limited number of (Indonesia); Kecamatan Development Project OED assessments. (Annex Q). (Indonesia), Andhra Pradesh Forestry (India), Northwest Frontier Province Community Infra- In addition, a small number of projects from structure Project (Pakistan), Community Devel- the portfolio were reviewed specifically for their opment Fund (Eriteria), and Rural Roads fiduciary compliance. A limited number of Coun- Rehabilitation and Maintenance Project (Peru), try Financial Accountability Assessments (CFAAs), Northern Resource Management Project (Pak- Country Procurement Assessments (CPARs), and istan). Internal Audit Department reports were also re- viewed. Review of All Relevant OED Work All relevant Country Assistance Evaluations, Im- Literature Review and Associated Events pact Evaluations, and studies were reviewed, in- on CBD/CDD around the Bank cluding: The Next Ascent: An Evaluation of the The Literature Review had four objectives: (i) to Aga Khan Rural Support Program, Pakistan; So- gather qualitative, quantitative, and anecdotal cial Funds: A Review of World Bank Experience; evidence on participatory approaches to local India's Dairy Revolution; An OED Review of So- development; (ii) to draw on the evidence in the cial Development in Bank Activities; Non- literature to understand the different kinds of governmental Organizations in Bank-Supported "participatory spaces" that the Bank's CBD/CDD Projects: An OED Review; a participation process interventions have fostered at the local level;3 review; Books, Buildings, and Learning Out- 6 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT comes: An Impact Evaluation of World Bank Interviews with Bank Staff Support to Basic Education in Ghana, World Supplemental interviews were conducted with Bank Forestry Strategy, Striking the Right Balance, Bank staff working on CBD/CDD and related is- and associated country studies. sues to get their views on various aspects of the Bank's work. 6 6 ANNEX E: THE UNIVERSE OF CBD/CDD PROJECTS AND ITS DISTRIBUTION Universe search, it is likely that some projects with very The universe of CBD/CDD projects was identi- small CBD/CDD component were missed.2 fied using a key word search on a textbase of ap- The total number (and commitment) of Bank praisal documents (Project Appraisal Documents, projects that include a CBD/CDD component or PADs, and Staff Appraisal Reports, or SARs) for has increased substantially overtime (figure E.1). all Bank projects approved between fiscal 1989 and fiscal 2003.1 A total sampling frame of 847 projects was identified. The population of 847 (as Distribution of the CBD/CDD Portfolio of September 2004) includes projects that are Regional. Africa had the largest number of largely CBD/CDD and others with CBD/CDD CBD/CDD projects approved between fiscal components (a complete list of the CBD/CDD 1989 and fiscal 2003 (266 projects, 31 percent), portfolio as identified is available upon request). followed by Latin America and the Caribbean Since the portfolio was identified using a word (193 projects, 23 percent). South Asia, East Asia Figure E.1: Commitment and Number of CBD/CDD Projects Have Increased from Less than 5 Percent to 25 Percent of Bank Totals 6,000 100 90 5,000 80 70 4,000 60 projects US$ of 3,000 50 million 40 Number 20012,000 30 20 1,000 10 0 0 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 Commitment Projects Source: World Bank data. 6 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT and the Pacific, and Europe and Central Asia Table E.1: A Majority of CBD/CDD had 110 projects (13 percent), 118 projects (14 Projects Are Multisectoral (percent) percent), and 94 projects (11 percent), respec- tively. The Middle East and North Africa Region Fiscal years had the smallest portfolio with 66 projects (8 1989­ 1994­ 1999­ 1989­ percent). 93 98 2003 2003 CBD/CDD 43 51 53 51 Sector Board. The Rural Development (RDV) Non-CBD/CDD 28 31 33 31 Sector Board had the largest number of Source: World Bank data and calculations. CBD/CDD projects approved between fiscal 1989 and fiscal 2003 (226 projects, 27 percent). The Health Sector Board (HE) followed with 135 Sector. While the sector board under which a projects (16 percent) over the same period. So- project is categorized manages the project, each cial Protection (SP) was a close third with 131 project is also assigned, at most, five subsec- projects (15 percent). The other important sec- tors. The number of projects assigned two or tor boards for CBD/CDD projects were Education more sectors (multisectoral operations) has been (111 projects, 13 percent), Urban Development increasing over time (table E.1). The percentage (61 projects, 7 percent), Water Supply And San- of multisectoral projects has been rising for the itation (53 projects, 6 percent), and Environ- non-CBD/CDD portfolio as well. However, the ment (43 projects, 5 percent). percentage of multisectoral projects is much Figure E.2: Distribution by Lending Instrument 80 71.43% 70 60 50 40 Percentage 30 20 10.74% 10 6.97% 6.38% 1.53% 1.18% 1.06% 0.35% 0.12% 0.12% 0.12% 0 SIL APL SIM LIL ERL TAL FIL SAD PRC SAL SSL Lending Instrument Source: World Bank data and calculations. Note: APL=Adaptable Program Loan; ERL=Emergency Recovery Loan; FIL=Financial Intermediary Loan; LIL=Learning and Innovation Loan; PRC=Poverty Reduction Support Credit; SAD=Sector Adjustment Loan; SAL=Structural Adjustment Loan; SIM=Sector Investment and Maintenance Loan; SIL=Specific Investment Loan; SSL=Special Structural Adjustment Loan; TAL=Technical Assistance Loan. 6 8 A N N E X E higher for the CBD/CDD portfolio compared to approved have used either an APL or a LIL, com- the non-CBD/CDD portfolio for the period (fis- pared to 13 percent of the non-CBD/CDD proj- cal 1989­2003). ects. Lending Instrument (figure E.2). Of the 847 Income Category. The portfolio of 847 projects CBD/CDD projects, 841 are investment lending. was distributed among 119 countries: 60 in the The instrument chosen for 605 Bank CBD/CDD lower-income category, 38 in the middle-income projects was the Specific Investment Loan (SIL). category, and 18 in the upper-middle-income Lending instruments deemed to allow for category (figure E.3A). Three countries/territo- greater flexibility, APLs and LILs, were adopted ries in the portfolio, Barbados, Kosovo, and West in 91 projects and 54 projects, respectively. The Bank, had unspecified poverty categories. Of percentage of lending channeled through the the 847 projects, 503 were in low-income coun- two instruments has increased over time (the in- tries, 230 were in middle-income countries, 101 struments were introduced in 1997). Since fis- were in upper-middle-income countries, and 13 cal 1999, 32 percent of CBD/CDD projects were in the unspecified category (figure E.3B). Figure E.3: Distribution of the Portfolio Countries/Projects by World Development Indicator (WDI) Poverty Category A: Portfolio countries (119 countries) B: Portfolio projects (847 projects) Middle income Lower income Lower income Middle income 31.93 59.39% 50.42% 27.15% Upper-middle Upper-middle Unspecified income 15.13% Unspecified income 11.92% poverty category poverty category 2.52% 1.53% Source: World Bank data, SIMA, and calculations. 6 9 ANNEX F: SAMPLE OF CBD/CDD AND CDD PROJECTS According to accepted sampling methodology, a 10 percent proportionate random sample of 84 Table F.1: CBD/CDD Projects Approved by the Bank projects, stratified by time and sector board, was drawn from the universe of 847 projects for in- Sample Universe tensive review. The stratification was done to CBD/CDD projects 84 847 ensure that important characteristics in the uni- CDD projects 19 192 [= 19/84*847] verse of 847 projects were adequately repre- sented in the sample. (See table F.3 for the list of CBD/CDD sample projects.)1 The CDD Anchor classifies CDD projects in was randomly selected) implies that the World fiscal year 2000 and beyond into four categories Bank has approved close to 192 CDD projects in that are not mutually exclusive. It was difficult the fiscal years 1989­2003 period (table F.1). to apply this classification to projects approved Some characteristics of the sample of in the early 1990s, when the four kinds of CDD CBD/CDD and CDD projects are presented in had not been identified. Hence, OED devel- table F.2. CDD projects have grown at an an- oped a simple methodology to categorize the nual rate of 19.6 compared with 11.7 for the projects into two broad groups: CBD/CDD CBD/CDD projects (excluding CDD). Nearly and CDD based on percentage of project cost three-quarters of the CDD are multisectoral, (box F.1). compared to 40 percent of the CBD/CDD proj- In the sample of 84 projects, 19 (23 percent) ects (excluding CDD). Over 80 percent of the were identified as CDD. Extrapolating this pro- CDD projects are under two sector boards--So- portion to the entire universe (since the sample cial Protection and Rural Development. Box F.1: Methodology for Identifying CDD Projects Step 1: For each project in the sample of 84, project cost was di- sions and/or resources, (ii) creating an enabling environment, and vided among different aspects of community participation, start- (iii) capacity enhancing for the community were combined and ing from information sharing and ranging to community control this cost was divided by total project cost. over decisions and resources based on information in the World Bank appraisal documents (PAD/SARs).a Step 3: A project was classified as CDD if the percentage cal- culated in step 2 was 85 percent or more (85 percent being an Step 2: All costs devoted to (i) community control over deci- arbitrary cutoff). a. The level of information contained in appraisal documents varies widely, so some value judgments were required. For example, if the project was participatory, but only in a consulta- tive sense, it was designated CBD, but if it involved communities in a more holistic sense it was designated CDD. Within the latter, it was a matter of determining whether or not com- munities would be in charge of the funds allocated to them or if an outside group would manage their funds. Consequently, the amount of CDD in a project as classified by OED may be different from the amount reported by the CDD Anchor. However, an attempt was made to follow a clearly defined strategy. 7 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT The CDD operations are designed to provide Table F.2: Composition of the Sample communities with greater responsibility for each aspect of the subproject cycle. Based on infor- CDD CBD/CDD mation from the appraisal documents, the com- Number of projects 19 84 munity's role in different aspects of the Annual growth rate (%) 19.6 14.2 subproject cycle was categorized (figure 2.2 in Multisectoral 74 48 Chapter 2). The analysis revealed that commu- Sectoral (single sector) 26 44 nities were responsible for subproject design in Sectoral (single subsector) 0 8 nearly all CDD projects (compared with a third Sector Board in other CBD/CDD projects, excluding the CDD projects). Communities were also responsible for Social Protection 47 18 subproject operation and maintenance for over Rural Development 37 29 75 percent of the CDD projects, and for sub- Education 5 13 project implementation, for nearly two-thirds Urban Development 5 6 projects of the CDD projects. of Water Supply and Sanitation 5 2 While the community's role in subproject Health, Nutrition and Population 0 18 M&E for CDD projects was higher than that of Transportation 0 7 other CBD/CDD projects, the absolute number Region Percentage of projects identifying community responsibili- Africa 26 32 ties was low. East Asia and the Pacific 21 17 Europe and Central Asia 5 11 Latin America and Caribbean 32 23 Middle East and North Africa 0 7 South Asia 16 11 Source: World Bank data and calculations. Table F.3: The List of 84 Projects, Their Regions and Sector Boards Lending Fiscal Date, rev Commit- Project name Country Project ID Sector Board instrument year closing ment ($m) Africa Region Borgou Pilot Benin P057345 Rural Sector LIL 1998 6/30/02 4.00 Community-Based Rural Burkina Faso P035673 Rural Sector APL 2001 6/30/06 66.70 Development Urban II Burundi P000205 Urban Development SIL 1989 12/31/96 21.00 SDA/Human Resources Cameroon P000405 Social Protection SIL 1990 21.50 National Livestock Central African P000474 Rural Sector SIL 1995 6/30/00 16.60 Development Republic Public Works and Chad P000533 Social Protection SIL 1994 6/30/99 17.40 Capacity Building Third Education Comoros P000603 Education SIL 1997 12/31/03 7.00 Emergency Recovery/ Congo, P081924 Poverty Reduction ERL 2003 12/31/07 41.00 Community Project Republic of 7 2 A N N E X F Table F.3: The List of 84 Projects, Their Regions and Sector Boards (continued) Lending Fiscal Date, rev Commit- Project name Country Project ID Sector Board instrument year closing ment ($m) Pastoral Community Ethiopia P075915 Rural Sector APL 2003 12/31/08 30.00 Development AG Services Gambia, The P000818 Rural Sector SIL 1993 3/1/99 12.30 Secondary Schools Ghana P000954 Education SIL 1991 6/30/95 14.70 National Health Guinea-Bissau P035688 Health, Nutrition SIL 1998 6/30/05 11.70 Development Program and Population Sexually Transmitted Kenya P001333 Health, Nutrition SIL 1995 6/30/01 40.00 Infections Project and Population Rural Transport Project Madagascar P073689 Transport APL 2003 6/30/09 80.00 Urban Development Program Mauritania P069095 Urban Development APL 2002 12/31/06 70.00 HIV/AIDS Response Project Mozambique P078053 Health, Nutrition APL 2003 12/31/08 55.00 and Population Community Based Nigeria P069086 Social Protection SIL 2001 2/28/06 60.00 Poverty Reduction Human Resources Rwanda P045091 Education SIL 2000 6/30/06 35.00 Development Rural Water Supply Rwanda P045182 Water Supply SIL 2000 12/31/06 20.00 & Sanitation and Sanitation Quality Education For All Senegal P047319 Education APL 2000 12/31/04 50.00 Social Development Fund Senegal P041566 Social Protection APL 2001 12/31/05 30.00 HIV/AIDS Prevention Senegal P074059 Health, Nutrition APL 2002 9/30/07 30.00 & Control and Population Health Sector Sierra Leone P074128 Health, Nutrition SIL 2003 2/28/08 20.00 Reconstruction & and Population Development Poverty & Social Costs Uganda P002966 Health, Nutrition SIL 1990 9/30/95 28.00 and Population Small Towns Water Uganda P002957 Water Supply and SIL 1994 6/30/03 42.30 Sanitation EMCBP Uganda P002978 Environment SIL 1996 6/30/01 11.80 Pilot RDC Zimbabwe P045029 Urban Development SIL 1997 6/30/00 12.30 Total number of African projects 27 Total from 22 countries 848.30 East Asia & Pacific Region Social Fund Cambodia P037088 Social Protection SIL 1995 6/30/00 20.00 Shanxi Poverty Alleviation China P003649 Rural Sector SIL 1996 12/31/03 100.00 Disease Prevention (Hlth7) China P003589 Health, Nutrition SIL 1996 6/30/04 100.00 and Population Anning Valley Agricultural China P049665 Rural Sector SIL 1999 12/31/04 120.00 Development (continued on following page) 7 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table F.3: The List of 84 Projects, Their Regions and Sector Boards (continued) Lending Fiscal Date, rev Commit- Project name Country Project ID Sector Board instrument year closing ment ($m) Sustainable Forestry China P064729 Rural Sector SIL 2002 8/31/09 93.90 Development Irrigation Subsector II Indonesia P003953 Rural Sector SIL 1992 7/31/95 225.00 Third Community Health Indonesia P003914 Health, Nutrition SIL 1993 3/31/01 93.50 and Nutrition and Population WSSLIC II Indonesia P059477 Health, Nutrition SIL 2000 6/30/09 77.40 and Population Second Kecamatan Indonesia P073025 Social Development SIL 2001 12/31/06 320.20 Development Project Community Based Philippines P004595 Rural Sector SIL 1998 6/30/06 50.00 Resource Management Kalahi-CIDSS Project Philippines P077012 Social Development SIL 2003 6/30/09 100.00 Agriculture Rehabilitation Timor-Leste P070533 Rural Sector SIL 2000 3/15/03 6.80 Project Small Enterprises Project II Timor-Leste P072654 Private Sector SIL 2002 12/31/06 7.50 Development Second Education Project Vanuatu P004823 Education LIL 2001 6/30/05 3.50 Total number of East Asia and Pacific projects 14 Total from 6 countries 1,317.80 Europe & Central Asia Region Natural Resource Armenia P057847 Rural Sector SIL 2002 7/31/08 8.30 Management Highway Azerbaijan P040716 Transport SIL 2001 6/30/05 40.00 Farmer Support Services Croatia P008335 Rural Sector SIL 1996 12/31/02 17.00 Social Safety Net Kyrgyz Republic P008515 Social Protection SIL 1995 4/30/00 17.00 Rural Development Poland P058202 Rural Sector SIL 2000 6/30/05 120.00 SDF 2 (APL 2) Romania P068808 Social Protection APL 2002 8/31/06 20.00 Rural Education Romania P073967 Education SIL 2003 9/15/09 60.00 Rural Infrastructure Tajikistan P058898 Rural Sector SIL 2000 3/31/06 20.00 Rehabilitation Health I Uzbekistan P009125 Health, Nutrition SIL 1999 12/31/04 30.00 and Population Total number of Europe and Central Asia projects 9 Total from 8 countries 332.30 Latin America & Caribbean Region Renewable Energy in Argentina P006043 Energy and Mining SIL 1999 9/30/05 30.00 Rural Markets Health Sector Reform Bolivia P074212 Health, Nutrition APL 2001 6/30/06 35.00 and Population Land Management 3 Brazil P006474 Rural Sector SIL 1998 12/31/05 55.00 (Sao Paulo) 7 4 A N N E X F Table F.3: The List of 84 Projects, Their Regions and Sector Boards (continued) Lending Fiscal Date, rev Commit- Project name Country Project ID Sector Board instrument year closing ment ($m) Basic Education Costa Rica P006938 Education SIL 1992 9/30/00 23.00 Provincial Health Dominican P007015 Health, Nutrition SIM 1998 6/30/04 30.00 Services Project Republic and Population Reconstruction & Local Guatemala P049386 Social Protection SIL 1999 6/30/05 30.00 Development Nutrition/Health Honduras P007392 Health, Nutrition SIL 1993 6/30/01 25.00 and Population PROFUTURO Honduras P057350 Environment SIL 1999 10/31/04 8.30 Fifth Social Investment Honduras P064895 Social Protection SIL 2001 6/30/05 60.00 Fund Project On-Farm & Minor Irrigation Mexico P007701 Rural Sector SIL 1994 3/31/02 200.00 Second Decentralization Mexico P007702 Private Sector SIL 1995 6/30/00 500.00 Development Basic Education Mexico P040199 Education APL 1998 12/31/01 115.00 Development APL I Rural Development in Mexico P007711 Rural Sector APL 1998 6/30/03 47.00 Marginal Areas Rural Development in Mexico P057530 Rural Sector APL 2000 6/30/05 55.00 Marginal Areas II Social Investment Fund Nicaragua P007786 Social Protection SIM 1993 9/30/96 25.00 Basic Education II Panama P052021 Education SIL 2001 6/30/05 35.00 Social Development Fund Peru P008062 Social Protection SIM 1994 6/30/97 100.00 Second Rural Roads Project Peru P044601 Transport SIM 2001 6/30/05 50.00 Caracas Slum Upgrade Venezuela P040174 Urban Development SIL 1999 6/30/05 60.70 Total number of Latin America and Caribbean projects 19 Total from 12 countries 1,484.00 Middle East & North Africa Region Social Fund II Egypt, Arab P043102 Social Protection SIL 1996 6/30/01 120.00 Republic of NW Mountainous and Tunisia P072317 Rural Sector SIL 2003 12/31/08 34.00 Forest Areas Development OT - Emergency West Bank P034112 Transport SIL 1994 12/31/98 30.00 Rehabilitation I and Gaza Palestinian NGO Project II West Bank P071040 Social Protection SIL 2001 8/31/05 8.00 and Gaza Rural Access Improvement Yemen, P070391 Transport APL 2001 12/31/05 45.00 Program Republic of Taiz Municipal Development Yemen, P070092 Urban Development SIL 2002 12/31/05 45.20 & Flood Protection Republic of Total number of Middle East and North Africa projects 6 Total from 4 countries 282.20 (continued on following page) 7 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table F.3: The List of 84 Projects, Their Regions and Sector Boards (continued) Lending Fiscal Date, rev Commit- Project name Country Project ID Sector Board instrument year closing ment ($m) South Asia Region Social Investment Bangladesh P053578 Rural Sector SIL 2003 6/30/07 18.20 Program Project Rural Access Roads Bhutan P059481 Transport SIL 2000 4/30/05 11.60 Population Training (VII) India P009940 Health, Nutrition SIL 1990 6/30/98 96.70 and Population Uttar Pradesh Sodic India P009961 Rural Sector SIL 1993 3/31/01 54.70 Lands Reclamation Andhra Pradesh Forestry India P010449 Rural Sector SIL 1994 9/30/00 77.40 Blindness Control India P010455 Health, Nutrition SIL 1994 6/30/02 117.80 and Population Community School Nepal P082646 Education LIL 2003 9/30/06 5.00 Project Social Action Program Pakistan P010456 Education SIL 1994 12/31/97 200.00 NWFP On-Farm Water Pakistan P071092 Rural Sector SIL 2001 6/30/06 21.40 Management Project Total number of South Asia projects 9 Total from 5 countries 602.80 Total number of projects in sample 84 Total from 57 countries 4,867.40 7 6 ANNEX G: OED RATINGS OF COMPLETED PROJECTS The performance of the completed projects in the portfolio is analyzed using OED ratings for Table G.1: Project Sets Compared outcome, sustainability, and institutional im- pact.1 Of the 3,917 projects approved between CBD/CDD Non-CBD/CDD fiscal 1989 and fiscal 2003, 2,187 were inactive as Number of projects approved, of end of fiscal year 2003. To make the compar- 1989­2003 847 3,070 ison between CBD/CDD and non-CBD/CDD proj- Number of closed projects 336 1,851 ects robust, a few modifications were made that resulted in a database of 1,728 inactive projects Number of closed (table G.1): investment projects 334 1,421 Number of closed investment · Adjustment lendings were dropped (only 6 projects, 1994­2003 334 1,394 CBD/CDD adjustment lending projects). Source: World Bank database. · Projects exiting before 1994 were dropped (the first CBD/CDD project exited in 1994). The analysis did not attempt an annual com- rising one. Outcome ratings of CBD/CDD projects parison, because only 5 CBD/CDD projects ex- have been better when they are disbursement- ited in 1994; 9 in 1995, and 4 in 1996. Instead, weighted. the entire period from 1994 to 2003 was divided This evaluation also found no evidence to into two phases: 1994­98 (phase 1) and support the hypothesis that the CBD/CDD proj- 1999­2003 (phase 2). Of the 334 CBD/CDD proj- ects in conflict/post-conflict countries outper- ects, 70 projects exited during phase 1 and 264 form the CBD/CDD projects in non-conflict projects exited during phase 2. Of the non- countries for the period 1999­2003 by 4 per- CBD/CDD projects, 508 exited in phase 1 and 886 centage points (table G.2).3 in phase 2. Regional:4 The Africa Region has the largest Outcome Ratings for Completed CBD/CDD portfolio, but is the lowest-perform- Investment Projects2 ing Region on outcome. While only 61 percent of the Africa CBD/CDD projects were rated sat- Overall: About 74 percent of the CBD/CDD proj- isfactory for the aggregate period 1994­2003, ects were rated "satisfactory" on outcome for the Region has improved by 4 percentage points both phase 1 and phase 2. The corresponding from phase 1 to phase 2 (table G.3). The Latin numbers for the non-CBD/CDD projects are 66 America and Caribbean Region, with the sec- percent and 72 percent (figure G.1). Though the ond-largest portfolio, is the best-performing Re- CBD/CDD portfolio outperforms the non- gion on outcome. However, the percent CBD/CDD portfolio, the difference between the satisfactory rating in Latin American and the ratings for fiscal years 1999­03 is statistically in- Caribbean on outcome has declined by 10 per- significant and the trend for non-CBD/CDD is a centage points for the CBD/CDD projects.5 The 7 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure G.1: Projects--the Gap in The non-CBD/CDD portfolio, in aggregate Satisfactory Outcome Ratings Between for 1994­2003, performed better than the CBD/CDD and Non-CBD/CDD Projects CBD/CDD portfolio for the environment, health, Has Narrowed and transport sectors, and CBD/CDD portfolio for education, rural development, social pro- 100 CBD/CDD tection, and water supply outperformed that of Non-CBD/CDD non-CBD/CDD portfolio for satisfactory ratings 80 on outcome. However, the differences were not statistically significant between the two groups (%) 74% 73% 72% for any of the sectors. 60 66% Sustainability and Institutional Impact Satisfactory 40 Sustainability. While sustainability ratings have improved for both the CBD/CDD and non- 20 CBD/CDD projects, a significantly lower per- centage of CBD/CDD projects were rated "likely" or better on sustainability compared with the 0 1994-98 1999-2003 non-CBD/CDD portfolio (figure G.2).7 The Re- gional variation is somewhat similar, as seen for Source: World Bank database. the outcome ratings--the Africa Region has the Note: OED ratings are based on OED reviews of ICRs, 25 percent of which smallest percentage of projects rated "likely" or are subsequently revisited through OED field assessments. The outcome better, and the Middle East and North Africa Re- ratings of the closed investment projects reveal insignificant differences gion has the highest percentage, followed closely between CBD/CDD and non-CBD/CDD projects in the two phases. The differences between CBD/CDD and non-CBD/CDD projects were also by Latin America and the Caribbean (table G.3). insignificant for each exit year between fiscal 1999 and 2003. While both Africa and Latin America and the Caribbean (the two Regions with large CBD/CDD portfolios) indicate improving sustainability for South Asia Region has maintained its perform- CBD/CDD and non-CBD/CDD projects, the ance over time, but that of the East Asia and Pa- CBD/CDD projects in Africa outperform non- cific Region has declined (10 percentage points). CBD/CDD projects in phase 2 and that of non- CBD/CDD projects outperform CBD/CDD Primary Sector: CBD/CDD projects coded projects in Latin America and the Caribbean. under the education sector show the best per- formance on outcome, followed by projects Institutional Development Impact.8 The per- under the transport, urban development, and so- centage of projects rated "substantial" or better cial protection sectors. Those under the rural de- on institutional development (ID) impact was velopment sector, with the largest CBD/CDD portfolio, performed below average on outcome Table G.2: Satisfactory Outcome in aggregate, as did projects under water supply, Ratings Higher for Conflict/Post- health, and environment. The CBD/CDD projects Conflict Countries (1999­2003) under the rural development sector show stable performance; however, those under the social CBD/CDD Non-CBD/CDD protection sector (second-largest CBD/CDD (%) (%) portfolio) show a marginal decline (table G.4). Conflict/post-conflict Projects under the education sector indicate an countries 76 69 11 percentage point improvement, and those Non-conflict countries 72 73 under the health sector a 7 percentage point All countries 73 72 decline.6 Source: World Bank database. 7 8 A N N E X G Figure G.2: Project Sustainability Has lower for the CBD/CDD portfolio (29 percent) Been Consistently Lower for CBD/CDD than for the non-CBD/CDD portfolio (36 per- Projects But Is Improving cent) for the exit period 1994­98. However, the difference between the two groups was negligi- 100 ble for the exit period 1999­2003. While the per- CBD/CDD formance has improved for both groups, Non-CBD/CDD especially for the CBD/CDD portfolio, the per- 80 centage for both groups is still low (46 percent). (%) 60 better 63% or 55% 48% Likely40 37% 20 0 1994-98 1999-03 Exit fiscal year Source: World Bank database. Table G.3: Regional Variations (1994­2003) Region East Europe & Latin Middle Asia & Central America & East & South Africa Pacific Asia Caribbean N. Africa Asia Total Number of projects 26 9 2 21 3 9 70 Phase 1 Outcome (%) 58 78 50 95 67 78 74 1994­98 Sustainability (%) 31 33 50 43 67 33 37 Institutional development impact (%) 27 33 50 29 67 11 29 Number of projects 89 31 20 65 18 41 263 Phase 2 Outcome 62 68 75 85 83 78 73 1999­2003 Sustainability (%) 39 45 55 72 67 66 55 Institutional development impact (%) 33 29 50 63 61 54 46 Number of projects 115 40 22 86 21 50 333 Total Outcome (%) 61 70 73 87 81 78 73 1994­2003 Sustainability (%) 37 43 55 65 67 60 51 Institutional development impact (%) 31 30 50 55 62 46 42 Source: World Bank database. 7 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table G.4: Education Projects Lead in Percentage of Projects Satisfactory on Outcome (1994­2003) CBD/CDD Non-CBD/CDD Non- No. of Phase 1 Phase 2 CBD/CDD No. of Phase 1 Phase 2 CBD/CDD Sector projects (%) (%) (%) projects (%) (%) (%) Education 43 80 91 88 139 85 82 83 Environment 21 0 65 62 44 60 68 66 Health 56 64 57 59 83 65 70 69 Rural development 77 69 69 69 249 61 66 63 Social protection 69 82 81 81 34 77 76 76 Transport 11 100 75 82 183 82 85 84 Urban development 27 100 79 81 76 70 61 64 Water supply 19 50 73 68 66 50 67 62 Grand total 323 74 73 73 874 69 74 72 Source: World Bank database. 8 0 ANNEX H: FOCUS ON CBD/CDD AND RELATED ASPECTS IN BANK AND BORROWER STRATEGY The sample of 84 projects spans 57 countries. approach: donor coordination, enabling envi- Twenty-eight of these countries had completed ronment, and decentralization. Poverty Reduction Strategy Papers (PRSPs) as of · There is similarity between CASs and PRSPs April 2004. All 28 PRSPs were reviewed to assess on many issues, including one basic issue: the borrower countries' focus on CBD/CDD and over four-fifths of each are silent on com- CDD-related aspects. The borrower country munity management and control of resources focus was compared with the CDD focus of the as a strategy for the CBD/CDD approach (fig- World Bank Country Assistance Strategy for the ure H.1). same country. Only CASs prepared since fiscal 1999 were considered. As a result, 26 CASs were To assess the evolution of the focus on reviewed. The relevant questions and the re- CBD/CDD and CDD-related aspects, 58 CASs spective responses are presented in the table H.1. were reviewed, 2 from each of 29 countries (17 Overall, the review concluded: with a PRSP, and 12 without). The selection was made based on the availability of a CAS for a · There are significant differences between CASs country from two time periods, one from fiscal and PRSPs on three issues related to CBD/CDD years 1994­98 and one from fiscal years Figure H.1: Fewer Than a Fifth of CASs and PRSPs Identify Community Control over Resources CASs (n = 26) PRSPs (n = 26) Identifies community participation in decision making & resource 0% 0% 8% allocation, especially of the poor 12% 15% 27% Identifies partnership (community role in decision making & planning responsibilities) Identifies information & coming closer to communities Top-down approach (no participation 73% 65% in the formal decision making) Source: Review of CASs and PRSPs. 8 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 1999­2004. The relevant questions and the re- · Attention has increased over time to issues spective responses are presented in table H.1. related to the CBD/CDD approach: capacity en- Overall, the review concluded: hancement, dissemination, donor coordina- tion, enabling environment, decentralization, · The emphasis on the CBD/CDD approach in and monitoring and evaluation. the Bank's country-level strategy has increased · The emphasis on a multisectoral approach in over time (see figure 3.4 in main text). country strategies has changed little over time. Table H.1: Details of the CAS and PRSP Review (percent) CAS CAS Does the strategy Options CAS PRSP FY99­2004 FY94­98 Identify level of In decision making & resource allocation, especially of the poor 15 8 21 0 community participation In decision making & planning responsibilities 73 65 59 66 as important for poverty In information sharing & coming closer to communities 12 27 7 7 alleviation? No participation in the formal decision making 0 0 14 28 Reflect participatory Reflects with examples of involvement of all stakeholders 12 12 17 7 approach in other Reflects with examples of local government (+) involvement 46 38 41 14 economic, macro, Reflects, but without explicit involvement of local 38 42 41 31 sector work and government and/or grassroots analysis? Not reflected 4 8 0 48 Emphasize a Yes, with open menu 4 4 3 7 multisectoral approach? Yes, with a positive list on the menu 23 12 21 7 Indicates community choice 31 73 24 21 No mention of multisectoral approach 42 12 52 66 Link decentralization Links to lending and to community participation 15 12 17 3 to participation? Links to community participation 19 54 17 14 Links without explicit linkage to community participation 58 31 52 31 Decentralization not addressed as an issue 8 4 14 52 Focus on improving Yes, with emphasis on communities 15 23 21 3 the dissemination Emphasis either on how or to who, not both 31 42 41 14 of information? Some indication; but not on how and to who 35 31 21 17 No focus on dissemination of information 19 4 17 66 Focus on capacity Emphasis on community & local government CB 12 4 17 0 building (CB)? Emphasis on local government or community CB, not both 46 65 59 45 Refers to capacity building 42 31 24 52 No focus on capacity building 0 0 0 3 8 2 APPENDIX H: RESULTS FROM THE MAP SELF-ADMINISTERED QUESTIONNAIRE Table H.1: Details of the CAS and PRSP Review (percent) (continued) CAS CAS Does the strategy Options CAS PRSP FY99­2004 FY94­98 Focus on building an Lending programs developed to support enhancing or 8 0 7 0 enabling environment building an enabling environment for CDD projects within the country for Indication on enhancing or building an enabling 27 69 17 7 supporting projects? environment for CDD projects Emphasis on enhancing or building an enabling environment 62 27 72 86 for supporting projects No focus on building an enabling environment within 4 4 3 7 the country for supporting projects Put emphasis on Links lending to improving M&E and CB to carry out M&E 19 4 14 0 monitoring and Emphasis on improving M&E or M&E capacity, not both 58 69 55 21 evaluation (M&E) Refers to M&E 23 27 28 28 of activities? No emphasis on M&E 0 0 3 52 Address donor Links lending to strategies (or developing strategies) 27 0 28 17 harmonization and dealing with donor coordination issues coordination issues Developing strategies/multilateral networks to deal 54 27 59 52 to foster cooperation, with donor coordination issues and less competition Indicates donor coordination; but no explicit 19 58 10 24 (MDG 8)? strategy/multilateral networks No explicit suggestion to address donor coordination issues 0 15 3 7 Total number 26 26 29 29 Source: Review of CASs and PRSPs. 8 3 ANNEX I: CENTRAL GOVERNMENT AND LOCAL GOVERNMENT SURVEYS Central Government tions on its own. In Benin and Brazil, about 50 A structured survey of government officials was percent of the officials agreed that the Bank has conducted in four case countries--Benin, Brazil, the expertise to build or enhance local govern- Nepal, and Vietnam--and unstructured inter- ment capacity to support participatory inter- views of government officials were conducted in ventions; the percentage was much lower in Egypt, to assess, among other things, the extent Nepal (only 19 percent). to which Bank-supported participatory inter- ventions have been relevant to government and The Bank's ability in using participatory community priorities, and to what extent these approaches to address pertinent issues: interventions helped improve the institutional ca- Fewer than a quarter of the officials surveyed in pacity of the government at both the central Nepal and Vietnam and fewer than a third in and local levels. Unstructured interviews of gov- Benin perceive that the Bank can account for ernment officials were also conducted in Turkey social and cultural factors influencing out- in conjunction with an OED project assessment come, ensure sustainable flow of benefits, or mission. ensure downward accountability to the lowest In Benin, a total of 26 interviews were con- level of government using participatory ap- ducted with different ranking officials based in proaches. In Brazil, although 75 percent of the Cotonou; in Nepal, 16 central government offi- officials think that the Bank has the ability to en- cials based in Kathmandu were interviewed; and sure downward accountability to the lowest level in Brazil, 8 state government officials from Natal of government using participatory approaches, were interviewed (interviews with the Inter- only one official indicated that the Bank can en- American Institute for Cooperation in Agriculture sure a sustainable flow of benefits after projects (IICA) were not considered). In Vietnam, the finish. Project Management Unit (PMU) director (a high-ranking official from the ministry) for each The change in coordination between gov- of the participatory projects was interviewed, ernmental units: A majority of officials in Brazil for a total of 14 interviews. The most pertinent and Vietnam indicated increased frequency of results for the evaluation are presented below. meetings within the ministry and among min- istries since the initiation of the Bank's partici- The Bank's comparative advantage and patory intervention. In Nepal, however, only 44 expertise: Sixty percent of officials agreed that percent reported an increase in the frequency of the Bank has a comparative advantage in advis- meetings among ministries, compared with 81 ing government on the basis of analytical and percent reporting an increase in the frequency evaluative evidence, rather than in working di- of meetings within ministries. rectly with communities. Thus, predictably, nearly 60 percent of the government officials agreed Responsibility for monitoring and evalua- that the Bank should provide resources to the tion of Bank-funded participatory proj- central government to carry out participatory ects: In Nepal and Vietnam,1 a majority of the projects rather than undertake these interven- officials indicated that the responsibility of mon- 8 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT itoring and evaluation of Bank-funded partici- percentage of respondents did not pick any gov- patory projects rests with the central or the re- ernment level or communities to be responsible. gional government. In Benin and Brazil, a large A majority had picked others, with 11 to 13 per- Table I.1: Government Officials Survey Results by Country (percent) Benin Brazil Nepal Vietnam Total Participation leads to better outcome 65 83 63 86 73 Bank knowledge and expertise. Agree that Bank: Has comparative advantage in advising government on the basis of analytical and evaluative evidence rather than work directly with communities 54 63 69 60 Should provide resources to the central government to carry out participatory projects rather than undertaking them directly 50 50 69 64 58 Has the expertise to build/enhance local government capacity to support participatory interventions 50 50 19 40 Bank has substantial ability of using participatory approaches on the following aspects: Account for social and cultural factors influencing outcome 15 50 6 14 17 Ensure sustainable flow of benefits after projects finish 23 13 13 21 19 Ensure accountability downward to lowest level of government 27 75 13 21 28 Intra-government coordination. Since the initiation of Bank participatory interventions increase in frequency of meeting: Within ministry 75 81 64 74 Between ministries 63 44 64 55 Level of empowerment effective and efficient for development approaches: Community is informed and consulted on government development plan 79 25 Community prepares and/or implements a development plan 0 38 Community prepares a development plan with the help of gov/NGOs 17 19 Community has control over decisions and resources 4 19 More than 75 percent of the communities have the ability to: Identify needs and prioritize them 17 21 Manage financial resources 0 29 Participatory approaches increase time in involving communities 80 50 81 79 71 M&E responsibility for the Bank-funded participatory projects is with: Central government 19 13 56 50 34 Regional government 0 13 13 36 13 Local government 4 0 0 21 6 Communities 8 0 0 7 5 NGOs 0 0 6 14 5 Do not know 12 13 6 7 9 Total number of observations 26 8 16 14 64 Source: Government Officials Survey. 8 6 A N N E X I cent reporting no knowledge of who was re- Table I.2: Percentage of Communities sponsible for M&E. That Can Identify and Prioritize Their Needs Local Government Surveys Brazil Benin Surveys of local government officials were con- Range Percentage Range Percentage ducted in two countries where more intensive Above 75% 33 Above 75% 45 fieldwork was carried out. In Brazil, structured 50­75% 37 50­75% 18 surveys were conducted with 38 local govern- 25­50% 7 25­50% 18 ment representatives, while in Benin interviews Below 25% 7 Below 25% 14 were conducted with 24 local government rep- None 17 None 0 resentatives to assess, among other things, the Total 100 Total 100 extent to which Bank-supported participatory in- Source: Local Government Officials Survey. terventions have been relevant to local govern- ment and community priorities. It should be When asked how many communities had the noted that because of Benin's Decentralization capacity to manage the financial resources in- Program and recent elections, many of the com- volved in a subproject, 52 percent of local officials munal representatives were relatively new to in Brazil estimated that fewer than 25 percent of their positions. communities were capable; in Benin, 57 percent Among the interesting findings in these sur- of local officials gave this same estimate. veys: local government representatives in both Despite these grim assessments of community countries appear to be skeptical about the level capacity by the local government representa- of competency of their constituent communities tives in Benin and Brazil, the survey respon- to take charge of their own development. In dents support the participatory process in Brazil, for example, only 33 percent of local gov- general. When asked to what extent they felt ernment officials surveyed believed that more the participatory approach should be extended, than 75 percent of communities have the ca- 60 percent of Brazilian government officials re- pacity to identify and prioritize their needs; in sponded "all sectors" and 54 percent responded Benin, the figure was 45 percent. It is worth not- "all communities." A smaller proportion argued ing that in Benin, 32 percent of officials said that for "some" sectors and communities, and only between 0 and 50 percent of communities have one respondent responded "none" to either this capacity; in Brazil, the figure was 30 percent, question. In Benin, results were similar: 84 per- with a full 16.7 percent feeling that none of the cent believed that the participatory approach communities had this capacity. (See table I.2.) should be scaled up to more communities, and Officials in both countries also were asked 75 percent believed that participatory approaches what percentage of communities had the capac- should be scaled up to all sectors. ity to prepare a development plan. In Brazil, 47 The reasons for this dichotomy are unclear. percent said that fewer than 25 percent of com- While it may be that local governments were of- munities had this capacity; 24 percent said that fering a "politically correct" answer to these ques- none of their communities had this capacity. In tions, it is also possible that local governments are Benin, the numbers were similar: 27 percent said looking to either increase or maintain their in- that fewer than 50 percent of communities had volvement in local development activities and may this capacity and 32 percent believed that fewer be threatened by the notion of complete control than 25 percent did. of subproject implementation by communities.2 Officials in both countries also were asked Other issues of interest that were raised dur- what percentage of communities had the capac- ing local government surveys were: ity to implement and maintain a subproject. In Brazil, 57 percent of respondents believed that · NGO Capacity. Local officials in Benin were fewer than half of communities had this capac- divided on whether NGOs should be involved ity; in Benin, the figure was closer to 67 percent. in the implementation of participatory projects. 8 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT A common view expressed was that the qual- in Benin and Brazil agreed that involving com- ity of NGOs can sometimes be suspect, and munities in participatory development ap- thus there seems to be some apprehension proaches requires an increase in time, which about NGOs in general thereby indicates increased costs associated · Central/local government coordination. with the approach. In the case of Brazil, it is Local officials in Benin indicated that the level interesting to note that the response rates to of coordination has increased substantially this question at the central level (50 percent) since the onset of participatory projects in are significantly lower than the response rates their area. However, in Brazil, only 43 percent at the local level (80 percent). In Benin, re- of local officials agreed that this was the case, sponse rates to this question between the and 24 percent believe that coordination has central and local level were similar, with the 75 actually decreased. percent at the local level and 80 percent at the · Increased time needed to involve com- central agreeing that participatory approaches munities. The vast majority of local officials require more time. 8 8 ANNEX J: EFFICIENCY This annex explores the costs of adopting a fare gains, socially weighted as appropriate for CBD/CDD approach to multiple actors and the poverty objectives, and secondary benefits that benefits for poverty impact in an attempt to bet- might arise at a later date from improved ca- ter understand efficiency in CBD/CDD projects. pacity. Each of the above will be reviewed in this The data are limited, but CBD/CDD projects annex. seem to cost more to design and implement for all the players, but may provide offsetting savings Operational Costs to the Bank in infrastructure costs. Whether a sufficient en- The Bank's operational costs have been assessed hanced poverty impact occurs to justify the extra by three means: (i) actual Bank operational cost costs incurred is not evident in the cases stud- data against project commitment size and by ied, but poverty impact is not well evaluated. type of project--CBD/CDD or non-CBD/CDD; A typical project has multiple layers of oper- (ii) a staff survey to assess staff perceptions about ational costs. These are incurred by the Bank as Bank costs; and (iii) an earlier study that also lender, by the borrower (perhaps at several lev- used staff interviews and actual cost data. els) as implementer, possibly by a contractor, and finally by the households of the community of Bank Costs Based on Data beneficiaries. An efficient system would be one Bank costs for projects with a CBD/CDD ap- that, for a given resource transfer and project out- proach are higher than for non-CBD/CDD ap- come, would be "least cost," with due social proaches. The Bank cost graphs in the main weighting of costs and benefits in favor of any report (Chapter 3, figure 3.8), read in conjunc- poverty objective. Presumably the system should tion with table J.1, show, for the project uni- maximize the incentives down through the chain verse, the lending costs up to Board approval, of actors. and the supervision costs thereafter. For the su- There are four main categories of cost1 that pervision costs, only the completed projects can be compared between CBD/CDD and non- were taken, leaving 1,493 non-CBD/CDD and CBD/CDD interventions: 374 CBD/CDD from the total of 2,361 and 839, respectively. They also show that CBD/CDD proj- · Operational costs to the Bank for appraisal and supervision Table J.1: Mean Bank Operational · Operational costs to the borrower for appraisal Costs by Type of Lending (US$'000) and supervision for the Mean $50 to $60 Million · Unit costs of project investments, such as costs Commitment Size of contracted construction per kilometer of road Non- · Opportunity costs to beneficiaries of partici- CBD/CDD CBD/CDD pation. Average costs to approval 395 355 Average supervision costs 430 356 Benefits can be divided into primary benefits Total Bank operational costs 825 711 from investments, such as productivity or wel- Source: World Bank database and calculations. 8 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT ects cost the Bank more to prepare, appraise, and agreed, with 23 percent disagreeing or strongly supervise across the whole size range. For Bank disagreeing. This answer is consistent with the costs up to Board approval, at the CBD/CDD previous answer. project mean commitment size of $50 to $60 million (the average CBD/CDD project is a $57 The 1994 Hentschel Paper million commitment), the cost of CBD/CDD is Hentschel (1994) found higher costs for partic- about 11 percent higher than non-CBD/CDD ipatory projects based both on interviews with ($430,000 compared with $356,000, a difference staff associated with 21 participatory operations of $74,000). For supervision, in the same size and on data drawn from the Bank management bracket, CBD/CDD costs the Bank about 21 per- information system. Hentschel compared a sam- cent more ($430,000 compared with $356,000 for ple of 42 participatory projects between 1987 non-CBD/CDD). The aggregate difference of op- and 1994 with a Bankwide control group. But erational costs, including costs before and after costs were compared on a per project basis, with approval, is 16 percent for the relevant com- no attempt to analyze cost per dollar lent or per mitment size. These costs include trust funds. To dollar of total project cost. Interestingly, the look at it another way, the average non-CBD/CDD paper stopped short of aggregating the two sets project of $100 million commitment could be of budget-origin data from Bank and non-Bank, prepared for about the same cost as a $65 mil- mostly trust fund, sources. This OED study has lion CBD/CDD project. The cost gap is largely somewhat extended the analysis. Taking the total sustained across project commitment sizes. But resources given in the Hentschel study from all the gap in supervision costs is narrower for the budget sources and for all stages of the project smaller projects and widens with size, perhaps cycle, and assuming that both the participatory indicating some added challenge with scaling and the Bankwide control group projects would up of CBD/CDD. be five-year projects, suggests a total of staff Does the cost difference matter? An average weeks for the full cycle of 313 for the participa- operational cost increase of about 16 percent tory sample and 223 for the Bankwide control. across the Bank as a whole would certainly be Under that assumption, the costs would be about significant. 40 percent higher for CBD/CDD on a per-project basis. The mean project sizes in the Hentschel Staff Perceptions Drawn from Surveys sample are not given, so it is not possible to nor- Staff perceive the costs of CBD/CDD to be malize for the costs per dollar lent/project size higher. The staff survey asked questions about relationship. staff perceptions of such relative costs. In re- sponse to the statement (Survey Question 6) that Operational Costs to the Borrower implementation costs per dollar lent for The evidence suggests that costs to the bor- CBD/CDD projects are higher than other more rower for CBD/CDD operations are higher than traditional types of projects, 41 percent of staff for non-CBD/CDD.2 However, the evidence is either agreed or strongly agreed, and 27 percent scattered and limited. It is drawn from two disagreed or strongly disagreed, with 31 per- sources: first, surveys of borrower perceptions cent either neutral or saying they did not know. in four case study countries and, second, some This suggests that a majority of those taking a po- data from Indonesia and Egypt. sition perceived what the data show--that Bank About 80 percent of borrower officials who costs are higher for CBD/CDD. In response to were asked in case study country surveys whether the related but more specific statement (Ques- CBD/CDD projects took more staff time re- tion 7) that CBD/CDD approaches across the sponded "yes" (Benin, 80 percent; Vietnam, 73 whole project cycle, from identification to com- percent; Brazil [state], 50 percent; Brazil [mu- pletion, take more Bank staff resources per dol- nicipal], 79 percent; and Nepal, 81 percent). lar of lending than other types of investment The sample size by country was in the range of projects, 49 percent of staff agreed or strongly 7 to 15. So the perception seems to be quite 9 0 A N N E X J strong that CBD/CDD costs more in borrower 1.0 percent and 3.0 percent, with a modal figure staff time. of 2.6 percent. Although a very small sample, this Data from the Kecamatan Development Pro- suggests a difference of about 3 percent, with ject (KDP) in Indonesia, being assessed by OED, CBD/CDD being the more costly. However, in suggest the following: Egypt it is probable that a number of costs were carried by government outside the defined proj- · At the subdistrict level, the operational costs ect funding, making a comparison with Indonesia of the Financial Management Units, which was difficult. Also in Egypt, within the Public Works deducted from the grants, was 5 percent of Program of the Social Funds III Project, the more grants/loans. This proved just enough, but CBD/CDD-oriented Community Development barely, to keep the units funded. Program component had administrative costs · In addition, $61.9 million was provided for fa- that, at 8 to 10 percent, were about 6 percent cilitators, implementation technical assistance, higher than the parallel non-CBD/CDD Public and government administrative costs for a Works Program, at 2 to 4 percent. grant component of $189 million (about 33 percent). However, a modest portion of the Unit Costs of Project Investment technical assistance costs could be considered The evidence on the costs of construction pro- outside of the normal operational costs. Nev- vides a mixed picture. In four of the cases re- ertheless, including the costs of Financial Man- viewed, unit costs of investment, such as village agement Units, the total operational cost road construction costs, have fallen with partic- appears to have been not less than 30 per- ipatory approaches. In no study cases have costs cent. This is somewhat higher than typical risen, although questions have been raised about break-even costs of operating microfinance, construction quality, and therefore whether it is which has been found to be around 25 percent a fair comparison. In Indonesia, in both the Vil- globally, including cost of funds at around 7 per- lage Infrastructure 2 Project and the Kecamatan cent, but which has often ridden on the local Development Project, the evidence suggests that institutional support of other community de- costs are about 20 to 30 percent lower in com- velopment project expenditures. munity-managed infrastructure than in the same infrastructure built by previous top-down The Indonesia KDP cost can be compared processes, often using public agency force ac- with the non-CBD/CDD Indonesia Sulawesi Agri- count or poorly supervised or corrupt and col- cultural Area Development Project, a more tra- luding contractors. In Brazil, a comparison by ditional project that did not perform well OED of the estimated cost per beneficiary of (although it had some elements of consultation Mossoro Municipality Pipeline with the Northeast in one component). The actual operating costs Rural Poverty Alleviation Program showed that in that project added to half the consultant costs the investment cost of the latter was about 40 (since some were technical agriculture support) percent of the former. comes to about 25 percent of the total project In Nepal, evidence shows lower subproject costs, notwithstanding its much smaller size. unit costs from CBD/CDD projects compared So here there appears to be a difference of at with more traditional government agency proj- least 5 percent, perhaps more if normalized for ects. For example, quoted rates in person days size. per cubic meter of earth moved for roads and In Egypt, drawing from the OED case study bridges in ordinary soil was 0.47 for the Rural analysis, data were limited. However, operating Community Infrastructure Project and 0.70 for costs as a percentage of the total project costs government projects, indicating costs that were across 8 CBD/CDD projects lay in the range of about 50 percent higher for the conventional 0.9 percent to 8.3 percent, with the modal fig- government project. Also, in Nepal,3 unit costs ure around 6 percent, while for 3 non-CBD/CDD of service delivery under community programs comparators, the operating costs were between were found to be significantly lower than under 9 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT agency programs, exhibiting in many cases over Opportunity Costs of Beneficiary 100 percent differences. However, it is unclear Participation how comparable the different programs were in Costs of participation are higher by definition in technical difficulty. participatory projects; the question is at what However, OED's 2002 Social Fund Evalua- level are the costs of participation in relation to tion did not find any clear advantage in cost ef- the benefits and, at household level, the proba- fectiveness between social funds, local bility of benefits. No cases were found where the government, other central agencies, and NGOs cost of participation had been analyzed either ex across the 27 countries studied. It found the data ante or ex post. Indeed, in the OED Egypt case to be highly variable, as might be suggested by study, it was noted that, with the many different the differences between the Indonesia and Egypt participatory approaches being followed, an op- data quoted above. There were problems in portunity had been squandered to compare pro- normalizing for quality. That study warrants gram efficiency. Given the lack of data, we draw being given more weight than the other cases from only one project case. quoted because it represents a larger sample The OED PPAR for the Indonesia Kecamatan with a comparable methodology across country Development Project offers an example of the cases. The study found some indication that costs to a representative household of the meet- unit costs tend to be somewhat lower where ings needed to actively participate in the eco- community contributions were high and/or nomic loans component and compared it to the where there was community management and benefits of the group credit provided. (See box contracting. Overhead expenses were found to J.1.) The costs were substantial. If all meetings be in the range of 7 to 14 percent of total pro- in the KDP project were attended, it would be gram costs. possible to go to about 16. Box J.1: Costs of Household Time in a KDP Village In a typical village in North Lampung, Sumatra, for an active par- in question. Thus, the probability of not getting any reward in this ticipant who took an economic loan, there were 5 decision meet- case was about 0.4. Applied to the economic loan size (Rp350,000 ings and a Verification Team meeting. Two of the decision * 0.4 = 140,000), this suggests an opportunity cost in terms of time meetings were 2 hours long and involved 1 hour of travel. Three of about 60 percent of the loan size (Rp84,000/Rp140,000). How- of the meetings were 4 hours long and were farther away, tak- ever, there would be other gains on the positive side. Some of ing 2 hours of travel. The Verification Team meeting was a whole the time given would have gained respect and position in the day long plus 1 hour of travel (9 hours total). Thus, the total time community. Some may have contributed to other infrastructure was 33 hours, or about 4 working days. At a minimum wage in benefits relevant to the participant's hamlet. Also, there was a plantations in this area of Rp21,000 per day, the opportunity cost probability of not having to repay the loan at all. (In this partic- was Rp84,000. The total time from initiation of the discussions to ular village loan repayments were mostly between 80 and 100 receipt of grants/loans was 1 year and 4 months. Thus, for an av- percent, well above the project average.) However, it is con- erage economic loan size of Rp350,000, this person was spend- cluded that, overall, the costs of full participation were sub- ing about 25 percent of the value of the economic loan in stantial. This probably worked against the full participation of the meetings, with that investment not paying off in terms of re- poorer households who could least afford to give time at the risk ceipt of the money for over a year. However, there is also a of no benefits. While the case given here is a composite individual probability factor. Since KDP funding was competitive, there case, a village-level calculation, assuming the levels of atten- was a significant chance of not receiving benefits at all. In this dance at meetings reported and the types of meetings, gener- kecamatan, 18 of 42 KDP proposals were accepted in the year ally supports the estimate presented. 9 2 A N N E X J Benefits the extra cost to the Bank is significant: an The primary benefit expected from a CBD/CDD overall 16 percent increase in Bank costs for the intervention would be its impact on poverty in same output across the whole Bank program the broadest sense, which would call for esti- would be substantial. mating the benefits reaching the lower quintiles · Costs to the borrower at the government level and might also place some social weighting on are perceived to be higher by most officials, but those benefits. Here we explore two types of it has not been possible to find comparable ac- evidence, the evidence on poverty impact and, tual cost data. Approximate costs are known in more broadly, the project outcome data relative some individual cases, although there are ques- to costs to assess the development efficacy of tions about cost categories. They seem to sup- CBD/CDD projects relative to non-CBD/CDD port the perception of higher CBD/CDD which, among other things, accommodates the operational costs to government, but the sam- different project objectives. ple is very small. With respect to poverty, in the four study · Costs of construction of subprojects appear to cases where household surveys were done4 be lower, perhaps typically around 20 percent (Benin, Brazil, and Madhya Pradesh and Uttar lower, although there are cases (Nepal) where Pradesh in India), the impact on the poorest cost savings are claimed to be much greater CBD/CDD quintile over the non-CBD/CDD quin- than 20 percent, and recent data from In- tile on consumption and expenditure was small. donesia are showing savings of over 50 percent It was statistically insignificant in all cases, except in some cases compared with government- for consumption (but not expenditure) in the managed contracts. There is some question Madhya Pradesh case. about the robustness of the data in some coun- With respect to Bank project outcome per- tries.5 formance, CBD/CDD has a slight edge over non- · Opportunity costs to the beneficiary of time CBD/CDD, but only 74 percent satisfactory or given both for consultation and implementa- better compared with 72 percent (from 1989 to tion appear to be significantly higher, in some 2003)--not a large difference. Moreover, in re- cases as high as 10 to 20 percent of the in- cent years CBD/CDD has not improved per- vestment resources provided to the house- formance as much as non-CBD/CDD, which has hold, but again the data are very limited.6 been closing rapidly. This perhaps suggests that the Bank has learned more about how best to de- Based on the above, the reduced unit costs of sign and implement non-CBD/CDD than investment would need to be substantial, perhaps CBD/CDD. However, it is probably also a func- as much as 30 percent, to cover the extra con- tion of the fact that CBD/CDD performance was sultation and management costs, or, alterna- closer to a reasonable ceiling of expectation. tively, benefits in terms of poverty impact would need to be high. But this conclusion would be The Net Effect of All Cost Differences different, particularly if operational costs to gov- between CBD/CDD and Non-CBD/CDD ernment are not actually as high as surveyed of- ficials seem to suggest. Costs The above data are indicative of costs at differ- Benefits ent levels in the system. What the data appear to CBD/CDD projects do not significantly outshine show is the following: non-CBD/CDD in outcome performance and, so far, there is limited evidence of significant · Costs to the Bank are about 16 percent higher poverty reduction differences. With respect to the for CBD/CDD. Given the large sample, this is outcome rating, CBD/CDD projects were rated a fairly robust figure. Bank operational costs satisfactory in 74 percent of cases over the pe- themselves are small compared with those of riod 1989­2003, compared with 72 percent for government or communities. Nevertheless, non-CBD/CDD. Thus, CBD/CDD has performed 9 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT marginally better. But this does not offer a de- and the rating is still low in absolute terms. This cisive outcome performance edge. Moreover, is important in assessing CBD/CDD participa- as noted, the performance trend for non- tory performance, since the performance of CBD/CDD has gained steadily, while CBD/CDD community processes is a significant element has remained almost static. of the overall ID performance rating. In other With respect to poverty impact, the case stud- words, substantial gains in participatory ies and surveys found little evidence that processes at the community level should partly CBD/CDD projects have realized significant show up in gains in ID rating. poverty impact gains, despite their poverty ob- As noted earlier, secondary benefits to jectives. As found in the OED Egypt Matrouh Re- CBD/CDD may be relevant here as well as sec- source Management Project, this is partly ondary costs. On the benefits side, there may be because many investments are land-related, so improved efficiency in consultative processes that benefits are almost bound to reflect the with payoffs outside the project. On the costs existing inequity of land ownership. Although in side, there may be costs such as the cost of di- the Matrouh case OED believed that, because verting an NGO from a more important task to water cisterns were targeted to the poor, there one that is less important, but more immediately had been some reduction in the level of re- rewarding. However, non-CBD/CDD projects gressiveness due to the project. In the Turkey that mostly seem to be focused more on growth Eastern Anatolia Watersheds Management Pro- than equity may have substantial poverty im- ject, some of the poorer herders, who were not pacts through growth. closely linked to the decision communities, ap- peared to have been losers rather than gainers Lack of Data due to grazing land closure. On the related issue The many data and analysis gaps in the efficiency of gender, while there is evidence of some story need to be filled, particularly regarding bor- progress, there is still far to go, especially in rower costs. Indeed, in the Egypt case study, as very conservative situations such as the Egypt noted above, OED pointed out a missed oppor- Matrouh Project. tunity to compare poverty impact efficiency across CBD/CDD projects only perform 2 percentage a range of Bank-funded project approaches, from points better than non-CBD/CDD on the insti- the very intensive CBD/CDD approaches to the tutional development (ID) rating (44 percent less-intensive social fund approaches. versus 42 percent). This is a very small difference, 9 4 ANNEX K: NKAYI DISTRICT FORMAL AND INFORMAL SYSTEMS The Formal System Open access and the use of multiple In Nkayi District, western Zimbabwe, water is water sources: People prefer to maintain ac- supposedly managed at the community level cess to a number of different water sources over through formal waterpoint committees, usually a wide area, not just to the local one that they made up of three women (representing users) "own." This is partly because certain sources and one man (representing authority). The com- are preferred for particular purposes. It is also mittee is technically a subcommittee of the vil- for "insurance" reasons, because if one source lage development committee, and is part of a dries up, breaks down, or access to it is re- tiered maintenance system involving structures stricted, the users want to be sure of being able at the ward and district levels. The system is to draw water elsewhere. The Nkayi people be- based on the concept of establishing one com- lieve strongly that everyone should have access mittee for each waterpoint, representing the to water sources to secure at least the mini- users of that point, and great emphasis is placed mum necessary for survival. But such universal in training on encouraging a sense of "owner- access becomes increasingly disputed during ship" for the waterpoint. Training also empha- dry months. As water sources diminish, some sizes the requirement that committee members users (often committee members) try to con- are elected, that meetings are held regularly, serve the remaining supplies by restricting ac- and that proper minutes are taken. The com- cess to community members in the immediate mittee is expected to undertake routine pre- vicinity and to those who have participated in ventive and minor corrective maintenance, and implementation. Such action is reinforced in to guide the community in agreeing rules or many cases by "ownership" messages intro- bylaws relating to the waterpoint. Models of duced by project mobilization and implemen- such bylaws are given at training sessions. This tation activities. formal management system is based on the as- sumption that people will use and manage one Scarcity, conventions: People in Nkayi use water source only, and that there is a need to re- very small quantities of water for domestic pur- strict irresponsible use of the water source. But poses (estimated at a maximum of 8 to 10 liters research in Nkayi District has uncovered local per person per day in the dry season--a desir- practices of water use and decision making that able amount would be 15 to 20 1/p/d). Even are contrary to the formal system as manifested when water is relatively plentiful (for example, through committee structures. at a fast-flowing borehole), people do not in- crease the amount they use substantially. There Local Principles are two likely reasons for this: first, the deeply Many local principles of water use and manage- rooted fear of drought and the perception of ment are not explicit rules or regulations, but water as a scarce resource means that people ha- rather customs and conventions, or what people bitually employ practices that are water con- suggest is the "right way of doing things." These serving. Secondly, water use is partly determined often predate (by many decades) the establish- by who and how many in the family can collect ment of waterpoint committees. water--those households with lots of small chil- 9 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT dren and only one adult to carry water use rel- people do not generally use water sources irre- atively small quantities. sponsibly as their proper use is defined by cus- tom and practice. Project mobilization needs to Water-use preferences: The Nkayi men and take account of such complexities and to rec- women have markedly different priorities where ognize local cultures of water use. water use is concerned--men want to ensure that they can water their cattle, while women are Decision making: Committees are not nec- more preoccupied with having enough water essarily the Nkayi villagers' preferred way of con- for drinking, washing, and cleaning. ducting local business. In fact, most decisions of importance (such as restricting access to the Ownership equals access? As people use water source, or rationing the amount of water multiple water sources over a wide area, the ad- available, or deciding to make cash contribu- ministrative boundaries through which water is tions for maintenance), are made at "meetings managed are not necessarily appropriate. The wa- of the people" nominally held under the auspices terpoint committees are largely established on of the village development committee. A num- the basis of village boundaries and are ineffec- ber of decision-making principles are apparent tive in area wide resource management, as they at community level. The villagers believe that have no remit outside their own restricted area. everyone potentially affected by a decision This is the case even if people of that commu- should be present when it is made; therefore nity depend on "external" sources of water (a dis- meetings of all available adults in the community tant borehole or a dam) for their livelihoods. are held to discuss issues of water-resource man- Attempts to introduce greater "ownership" of agement (and other related issues, such as graz- new water supplies may result in restricting ac- ing). Wherever possible the use and regulation cess. It is generally the poorer households and of local resources is conducted both through families living on the outskirts that suffer from informal decision-making and through adher- such restricted access. So, under such ownership ing to custom and practice. Meetings are only policies, improved management of the water- held when a problem arises and action taken point can on occasion be achieved at the ex- only when absolutely necessary. Many of the re- pense of equity. People's preferences regarding source use management and decision making different sources of water are complex and their arrangements are strongly influenced by the de- choice of waterpoint not attributable to single fac- sire to avoid conflict between neighbors. tors such as cleanliness or time. Additionally, Source: From Cleaver 1998 (also cited in Kumar 2003). 9 6 ANNEX L: RESULTS OF BANK STAFF SURVEY A survey was conducted to seek the perceptions that the Bank-supported CBD/CDD projects are of selected Bank staff and managers on such is- generally sufficiently targeted at the poorest, 16 sues as CBD/CDD project performance, incen- percent disagreed with the statement, and 20 per- tives, process, and resources. A total of 400 cent neither agreed nor disagreed. surveys were electronically mailed to a select but varied set of staff. The response rate was 38 Bank strategy, processes, and products: percent. The following analysis is based on the The Bank is increasingly decentralizing its op- 152 completed surveys received by OED by the erations to field level. Thirty-nine percent of the specified date (tables L.1 to L.4). Both response respondents were satisfied with the impact of rate and multivariate analysis were conducted. Bank decentralization on the efficacy of Bank The most pertinent results for the CBD/CDD support for CBD/CDD projects. However, only evaluation are presented below. 27 percent of the respondents were satisfied with guidance and support from either the man- Targeting: Fifty-two percent of the respondents agement in the region over the project cycle, or indicated that they agree with the statement the relevant thematic groups, or the Social An- Figure L.1: Fewer Than a Third of the Respondents Are Satisfied with the Bank's Support and Guidance Guidance / support for CBD/CDD projects from Bank matrix management organizational structure CDD Social Anchor team in ESSD Thematic groups Bank's management over project cycle 0 20 40 60 80 100 Percent Satisfied Somewhat Not satisfied No response satisfied Source: Bank staff survey. 9 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT chor team in ESSD, or the matrix-management task managers of CBD/CDD projects could mon- organizational structure for the needs of itor fiscal accountability as satisfactorily as in CBD/CDD projects (figure L.1). Only 9 percent more traditional projects; 36 percent agreed, were satisfied with coordination within the Bank and 28 percent were neutral. Less than a fifth of across sectors.1 There were no significant varia- the respondents were satisfied with the quantity tions in the response to the above queries by re- and quality of training on CBD/CDD over the past spondent profession/specialty or association 2 years. with CBD/CDD projects (table L.5). Coordination with other players: About a Cost of doing business: Thirty-nine percent of fourth of the respondents indicated that an- the respondents agreed that implementation other agency collaboration (NGO or bilateral costs per dollar lent for CBD/CDD projects are donor) significantly enhances the quality of the higher than for more traditional projects (26 per- CBD/CDD project in over 60 percent of the proj- cent of the respondents disagreed, and 13 per- ects. Thirty-nine percent of the respondents also cent neither agreed nor disagreed).2 Also, only agreed with the statement that inadequate donor about a quarter agreed that sufficient resources coordination in a cofinanced project is likely to were made available to effectively appraise and have a greater negative impact on outcomes in implement CBD/CDD projects or address safe- a CBD/CDD project than in a more traditional guard issues.3 Despite the lack of resources to ef- project (18 percent disagreed and 22 percent nei- fectively implement participatory projects, 35 ther agreed nor disagreed).4 percent indicated that the Bank has scaled up the project in over 60 percent of the projects. Comparative advantage: Thirty-eight per- cent of the respondents disagreed that bilateral Sustainability: Only 18 percent of the re- donors' interventions generally achieve a better spondents indicated that they agree with the CBD/CDD outcome than Bank interventions. statement that community maintenance contri- Thirty-one percent also disagreed that NGO- butions are sufficient to sustain infrastructure in- supported interventions generally achieve a bet- vestment for Bank-funded CBD/CDD projects. ter CBD/CDD outcome than Bank interventions.5 Further, only about a quarter agreed with the Forty-four percent of the respondents indicated statement that the Bank generally contributes that they agree with the statement that the Bank funding for CBD/CDD projects long enough to has a comparative advantage over bilaterals to reach a satisfactory level of sustainability of com- achieve development impact in CBD/CDD in- munity processes. terventions; 13 percent disagreed with the state- ment, and 27 percent neither agreed nor Knowledge and skills: Nearly a quarter of the disagreed. respondents disagreed with the statement that 9 8 A N N E X L Table L.1: Response Rates for Bank Staff Survey (percent) Agree or Neither Disagree strongly agree nor or strongly agree disagree disagree The Bank sufficiently addresses the policy issues needed to support successful CBD/CDD interventions. 38.82 23.03 26.97 Bank CBD/CDD projects generally have been sufficiently targeted at the poorest. 51.97 19.74 16.45 Bank CBD/CDD projects have addressed sufficiently, and had been consistent with, broader institutional and fiscal decentralization. 32.89 22.37 30.26 Sufficient resources (relative to non-CBD/CDD projects) are made available by country directors to effectively appraise and implement CBD/CDD projects. 25.00 21.71 36.18 Sufficient resources are made available by country directors to effectively address safeguard issues related to CBD/CDD projects. 21.71 26.97 30.92 Implementation costs per dollar lent for CBD/CDD projects are higher than other traditional types of projects. 38.82 12.50 25.66 CBD/CDD approaches across the whole project cycle, from identification to completion, take more Bank staff resources per dollar of lending than other traditional types of investment projects. 46.71 12.50 21.71 CBD/CDD projects are more risky than traditional non-CBD/CDD projects. 24.34 21.71 44.74 Task managers of CBD/CDD projects can monitor fiscal accountability as satisfactorily as more traditional non-CBD/CDD projects. 35.53 27.63 23.03 In Bank-funded CBD/CDD projects, community maintenance contributions generally are sufficient to sustain infrastructure investments. 17.76 27.63 36.84 The Bank generally continues funding CBD/CDD projects for long enough (e.g., if necessary, into a second or third phase) to reach a satisfactory level of sustainability of community processes. 24.34 23.68 26.97 Sectoral technical standards (e.g., irrigation or curriculum design standards) in CBD/CDD operations have not been excessively compromised by CBD/CDD approach compared with traditional operations. 43.42 23.03 11.84 Bilateral donor interventions generally achieve a better CBD/CDD outcome than the Bank interventions. 10.53 25.66 38.16 NGO-supported interventions generally achieve a better CBD/CDD outcome than the Bank interventions. 26.32 24.34 30.92 Inadequate donor coordination in a co-financed project is likely to have a greater negative impact on outcomes in a CBD/CDD project than in a more traditional non-CBD/CDD project. 38.82 21.71 17.76 The Bank has a comparative advantage over bilaterals to achieve development impact in CBD/CDD interventions. 44.08 26.97 12.50 Your Region has sufficient CBD/CDD-related skills to achieve satisfactory CBD/CDD project performance. 48.68 15.79 18.42 CBD/CDD projects that use program specific committees to make investment allocation decisions, while perhaps facilitating project implementation in the short term, may often fail in the long term to significantly strengthen decentralized local institutions because they operate--parallel to, rather than integrated with, local government. 44.74 20.39 21.05 Note: Based on 152 responses. 9 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table L.2: Response Rates for Bank Staff Survey (percent) Satisfied Somewhat Not or better satisfied satisfied Understanding by the management in your region of the objectives and design of CBD/CDD projects. 40.13 28.95 15.13 Guidance by the management in your Region over the project cycle on CBD/CDD projects. 26.97 35.53 16.45 Guidance for CBD/CDD work provided by the CDD Social Anchor team in ESSD. 21.05 29.61 19.74 Support for CBD/CDD work from the relevant Thematic Groups. 26.97 29.61 17.11 Support by the current Bank matrix-management organizational structure for the needs of CBD/CDD projects. 13.82 29.61 27.63 Coordination within the Bank across sectors in CBD/CDD interventions. 9.21 34.21 36.18 Coordination within government in borrowing countries for CBD/CDD interventions. 19.08 30.26 30.92 Amount of training on CBD/CDD over the last two years (including clinics, brown-bag lunches, etc. as well as longer training). 19.08 26.97 22.37 Quality of training on CBD/CDD over the last two years (including clinics, brown-bag lunches, etc., as well as longer training). 17.11 23.68 9.87 The impact of Bank decentralization to field offices on the efficacy of Bank support for CBD/CDD projects. 39.47 21.71 12.50 Emphasis placed by the Bank on donor coordination in CBD/CDD projects. 28.29 36.84 14.47 Relevance of current Bank safeguards for CBD/CDD projects. 16.45 35.53 25.66 Note: Based on 152 responses. Table L.3: Response Rates for Bank Staff Survey (percent) More Less than 80% 60­80% 40­60% 20­40% than 20% In what percentage has the Bank committed to continuing support to community groups to the point of satisfactory sustainability of those group processes? 9.21 18.42 19.74 17.11 8.55 In what percentage have community groups and associated community processes reached to a level you would rate as "likely" for sustainability? 2.63 25.00 27.63 16.45 9.21 In what percentage has another collaborating development agency (e.g., other donor, NGO, etc.) significantly enhanced the quality of the CBD/CDD elements of design in the project? 6.58 18.42 21.05 11.84 10.53 In what percentage has the Bank scaled up the project? 10.53 24.34 19.74 5.92 9.87 Note: Based on 152 responses. 1 0 0 APPENDIX H: RESULTS FROM THE MAP SELF-ADMINISTERED QUESTIONNAIRE Table L.4: Response Rates for Bank Staff Survey (percent) 1-5 6-10 11-15 16-20 20+ In your experience, what would be the average number of years needed for project support of community groups initially formed under the project to reach a level of sustainability of community processes requiring very limited outside support (such as simply a supporting/maintenance visit once a year). 23.68 51.97 11.84 0.00 0.66 Note: Based on 152 responses. 1 0 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT stcejorp DDC/DBC rof troppus knaB fo ycaciffe eht no seciffo dleif ot noitazilartneced knaB fo tcapmi ehT 0.09 0.19 0.34 0.06 0.23 0.16 0.61 0.12 0.01 0.03 8.52 Coef. ­0.05 ­0.12 ­0.71 ­0.02 112.00 snoitnevretni DDC/DBC rof seirtnuoc * gniworrob ni tnemnrevog nihtiw noitanidrooC 0.33 0.04 0.58 0.54 0.54 0.17 0.60 0.35 0.58 0.00 0.39 0.02 0.04 Coef. ­0.27 12.50 122.00 snoitnevretni DDC/DBC * ** ni srotces ssorca knaB eht nihtiw noitanidrooC 0.06 0.05 0.45 0.30 0.04 0.44 0.00 0.04 Coef. ­0.54 ­0.86 ­0.66 ­1.04 ­0.39 ­0.67 10.67 121.00 stcejorp DDC/DBC ** fo sdeen eht rof erutcurts lanoitazinagro tnem -eganam-xirtam knaB tnerruc eht yb troppuS 0.18 0.42 0.33 0.13 0.76 0.05 0.05 Coef. ­0.05 ­0.27 ­0.18 ­0.15 ­0.02 ­0.02 ­0.09 12.24 108.00 tcejorp DDC/DBC-non naht DDC/DBC a ni semoc *** -tuo no tcapmi retaerg a evah ot ylekil si tcejorp 0.07 0.12 0.39 0.51 0.62 0.83 0.41 0.01 0.05 decnanif-oc a ni noitanidrooc ronod etauqedanI Coef. ­0.43 ­0.03 ­0.23 ­0.11 ­0.21 17.69 119.00 snoitnevretni * * ** knaB eht naht emoctuo DDC/DBC retteb a eveihca yllareneg snoitnevretni detroppus-OGN 0.00 0.09 0.27 0.09 Coef. ­0.74 ­0.64 ­0.80 ­0.48 ­0.16 ­0.20 ­0.40 ­0.04 ­0.29 ­0.04 31.80 124.00 stcejorp fo sepyt lanoitidart ** ** *** ** rehto naht rehgih era stcejorp DDC/DBC rof tnel rallod rep stsoc noitatnemelpmI 0.22 1.14 0.26 0.07 Coef. ­0.48 ­0.63 ­0.96 ­0.50 ­0.20 ­0.13 ­0.15 ­1.04 ­0.08 ­0.01 26.17 117.00 stcejorp DDC/DBC ot detaler seussi draug * * -efas sserdda ylevitceffe ot srotcerid yrtnuoc Survey yb elbaliava edam era secruoser tneiciffuS 0.98 0.21 0.26 0.31 0.03 0.11 0.00 0.05 Coef. ­0.13 ­0.20 ­0.60 ­0.27 ­0.39 ­0.02 16.42 121.00 Staff Bank of 1%. no. years at for years 2 and 5 specialist last Region past in above) in Region ***significant or Analysis thematic weeks managed Region topics 5%; Asia or anthropologist >2 at Caribbean level task & Pacific with managed Region & Central (sector task significant Region & technical projects CDD-related ** Asia America Asia specialist projects in CDD 10%; Multivariate Africa East Europe Latin South CDD economist sectoral sociologist/social manager CDD of projects at for for for for for for for for for for of no. training R-squared L.5: participation of of of non-CDD Significant * ableT Dummy Dummy Dummy Dummy Dummy Dummy Dummy Dummy Dummy Dummy Number Ratio Days Observations Pseudo Chi2 Note: 1 0 2 ANNEX M: METHODOLOGY FOR COMMUNITY-LEVEL DATA COLLECTION AND ANALYSIS Different stakeholders can have different (even hancement and empowerment of communities opposing) perspectives on the various aspects (Annex N); second, the sustainability of project and outcomes of a project. Hence, it is impor- investments (Annex P). The household data was tant to collect information from different stake- also used to assess the extent to which CBD/CDD holders to get a complete picture of alternative project investments met the priority needs of perspectives. Both qualitative and quantitative beneficiary communities (figure 3.5, Chapter 3). tools were used to collect data for this evaluation. In each of the four project areas, a total of 30 Sample Selection communities were selected, and roughly 40 OED's fieldwork adopted a non-experimental households were interviewed per community. evaluation design that compared randomly se- Two focus groups and two key informant inter- lected CBD/CDD communities with compara- views were also conducted in nearly all com- tor communities in the four projects using a munities. This annex is organized in three main comparison group methodology. In all four cases sections. The first one presents the methodol- the comparator group exhibited similar problems ogy used for the selection of communities and or issues as the project group and had similar so- households within communities for each of the cioeconomic and cultural characteristics. The four project areas. The second presents details selected comparators varied according to proj- of the survey instruments and timing of the field- ect and country context. In two project areas work. The final section describes the method- (Benin and Brazil), the comparison group com- ology adopted for household data analysis. munities had benefited from similar subprojects Four CBD/CDD projects (three of which were as the CBD/CDD communities but through a CDD) were selected for extensive fieldwork at the non-participatory approach adopted either by the community level: the Rural Poverty Alleviation government or a religious organization. These Project (RPAP) in Rio Grande do Norte, Brazil two cases allowed the evaluation to assess (henceforth Brazil); the Borgou Pilot Project in whether a program that involves communities is Benin (henceforth Benin); the Uttar Pradesh more effective than one that does not. In another Sodic Land Reclamation Project in India (hence- project area (Madhya Pradesh) comparator com- forth Uttar Pradesh), and the Madhya Pradesh munities benefited from a similar activity Forestry Project in India (henceforth Madhya (forestry) carried out through a participatory Pradesh). The two projects in India were single- approach, but supported by the government in sector interventions, while the projects in Benin India. Here the evaluation assessed whether and Brazil were multisectoral. Fieldwork in Brazil there is any difference in outcomes because of covered largely water supply subprojects; while the participatory approach pursued by the Bank in Benin it covered largely primary schools, as compared with the participatory approach health facilities, and storage houses. pursued by the government. Finally, in the fourth The community-level data collection was un- project area (Uttar Pradesh), comparison com- dertaken primarily to assess two issues: first, munities did not benefit from a similar activity the association between Bank-supported as project communities. Here the evaluation as- CBD/CDD interventions and social capital en- sessed the overall outcomes of the Bank 1 0 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT CBD/CDD project, and not only that of its par- from any of the following types of subprojects: ticipatory approach. construction of a school or other infrastructure (such as well, storage house, hangar, and the Community Selection like), functional literacy, training in beekeep- The process of community selection varied ac- ing, hygiene and nutrition, and provision of es- cording to the project context and is detailed sential drugs. The selection of the five below. In all four project areas, a larger sample comparator communities was undertaken by than actually required was selected to allow for the local expert contracted for the community- replacement of any community that was found level fieldwork, based on the following three not to satisfy the selection criteria after field criteria: communities that: (a) did not benefit verification. from any of the Bank CBD/CDD projects; (b) benefited from similar subprojects as the proj- Benin ect communities; and (c) were located in rural In the Borgou region of Benin, the Bank fi- areas. Comparator communities benefited from nanced three CBD/CDD projects in the past non-participatory projects funded by the State decade: the Borgou Regional Pilot Project and various religious organizations. These com- (PAMR); the Social Fund Project (AgeFIB); and munities received investments for the con- the Food Security Project (PILSA). While the struction of schools (two communities), wells focus of the evaluation was on PAMR (hence (two communities), and a health center (one the focus on the Borgou region), as project doc- community). uments revealed it to be a CDD project, com- Inaccuracies in subproject records required munities that benefited from AgeFIB and PILSA changes to the original sample of communities were also surveyed to allow for comparison as well as dropping some communities from among the three projects.1 The selection of the the analysis. After verification in the field of the PAMR, AgeFIB, and PILSA communities required project status of selected communities, a few first of all identifying, based on the subproject communities were replaced because they had ei- records of the three projects, communities ther benefited from more than one Bank within the Borgou region that had benefited CBD/CDD project, and/or had benefited from from only one of these three Bank CBD/CDD in- more than three subprojects. These communi- terventions.2 The selection was further restricted ties were replaced with randomly selected com- to communities that (a) were located in rural munities of the same project status. A few areas, (b) benefited from no more than three comparator communities were also replaced as subprojects, (c) benefited from specific type of they were found to have benefited from one of investments. For PAMR communities, the se- the three Bank CBD/CDD projects. Further, not lection was restricted to those that benefited all subprojects could be covered in the desired Table M.1: Coverage of Fieldwork in Benin Participatory approaches Top-down approaches PAMR AgeFIB PILSA Other donor State-funded Religious org. Fieldwork coverage Communities 17 7 2 1 3 2 Households 736 304 85 45 118 88 Analysis coverage Communities 13 4 2 0 3 2 Households 566 177 85 0 118 88 1 0 4 A N N E X M proportion since it was not uncommon to arrive by the State Technical Unit is more of a formality in a community that had, according to records, compared to PAC; providing the subprojects received funding for, say, a school, only to find meet the required technical specifications, the that what was actually funded was a storehouse State Technical Unit signs off on the proposal that had not been accurately registered in the made by the FUMAC Council. records. The inaccuracies in subproject-level · FUMAC-P: The procedures are the same as for records were detected by visiting sampled com- FUMAC, except that the FUMAC-P Council is munities and asking information of local leaders. given an annual budget, which it administers However, in some cases, community leaders' itself. The council signs agreements with the recollections also proved inaccurate, as focus CAs, transfers project funds to them, keeps group sessions later revealed the presence of track of receipts, and monitors physical other Bank projects that were unknown to local progress. It is accountable to state govern- leaders. Such communities, though surveyed, ment auditing procedures. If one CA fails to were dropped from the analysis. Further, one provide the necessary receipts, disbursements comparator community was also dropped from to all other CAs in that municipality may be the analysis because it was the only one to have frozen, paralyzing the project process. participated in a non-Bank CBD/CDD project. Table M.1 presents the final number of house- The selection of project communities was hold surveys conducted and the number actu- based on the project's monitoring and infor- ally used for the analysis. Two key informant mation system (MIS). To keep logistic and trans- interviews and one mixed (male and female) port costs within the budget, fieldwork was focus group were conducted in each of the restricted to the two regions (out of four) that communities. had the highest number of communities where only one subproject had been financed by the Brazil RPAP--Agreste and Oeste Potiguar. The crite- The RPAP (and the follow-on RPRP) adopted rion of one subproject per community was cho- three distinct community-driven implementa- sen for two main reasons. First, 79 percent of the tion modalities; in increasing order of decen- communities that benefited from the RPAP in Rio tralization, these were: Grande do Norte received only one subproject. Second, we wanted to avoid comparing com- · PAC: The Community Association (CA) submits munities that had received only one subproject a subproject proposal to the State Technical with those that had benefited from more. The se- Unit. Using a statewide vetting process, the lection of project communities was further re- State Technical Unit chooses the soundest pro- stricted to those that: (a) were located in the rural posals, with some reference to the evenness of areas, and (b) had benefited from one the fol- distribution among the various municipalities. lowing investments: water supply, electricity, ir- Once approved, project funds flow directly to rigation, or small bridges.3 The selection of a bank account set up locally by the Commu- project municipalities was limited to those that nity Association. had at least two communities that met the above · FUMAC: A municipal council (called the criteria. A stratified random sample of 11 proj- FUMAC Council), with representatives of civil ect municipalities was selected, with each of the society and the government, is set up by the three implementation modalities being repre- project at the municipal level. The proposals sented in proportion to the number of munici- prepared by the CAs are first reviewed and palities under each modality. A total of 24 ranked by the FUMAC Council, and only then communities were selected within these mu- submitted to the State Technical Unit. The nicipalities using a table of random numbers. council chooses among subproject proposals The selection of comparator communities re- with reference to an indicative budget com- quired, first of all, identifying municipalities that municated by the State Technical Unit. Vetting were targeted by the RPAP and the ongoing RPRP, 1 0 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT but that had not yet benefited from either proj- ticipatory component. All comparator commu- ect.4 Drawing on the MIS data for the RPAP and nities included in the analysis benefited from a ongoing RPRP (updated to October 15, 2003), six government-funded water pipeline constructed comparator municipalities were identified-- at the time when the RPAP was being imple- three in each of the two regions. The selection mented. None of them benefited from either of suitable comparator communities was un- the RPAP or the ongoing RPRP, while one bene- dertaken by the local expert contracted for com- fited in 1994 from another Bank CBD/CDD munity-level fieldwork. Comparator communities project--the reformulated Northeast Rural De- had to satisfy four main criteria: (a) they had to velopment Program (NRDP; 1993­96). This com- have benefited from a similar service as project munity was, however, retained for the analysis, communities around the same time as these as it did not differ from the other two compara- did; (b) they could not have benefited from a tor communities, and would therefore not bias Bank-financed CBD/CDD intervention; (c) they the results.5 had to be located in rural areas; and (d) they had Some of the selected communities were found to have more than 40 households at the time of to have fewer than 40 households. In these cases, the survey. a census was taken and, where possible, adjacent Inaccuracies in the project's MIS required CBD/CDD communities that received only one changes to the original sample of communities, similar subproject were selected to make up for as well as dropping some communities from the the missing number of respondents. Some mu- analysis. Two project communities were dropped; nicipalities that figured as PAC in the MIS had qualitative data revealed that one had recently been "upgraded" to FUMAC under the on-going applied for funds under the ongoing RPRP, while RPRP. These municipalities continued to be con- the other was the only one to have benefited sidered as PAC for the purpose of this evaluation from a rural electrification investment. The ma- only if no subproject had yet been financed jority of project communities used for the analy- through the FUMAC implementation modality. sis benefited from water supply investments, Table M.2 presents the number of household while three benefited from irrigation invest- surveys conducted and the number used in the ments and two from small bridges. Three of the analysis. six comparator communities surveyed were also Two key informant interviews were conducted dropped. The qualitative data revealed that two in all except five communities, where only the of them had recently applied for funds under the community leader was interviewed. With a few ongoing RPRP, while one of them was the only exceptions, two focus group interviews were one in the sample to have benefited from a conducted in each community. In six commu- government water supply program that had a par- nities only one focus group interview was carried out, and in two communities no focus group session was held. Table M.2: Coverage of Fieldwork in Brazil Madhya Pradesh Twenty project villages were randomly selected Modality FUMAC FUMAC-P PAC Non-Bank from two distinct types of forest zones under the Fieldwork coverage Joint Forest Management (JFM) project in Mad- Municipalities 5 2 3 3 hya Pradesh--10 from each zone. One of the Communities 15 8 4 6 zones is characterized by dense forests (ANR), the Households 514 240 118 225 other by degraded forests (VRDP). In order to Analysis coverage keep logistic and transport costs within the budget, fieldwork was restricted to two districts: Municipalities 5 2 3 2 Betul and Bilaspur, one from the western part of Communities 14 7 4 3 the state and the other from the eastern part Households 485 211 118 117 (now in the state of Chattisgarh). Bilaspur was 1 0 6 A N N E X M chosen because it had been surveyed in March Two focus group interviews and two key in- 2000 by the World Bank's Environmentally and formant interviews were conducted in each vil- Socially Sustainable Development (ESSD) Net- lage, with the exception of one where no focus work. Betul was selected from a list of four dis- group interview was conducted. tricts prepared by the implementing agency based on security concerns because it had a Uttar Pradesh good number of both ANR and VRDP villages. In March-June 2000, the World Bank's ESSD Net- Random selection of villages within each forest work surveyed 19 villages in the district of zone and district was done using a table of ran- Raibareli, which had benefited from the Bank's dom numbers. Sodic Land Reclamation Project. In order to The selection of comparator villages required, allow for comparison over time, fieldwork for the first, identifying areas in the districts of Betul and OED study was conducted in the same area. The Bilaspur under the government-supported JFM. ESSD survey covered villages that were treated This strategy was very similar to that supported during four of the five annual project phases by the Bank. Under JFM, villagers cooperate to (1993­98); none of the 19 villages was treated protect forests in exchange for a share in the during phase II (1994­95). In order to cover all usufruct and the final harvest. The selection of project phases, 4 of the 19 villages were replaced comparator villages from the government JFM with randomly selected villages within Raibareli area was undertaken by the project's imple- district that benefited from the project during menting agency based on the following five cri- phase II. The random selection of these four teria: villages that (a) were located in rural areas; villages was based on the project's database. (b) did not benefit from any Bank CBD/CDD in- The selection of comparator villages required tervention; (c) were within 5 kilometers of the identifying areas in Raibareli district that faced same forest block as project villages; (d) had problems of sodicity of land similar to those similar poverty levels as project villages; and (e) faced by project villages prior to the Bank's in- had between 40 and 80 households. tervention, but which never benefited from any Inaccuracies in subproject records required sodic land reclamation activity. The 11 com- changes to the original sample of communities, parator villages were selected by the imple- as well as dropping some communities from the menting agency of the Bank's Sodic Land analysis. One project village selected in the dis- Reclamation Project (UPBSN) from those that trict of Betul was not found on the map of Betul were in the pipeline to be treated if additional by the Forest Department. This village had to be funds to address sodicity became available. replaced by one in the same area with similar for- UPBSN selected comparator villages based on the est cover. Another village in the same district, following three criteria: villages that (a) were which the records showed as government-JFM, located in rural areas, (b) were located in the was actually a Bank-JFM village, bringing the total number of project villages to 21. Further, qualitative data revealed that two of surveyed Table M.3: Coverage of Fieldwork comparator villages did not have government- in Madhya Pradesh funded JFM, and were hence dropped from the final analysis. The survey data revealed that vil- Type of GoMP Non- lagers did not perceive any difference between forest ANR VRDP JFM JFM a Village Protection Committee, which is set up Fieldwork coverage in degraded forest zone, and a Forest Protection Communities 11 10 7 2 Committee, which is set up in dense forest zone. Households 414 393 261 79 Hence, data analysis was not differentiated by Analysis coverage type of forest zone. Table M.3 presents the num- Communities 11 10 7 -- ber of household surveys conducted and the Households 414 393 261 -- number used in the analysis. 1 0 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT same three blocks as project villages; and (c) Table M.4: Coverage of Fieldwork had similar population size as the project vil- in Uttar Pradesh lages. One comparator village was dropped from Project Comparator the analysis, as the comparison of baseline in- Fieldwork coverage formation collected for this study revealed sig- Communities 19 11 nificant difference between this and the other Households 757 440 villages. Table M.4 presents the number of Analysis coverage household surveys conducted and the number Communities 19 10 used in the analysis.. Households 757 400 Two focus group interviews and two key in- formant interviews were conducted in each vil- lage, with the exception of one where only one of the two key informant interviews was con- past eight years. The household survey en- ducted. quired about respondents': (a) demographic characteristics--age, education, gender, oc- Household Selection cupation, marital status, etc; (b) household Wherever possible, 40 households were selected characteristics, including variables capturing from each community. Two slightly different ap- economic status at the time of the survey and proaches were adopted for household selec- before subproject implementation; (c) aware- tion. In communities where the team had the ness of community problems and participa- information on the total number of households, tion in community-level project organizations; these were divided by the number of interviews (d) perception of sustainability of project in- to be conducted (40) to get an interval of R. vestments; and (e) perceptions of the levels of The households were then arranged in a con- and the changes in social capital and empow- centric manner on the drawing board and a ran- erment.6 dom starting household was selected. Every Rth · Second, semi-structured focus group inter- household was selected until the required num- views held with two groups in each community ber of interviews was complete. In rural dis- (one all-female and one all-male) of 10­15 self- persed communities where there was low initial selected participants. Focus group sessions knowledge of the number of households, the attempted to explore, among other things, community was divided in 4 zones, and 10 house- the following issues: (a) the process of sub- holds were covered in each zone. A similar strat- project selection, implementation, and oper- egy as above was adopted for each zone, but ation; (b) communities' access to information; with a rough estimate from the local leader on (c) the leadership structure within communi- number of households in each zone. In com- ties; (d) the levels and changes in empower- munities with 40 or fewer household, all house- ment; (e) the priority needs of the community holds were surveyed. at the time of the survey and before subpro- ject implementation. Survey Instruments and Timing · Third, structured key informant interviews of Fieldwork held with a community leader and a member Information was collected at the community of the community organization set up by the level using three instruments. These were all project. Key informant interviews used a struc- pilot tested in the field in each of the four proj- tured, open-ended questionnaire. The com- ect areas before being launched. munity leader interview consisted of questions about community facilities, ethnic make-up, · First, a pre-coded household questionnaire, and the like. The interview with a member of which was applied to one adult (25 years or the community organization set up by the proj- older) from each randomly selected house- ect addressed issues of community trust, co- hold who had resided in the community for the hesion, and solidarity, as well as providing 1 0 8 A N N E X M information on the functioning of the com- reported knowing local leaders and local elected munity organization set up by project. officials. In Benin, fieldwork was conducted in Octo- Bivariate analysis was used to compare the ber and November 2003 by a team headed by respondents' perceptions of levels of and Roch Mongbo from the University of Abomey- changes in social capital and empowerment be- Calavi. The fieldwork in Brazil was conducted be- tween the project and the comparator groups. tween November 2003 and January 2004 by a A test of proportion was performed for binary team headed by Alberto Costa from the Univer- variables and the Kruskall-Wallis nonparametric sity for the Development of the Itajaí River Val- test of differences for categorical variables (tables ley. In India, fieldwork was conducted in N.3­N.6). December 2003 and January 2004 by the Center for Development Economics, Delhi School of Multivariate analysis. In order to control for Economics. In all four project areas, an OED differences in geographic, demographic, and so- team member supervised fieldwork activity to en- cioeconomic factors between the project and sure quality. the comparator groups, multivariate analysis was performed on the variables that capture re- Methodology for Household Data Analysis spondents' perceptions of the changes in social capital and empowerment (see tables M.9­M.12 Comparison of ex-ante characteristics of for the list of dependent and independent vari- project and comparator communities. Re- ables used in each country). An Ordered Probit spondents' demographic and socioeconomic model was chosen because the dependent vari- information before the Bank intervention were ables are ordinal ranging from least to most, aggregated at the community level to provide a with most capturing greater outcome. The esti- general profile of the communities covered by mation was performed using population weights fieldwork. A Student t-test was performed on and adjusting for cluster effects.7 these aggregated variables to check whether Two specifications of the same model were the project and the comparator groups had the used; with and without interactive terms. The dis- same mean. While the difficulty of getting per- cussion of the overall association between the fect matches between the project and the com- project and the dependent variables is based on parator group must be acknowledged, an the specification without interactions. The spec- attempt was made to get as close a match as pos- ification with interactions was used in order to sible. As tables M.5­M.8 show, only minimal dif- explore the association between the dependent ferences were found between the project and variables and the project for women, the poor, comparator groups in the four project areas. In and members of project organizations.8 The re- Benin the project group reported a significantly sults of the specification with interactive variables greater number of children below the age of are presented in full (tables N.9­N.16), while a four than the comparator group. In Brazil, a summary of the results of the project dummies greater number of women were interviewed in for the specification without interactions is pre- the comparator group than in the project group. sented in tables N.7­N.8. In Madhya Pradesh, slightly more respondents As already mentioned, all dependent vari- in the comparator group reported knowing ables represent changes over time. It is how- the chairman of the Farmer's Club, while the ever important to note that there are two types project group reported greater ability to or- of change variables: (a) changes as perceived ganize self-help groups and raise resources from and directly reported by respondents, and (b) within the community. In Uttar Pradesh, re- changes derived from respondents' assessment spondents in the project group were slightly of the situation in two points in time--before and older than those in the comparator group, while after subproject implementation. All dependent more respondents in the latter than the former variables that capture changes in social capital 1 0 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT and empowerment are of the first type, with the the dummy variable for poor captures the bot- sole exception of the variable that capture re- tom quartile of economic status within each spondents' mobilization skills in the Brazil proj- community. Therefore, while (a) is a measure of ect, which is of the second type. economic status across the entire sample, (b) The independent variables include commu- represents the relatively poor households within nity characteristics (such as dummy for regions, each community. population of the community, and the like), As already mentioned above (pp. 104, 105), household characteristics (such as household the studies conducted in Benin and Brazil in- size and the index of economic status), and re- cluded three types of communities that bene- spondent characteristics (such as age, level of fited from Bank-supported CBD/CDD projects. education, and so forth). The model used for In Benin, only PAMR communities were con- Benin and Brazil also controls for the type of sub- sidered as project communities; AgeFIB and project financed. Household and respondent PILSA communities were included to elicit dif- characteristics were created drawing on demo- ferences in performance between the three graphic and socioeconomic information before CBD/CDD projects. In Brazil, the RPAP included the Bank intervention as reported by respon- three implementation modalities--PAC, FUMAC, dents. The model includes two variables repre- and FUMAC-P. These could not be combined in senting the respondent's economic status: (a) one project group, as they were differently as- the index of economic status, and (b) a dummy sociated with some of the dependent variables.9 variable for poor. The reason for including both For clarity of exposition, the bivariate analysis re- (a) and (b) is that these are defined differently. ports only the response rate of FUMAC com- While the index for economic status is an ab- munities, which account for 60 percent of solute figure calculated across all respondents, respondents in project communities. Table M.5: Benin: Comparison of Ex-Ante Characteristics of Project and Comparator Communities CBD/CDD Comparator Household size (member above the age of 4) 8.75 7.39 Number of children above the age of 4 3.00 2.50 Number of children below the age of 4 1.93 1.07 *** Schooling of the respondent 0.26 0.33 Age of the respondent 42.71 45.68 Dummy for female 0.31 0.31 Knew community leaders 0.91 0.94 Knew religious leaders 0.86 0.89 Knew the mayor 0.47 0.52 Participation in traditional events 2.86 2.73 Participation in political events 2.33 2.40 Ability to raise resources from within the community 0.58 0.67 Ability to raise funds outside the community 0.38 0.33 Ability to speak freely with community leaders 0.64 0.59 Ability to express community needs to local government officials 0.50 0.55 Blue collar skills 0.46 0.51 Note: Test of significance based on Student t-test. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 1 0 A N N E X M Table M.6: Brazil: Comparison of Ex-Ante Characteristics of Project and Comparator Communities CBD/CDD Comparator Municipal Human Development Index 0.62 0.62 Score for community 0.23 0.27 Population of community 59.56 100.00 Household size 4.42 4.83 Number of children 1.56 1.78 Medium consumer durables 1.74 1.80 Large consumer durables 0.23 0.32 Large animals 4.27 1.95 Small animals 7.91 4.14 Schooling of the respondent 2.63 2.80 Dummy for female 0.43 0.71 *** Age of the respondent 47.55 45.15 Dummy for agricultural laborer 0.56 0.38 Participation in political events 1.91 2.00 Participation in traditional events 1.94 2.05 Ability to raise resources from within the community 0.39 0.36 Ability to raise funds outside the community 0.35 0.30 Ability to speak freely with community leaders 0.50 0.35 Ability to express community needs to local government officials 0.58 0.60 Note: Test of significance based on Student t-test. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 1 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table M.7: Madhya Pradesh: Comparison of Ex-Ante Characteristics of Project and Comparator Communities CBD/CDD Comparator Population of community 117.76 127.00 Score for community 0.33 0.37 Household size 6.82 6.61 Number of children 2.84 2.99 Land owned 1.58 1.36 Small consumer durables 0.09 0.07 Medium consumer durables 0.45 0.47 Large consumer durables 0.02 0.00 Large animals 2.94 2.83 Small animals 0.71 0.40 Schooling of the respondent 0.36 0.37 Dummy for female 0.47 0.47 Age of the respondent 39.24 38.40 Knew village leader 0.59 0.47 Knew Sarpanch 0.80 0.80 Knew Farmers' Club chairperson 0.02 0.06 *** Knew local elected officials 0.15 0.16 Knew forest staff 0.63 0.67 Participation in traditional events 3.27 3.20 Participation in non-traditional events 2.27 2.28 Ability to organize self-help groups/raise resources from within the community 0.21 0.12 *** Ability to raise funds outside the community 0.12 0.10 Ability to express community needs to local government officials 0.37 0.39 Note: Test of significance based on Student t-test. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 1 2 A N N E X M Table M.8: Uttar Pradesh: Comparison of Ex-Ante Characteristics of Project and Comparator Communities CBD/CDD Comparator Population of community 276.89 160.50 Score for community 0.62 0.69 Household size 8.18 7.71 Number of children 3.49 3.36 Amount of land owned 81.72 68.72 Small consumer durables 0.26 0.43 Medium consumer durables 0.97 1.10 Large consumer durables 0.04 0.04 Large animals 2.42 2.19 Small animals 0.83 0.51 Schooling of respondent 0.46 0.45 Dummy for female 0.49 0.50 Age of the respondent 44.61 42.42 ** Knew local leaders 0.79 0.90 ** Knew Farmers' Club chairperson 0.01 0.01 Knew local elected officials 0.41 0.60 *** Participation in traditional events 2.96 2.89 Participation in non-traditional events 2.24 2.10 Ability to raise funds outside the community 0.13 0.09 Ability to express community needs to local government officials 0.45 0.37 Blue collar skills 0.36 0.34 Note: Test of significance based on Student t-test. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 1 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table M.9: Definition of Variables: Benin Dependent variable Definition Change in access to information Changes in access to information regarding development activities in the community (more=3, same=2, less=1) Change in mobilization skills Composite variable equal to the sum of four dummy variables (1=more, 0=everything else) that capture change in the respondent's ability to (a) mobilize community efforts and resources; (b) raise funds out- side the community; (c) speak freely with community leaders; (d) express the needs of the community to local government officials. Change in ability to reach agreement Change in the community's ability to reach an agreement (more=3, same=2, less=1) Change in community leaders' Change in the extent to which community leaders and local government officials listen and respond to responsiveness to community needs community needs (Listen and respond more=4, listen more=3, same=2, listen less=1). Change in trust Composite variable equal to the sum of four dummy variables (1=more, 0=everything else) capturing change in trust in: (a) community members, (b) community organizations, (c) local leaders, (d) government officials. Change in associational life Composite variable equal to the sum of two dummy variables (1=more, 0=everything else) capturing change in: (a) villagers' participation in groups and associations, (b) cooperation between groups and individuals. Change in participation in Change in the respondent's participation in community's traditional events (more=3, same=2, less=1) traditional events Change in participation in Change in the respondent's participation in community's political events (more=3, same=2, less=1) political events Change in circle of friends Change in the respondent's circle of friends (improved=3, same=2, deteriorated=1) Independent variable PAMR Equals one if PAMR community, zero otherwise AgeFIB Equals one if AgeFIB community, zero otherwise PILSA Equals one if PILSA community, zero otherwise Female in PAMR Equals one if respondent is a female from PAMR community, zero otherwise School construction subproject Equals 1 if school construction subproject, zero otherwise Training subproject Equals 1 if training subproject, zero otherwise Difficult access to community Equals one if community is geographically difficult to access, zero otherwise Household size Number of people living under the same roof Number of children Number of children below the age of 16 Dummy for female Equals 1 if the respondent is a female and zero if male Schooling Equals one if the respondent has attended school, zero otherwise Age Age of the respondent Age squared Age squared Number of leaders known Number of leaders the respondent knew prior to the Bank intervention Participation in traditional events Frequency of participation in traditional events prior to Bank intervention Participation in political events Frequency of participation in political events prior to Bank intervention Blue collar skills of the respondent Equals 1 if the respondent was able to do blue collar activities (such as masonry, carpentry, stitching, etc.) prior to Bank intervention Mobilization skills of the respondent Number of skills the respondent reported to have prior to Bank intervention. Composite variable equal to the sum of four dummy variables (1=able, 0=everything else) that captures respondent's ability to (a) mobilize community efforts and resources; (b) raise funds outside the community; (c) speak freely with community leaders; (d) express the needs of the community to local government officials. 1 1 4 A N N E X M Table M.10: Definition of Variables: Brazil Dependent variables Definition Change in access to information Changes in access to information regarding development activities in issues of interest to the commu- nity (more=3, same=2, less=1) Change in mobilization skills Composite variable equal to the sum of four dummy variables (1=more, 0=everything else) that capture changes in the respondent's ability to (a) raise resources from within the community; (b) raise funds out- side the community; (c) speak freely with community leaders; (d) express the needs of the community to local government officials. The dummies for change were derived from respondent's assessment of their skills in two points in time--before and after subproject implementation. Change in ability to reach agreement Change in the community's ability to reach an agreement (more=3, same=2, less=1) Change in leaders' responsiveness Change in community leaders' responsiveness to communities demands (more=3, same=2, less=1) Change in trust Composite variable equal to the sum of four dummy variables (1=more, 0=everything else) capturing change in trust in: (a) community members, (b) community associations, (c) municipal government officials, and (d) state government officials. Change in associational life Composite variable equal to the sum of two dummy variables (1=more, 0=everything else) capturing change in: (a) people's participation in groups, (b) cooperation between groups and individuals. Change in participation in Change in the respondent's participation in community's traditional events (more=3, same=2, less=1) traditional events Change in participation in Change in the respondent's participation in community's political events (more=3, same=2, less=1) political events Change in circle of friends Change in the respondent's circle of friends (improved=3, same=2, deteriorated=1) Independent variables Definition PAC Equals 1 if RPAP implemented through PAC modality, zero otherwise FUMAC Equals 1 if RPAP implemented through FUMAC modality, zero otherwise FUMACP Equals 1 if RPAP implemented through FUMAC-P modality, zero otherwise Poor in PAC Equals 1 if respondent is poor and in a PAC community, zero otherwise. Poor in FUMAC Equals 1 if respondent is poor and in a FUMAC community, zero otherwise. Poor in FUMAC-P Equals 1 if respondent is poor and in a FUMAC-P community, zero otherwise. Irrigation subproject Equals 1 if irrigation subproject, zero otherwise Small bridge subproject Equals 1 if small bridge subproject, zero otherwise Agreste region Equals 1 if Agreste region, and zero if Oeste region Municipal Human Development Index Municipal Human Development Index 2000 Score for community Level of basic infrastructure in a community (such as primary school, basic health post, water supply system, electrification, telephone boots, etc.) prior to Bank intervention (based on village leader inter- view). Economic status index Composite variable equal to the sum of two rebased variables that capture household's ownership of the following items prior to subproject implementation: (a) large animals (horse, cow, and ox), and (b) consumer durables (car, motorcycle, bicycle, freezer, television, satellite dish) Dummy for poor Equals 1 if respondent is from the bottom-quartile of the distribution along the Economic Status Index in his/her community, zero otherwise Household size Number of people living under the same roof Number of children Number of children below the age of 16 Member of CA Equals 1 if member of Community Association set up by the Bank CBD/CDD project, zero otherwise Dummy for female Equals 1 if respondent is a female, zero otherwise (continued on following page) 1 1 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table M.10: Definition of Variables: Brazil (continued) Independent variable Definition Schooling Level of education attained by the respondent (5=some secondary and above, 4=completed primary, 3=some primary, 2=literate, 1=illiterate). Agricultural laborer Equals 1 if the respondent is an agricultural laborer, zero otherwise Age Age of the respondent Age squared Age squared Participation in political events Frequency of participation in political events prior to Bank intervention Participation in traditional events Frequency of participation in traditional events prior to Bank intervention Mobilization skills Number of skills the respondent reported to have prior to Bank intervention. Composite variable equal to the sum of four dummy variables (1=able, 0=everything else) that captures respondent's ability to (a) raise resources from within the community; (b) raise funds outside the community; (c) speak freely with community leaders; (d) express the needs of the community to local government officials. Table M.11: Definition of Variables: Madhya Pradesh Dependent variable Definition Change in access to information Changes in access to information regarding issues of interest to the community (more=3, same=2, less=1) Change in mobilization skills Composite variable equal to the sum of three dummy variables (1=more, 0=everything else) that cap- ture change in the respondent's ability to (a) organize self-help groups and raise resources from within the village; (b) raise resources outside the village; (c) express the needs of the village to local govern- ment officials. Change in ability to reach agreement Change in the community's ability to reach an agreement (more=3, same=2, less=1) Change in community leaders' Change in the extent to which community leaders listen and respond to community needs (listen and responsiveness respond more=4, listen more=3, same=2, listen less=1). Change in trust Composite variable equal to the sum of five dummy variables (1=more, 0=everything else) that capture change in respondent's trust in: (a) village members, (b) village organizations, (c) village leaders, (d) local government officials, (e) staff of the forest department (implementing agency). Change in associational life Composite variable equal to the sum of two dummy variables (1=more, 0=everything else) that capture change in: (a) people's participation in groups, (b) cooperation between groups and individuals. Change in participation in Change in the respondent's participation in community's traditional events. (more=3, same=2, less=1). traditional events Change in participation in Change in the respondent's participation in community's non-traditional events. (more=3, same=2, political events less=1). Change in circle of friends Change in the respondent's circle of friends (improved=3, same=2, deteriorated=1) Independent variable Project village Equals 1 if project village, and zero if comparator Poor in project village Equals 1 if respondent is poor and in a project village, zero otherwise Betul district Equals 1 if Betul district, and zero if Bilaspur district Score for community Level of basic infrastructure in a community (such as primary school, basic health post, wells, hand-pumps) and distance to the closest market prior to Bank intervention (based on village leader interview) 1 1 6 APPENDIX H: RESULTS FROM THE MAP SELF-ADMINISTERED QUESTIONNAIRE Table M.11: Definition of Variables: Madhya Pradesh (continued) Independent variable Definition Economic status index Composite variable equal to the sum of three rescaled variables that capture household's ownership of the following items prior to subproject implementation: (a) land (linear log), (b) large animals (horse, cow, and ox), (c) consumer durables (car, bicycle, fan, radio) Dummy for poor Equals 1 if respondent is from the bottom-quartile of the distribution along the Economic Status Index in his/her community, zero otherwise Household size Number of people living under the same roof Number of children Number of children below the age of 16 Member of forest committee Equals 1 if member of forest committee set up by Bank CBD/CDD project, zero otherwise Dummy for female Equals 1 if respondent is female, zero otherwise Age Age of the respondent Age squared Age squared Schooling Equals 1 if the respondent has attended school, zero otherwise Number of leaders known Number of leaders a respondent knew prior to the Bank intervention Participation in traditional events Frequency of participation in the traditional events prior to Bank interventions Participation in non-traditional events Frequency of participation in the non-traditional/political events prior to Bank interventions Mobilization skills Number of skills the respondent reported to have prior to Bank intervention. Composite variable equal to the sum of three dummy variables (1=able, 0=everything else) that captures respondent's ability to (a) organize self-help groups and raise resources from within the village; (b) raise resources outside the village; (c) express the needs of the village to local government officials Table M.12: Definition of Variables: Uttar Pradesh Dependent variable Definition Change in access to information Changes in access to information regarding issues of interest to the community (more=3, same=2, less=1) Change in mobilization skills Composite variable equal to the sum of three dummy variables (1=more, 0=everything else) that cap- ture change in the respondent's ability to (a) organize self-help groups and raise resources from within the village; (b) raise resources outside the village; (c) express the needs of the village to local govern- ment officials. Change in ability to reach agreement Change in the community's ability to reach an agreement (more=3, same=2, less=1) Change in community leaders' Change in the extent to which community leaders listen and respond to community needs (listen and responsiveness respond more=4, listen more=3, same=2, listen less=1) Change in trust Composite variable equal to the sum of five dummy variables (1=more, 0=everything else) capturing change in trust in: (a) village members, (b) village organizations, (c) village leaders, (d) local elected officials, (e) implementing agency Change in associational life Composite variable equal to the sum of two dummy variables (1=more, 0=everything else) capturing change in: (a) people's participation in groups, (b) cooperation between groups and individuals Change in participation in Change in the respondent's participation in community's traditional events (more=3, same=2, less=1) traditional events Change in participation in Change in the respondent's participation in community's non-traditional events. (more=3, same=2, non-traditional events less=1). (continued on following page) 1 1 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table M.12: Definition of Variables: Uttar Pradesh (continued) Dependent variables Definition Change in circle of friends Change in the respondent's circle of friends (improved=3, same=2, deteriorated=1) Independent variable Project village Equals 1 if project village, and zero otherwise (if comparator) Poor in project village Equals 1 if respondent is poor and in project village, zero otherwise Female in project village Equals one if the respondent is a female in a project village, zero otherwise Amawa block Equals 1 if Amawa block, zero otherwise Maharajganj block Equals 1 if Maharajganj block, zero otherwise Rural community Equals 1 if it is a rural community, zero if rural dispersed Score for community Level of basic infrastructure in a community (such as primary school, basic health post, wells, hand-pumps) and distance to the closest market prior to Bank intervention (based on village leader interview) Economic status index Composite variable equal to the sum of three rescaled variables that capture household's ownership of the following items prior to subproject implementation: (a) land (linear log), (b) large animals (horse, cow, and ox), (c) consumer durables (car, bicycle, fan, radio) Dummy for poor Equals 1 if respondent is from the bottom-quartile of the distribution along the Economic Status Index in his/her community, zero otherwise Household size Number of people living under the same roof Number of children Number of children below the age of 16 Member of SIC Equals 1 if member of village organization set up by the Bank CBD/CDD project, zero otherwise Dummy for female Equals 1 if respondent is a female, zero otherwise Age Age of the respondent Age squared Age squared Schooling Equals 1 if the respondent has attended school, zero otherwise Number of leaders Number of leaders the respondent knew prior to the Bank intervention Participation in traditional events Frequency of participation in the traditional events prior to Bank interventions Participation in non-traditional events Frequency of participation in the non-traditional/political events prior to Bank interventions Blue collar skills of the respondent Equals 1 if respondent was able to do boring, construct field drains, link drains and/or construct irriga- tion channels prior to Bank intervention Mobilization skills of the respondent Number of skills the respondent reported to have prior to Bank intervention. Composite variable equal to the sum of three dummy variables (1=able, 0=everything else) that captures respondent's ability to (a) organize self-help groups and raise resources from within the village; (b) raise resources outside the village; (c) express the needs of the village to local government officials 1 1 8 ANNEX N: ENHANCING COMMUNITY CAPACITIES One of the premises of the CDD approach is that tions are presented in full (tables N.9­N.16), it fosters the formation of social capital at the while a summary of the results of the project community level and empowers communities to dummies for the specification without interac- take charge of their own development.1 The as- tions is presented in tables N.7 and N.8. Unless sessment of the extent to which the CDD proj- otherwise specified, the discussion of the sta- ects supported by the World Bank have tistical significance is always based on the prob- improved communities' capacity focuses on ability value of the regression coefficients of the these two processes. This annex presents the relevant specification. findings of community-level fieldwork under- taken for this evaluation in the Borgou region Empowerment of Benin (henceforth Benin); the state of Rio The World Bank's (2002d) sourcebook on em- Grande do Norte in Brazil (henceforth Brazil); powerment and poverty reduction identifies the Betul and Bilaspur districts in Madhya four key elements for a successful empower- Pradesh, India (henceforth Madhya Pradesh); ment strategy: (a) inclusion and participation, (b) and Raibarelli district in Uttar Pradesh, India access to information, (c) accountability, (d) and (henceforth Uttar Pradesh). Three of these proj- local organizational capacity. This understanding ects were CDD and one was CBD. Henceforth, of empowerment has informed data collection when reference is made to the four projects, the for this study, which explores both the levels of broader term CBD/CDD is used. In addition, empowerment at the time of fieldwork and where relevant, this annex draws on a review of respondents' perceptions of changes in em- appraisal documents of 84 sampled CBD/CDD powerment before and after subproject imple- projects and on the literature on participatory mentation. While a comprehensive assessment development. of the levels of empowerment and the empow- The field research at the community level in- ering effects of the Bank's CBD/CDD initiatives cluded household surveys, focus group inter- at the community-level was beyond the scope of views, and key informant interviews (see Annex this study, our analysis focuses on some aspects M for details). The methodology adopted for of the four above-mentioned elements. It should the analysis of the household data as well as de- also be noted that empowerment is multidi- tails of the model used for multivariate analysis mensional and it is hence possible for a person are also discussed in Annex M . Results of the bi- to experience empowerment in one dimension variate analysis are presented in tables N.3­N.6, and disempowerment in another. while those of the multivariate analysis are pre- sented in tables N.7­N.16.2 It should be noted Inclusion and Participation that in the first two sections, discussion of the CBD/CDD projects are operationalized at the results of the multivariate analysis refers to the community level through community organiza- model without interactive terms, while the last tions. While at times these predate CBD/CDD in- two sections draw on the model with interactive terventions, new ad hoc organizations are often terms (Annex M). As already mentioned in Annex created by CBD/CDD projects, as was the case M, the results of the specification with interac- in Benin, Brazil, Madhya Pradesh, and Uttar 1 1 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Pradesh.3 Because these organizations consti- aries reported speaking at the meetings in the tute the locus of decision making at the local four countries surveyed. Focus group sessions level, becoming a member (or attending meet- held with villagers across the four project areas ings) is extremely important in order to attain in- indicate that decision-making processes relative clusion in decision making. This, however, is to the Bank-funded subproject lacked broad not in itself sufficient. Drawing on the literature community participation. on participatory development, we make a dis- It can be argued that a low level of inclusion tinction between formal inclusion, which con- in community organizations responsible for sub- cerns the extent to which community members project selection and management is not in itself are able to enter decision-making arenas, and problematic. It is unrealistic and perhaps ineffi- substantive inclusion, which captures the ex- cient to expect communities to collectively un- tent to which different participants are able to dertake such activities, and a group of villagers exert influence over decisions.4 While assessing could instead be chosen to do so on behalf of the formal inclusion is a fairly simple exercise, as- community. Though valid, this argument raises sessing substantive inclusion is far more complex, concerns regarding the ways in which commu- as it requires a detailed analysis of the very nity representatives are selected and the inclu- process through which decisions are made. This sion of weaker social groups. In three of the type of investigation was beyond the reach of our four project areas for which information is field research. However, based on our data, we available, respondents who were members of can assess the extent to which villagers were community organizations set up by the Bank in- likely to attain substantive inclusion in decision tervention had a higher socioeconomic profile, making. including greater mobilization skills and a more Our household data reveal that in Benin, a extensive social network than non-members large share of respondents attained formal in- prior to subproject implementation.7 Similarly, clusion in subproject decision making, while the a large share of focus group interviews in India opposite holds for Brazil and Uttar Pradesh. In and Benin pointed out that decision making re- Madhya Pradesh the picture is somewhat mixed. garding the subproject was largely controlled As figure N.1 shows, in Benin, 72 percent of re- by local leaders (figure 3.6 in Chapter 3). Various spondents attended the meetings for subproject studies in the literature on participatory devel- selection, while in Brazil, only 37 percent of re- opment also point out that the better-educated spondents in FUMAC communities attended members of the community and the relatively these meetings, and in Uttar Pradesh only 16 better-off are often the ones who represented the percent of respondents were members of proj- community in participatory intervention (Desai ect organizations and only 13 percent attended 1996; Gibson and Marks 1995; van der Linden meetings regularly.5 In Madhya Pradesh, over 1997; Ribot 1998).8 half of the respondents were members of proj- ect organizations, but only a third of respon- Access to Information dents attended meetings regularly. In Brazil, the As the World Bank's (2002d) sourcebook on em- majority of respondents were also likely to have powerment puts it, "information is power.... exerted only minimal influence over subproject Without information that is relevant, timely, and decision making. Only 22 percent of the re- presented in forms that can be understood, it is spondents in FUMAC communities spoke during impossible for poor people to take effective ac- the meetings (figure N.1), and a large share of tion." Ensuring people's access to information is respondents would refrain from expressing griev- particularly important in CDD projects, in which ances with the subproject being implemented if communities are expected to take a proactive this risked losing projects funds or compromis- role in initiating the subproject cycle. A review ing relations with other villagers.6 The OED So- of the appraisal documents for our sample of 84 cial Fund Evaluation (OED 2002b) reports similar interventions reveals that fewer than half of them results; only around 15 percent of the benefici- included an extensive campaign to disseminate 1 2 0 A N N E X N Figure N.1: Beneficiaries' Inclusion and Participation in Subproject Decision Making 100 96% 80 72% 62% 60 respondents of 40 34% 37% 22% Percentage 20 0 Benin [N = 566] Brazil [N = 485] Aware of meetings Attended meetings Spoke at meetings 100 92% 80 60 52% respondents 50% of 40 33% Percentage 20 16% 13% 0 Madhya Pradesh [N=807] Uttar Pradesh [N=757] Aware of project Member of project Attended meetings organizations organizations regularly project information.9 When information is not Household data also reveal communities' lack disseminated widely, communities are likely to of information regarding the subproject imple- be dependent on a few informed individuals for mented. In Benin and Brazil, the vast majority of accessing development opportunities, and as respondents in project communities had no in- the focus group interviews in Benin and India re- formation on the cost of the subproject--86 and veal, these tend to be the local leaders. Con- 82 percent, respectively.10 Evidence from the lit- trolling information reinforces the position of erature on northeast Brazil supports these find- power of these leaders, and creates opportuni- ings (Tendler 2000).11 Communities' lack of ties for strengthening their clientelistic network information on the subproject affects the ability (Kumar and Corbridge 2002; Desai 1996; Das of the community to hold to account the peo- Gupta and others 2000). ple who managed the subproject investments on 1 2 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT its behalf, and provides further evidence of the information are critical for accountability, and the general lack of broad-based community partici- findings discussed above on access to infor- ma- pation in Bank-funded initiatives. tion already indicate weak accountability to com- In addition to exploring the level of informa- munities. Although necessary, access to tion at the time of our fieldwork, the household information is not in itself sufficient, as it does not surveys also captured respondents' perceptions automatically result in accountability; citizens of the change in access to information on is- must act upon the information they acquire (Jenk- sues of interest to the communities before and ins and Goetz 1999). While it was beyond the after subproject implementation. Our findings re- scope of our field research to explore accounta- veal a mixed picture (figure N.2). A significant bility issues extensively, a few aspects of these is- positive association was found between the sues were captured by our questionnaires. Bank's CBD/CDD projects in Madhya Pradesh A large share of respondents in project com- and respondents' access to information on issues munities in Benin, Uttar Pradesh, and Madhya of relevance to the community, while no signif- Pradesh and over half of those in FUMAC com- icant association was found in Benin and Uttar munities in Brazil agreed that if dissatisfied with Pradesh. In Brazil, respondents in comparator the performance of community leaders, villagers communities reported a significantly higher in- would call a meeting to discuss it. More than half crease in access to information than did re- of the respondents in project communities in spondents in FUMAC communities.12 Benin and Madhya Pradesh also agreed that if dis- satisfied with community leaders, the community Accountability would replace them. Fewer respondents agreed The notion of accountability has a range of con- with this statement in project communities in notations. It is used here to refer to citizens' abil- Uttar Pradesh and FUMAC communities in ity to hold local leaders and public officials Brazil--respectively 42 and 24 percent. Our accountable. The availability and accessibility of household data also capture respondents' per- Figure N.2: The Bank's CBD/CDD Projects and Access to Information 100 (%) 80 increase an 60 # # reporting 40 20 Respondents 0 Benin Brazil Madhya P. Uttar P. [N = 772] [N = 602] [N = 871] [N = 992] Project Comparator #Differenceisstatisticallysignificant Note: Statistical significance based on the model without interactions. 1 2 2 A N N E X N ceptions of the changes in community leaders' third aspect relevant to local organizational ca- responsiveness to community needs before and pacity (which is also a dimension of social cap- after subproject implementation. Multivariate ital) is the change in associational life, which analysis indicated no statistically significant as- captures changes in respondents' participation sociation between the Bank's projects in Benin in community groups and changes in coopera- and Madhya Pradesh and community leaders' tion between community groups. responsiveness to community needs. In Brazil, Multivariate analysis indicated no statistically respondents in all three types of project com- significant association between the Bank's proj- munities reported a significantly smaller increase ects and the changes in respondents' mobiliza- in community leaders' responsiveness than did tion skills, with the exception of Uttar Pradesh, respondents in comparator communities. where respondents in project communities re- ported a significantly greater increase in mobi- Local Organizational Capacity lization skills than did respondents in comparator According to the Bank's empowerment source- communities. The relation between Bank's book, local organizing capacity "refers to the CBD/CDD projects and communities' ability to ability of people to work together, organize reach an agreement was mixed (figure N.3). The themselves, and mobilize resources to solve projects in Benin and Uttar Pradesh were posi- problems of common interest" (World Bank tively associated with communities' ability to 2002b). In order to assess the extent to which the reach an agreement, while no statistical associ- Bank's interventions succeeded in fostering com- ation was found in Madhya Pradesh. In Brazil, re- munities' organizational capacity, our surveys spondents in all three types of project captured respondents' perceptions of the communities reported a significantly smaller in- changes in their mobilization skills, and in the crease in their ability to reach an agreement ability of the community to reach an agreement than did respondents in comparator communi- before and after subproject implementation.13 A ties. A more positive picture emerges from the Figure N.3: The Bank's CBD/CDD Project Communities' Ability to Reach an Agreement 100 # (%) 80 # increase # an 60 reporting 40 20 Respondents 0 Benin Brazil Madhya P. Uttar P. [N = 772] [N = 602] [N = 1,068] [N = 1,157] Project Comparator #Differenceisstatisticallysignificant Note: Statistical significance based on the model without interactions. 1 2 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT analysis of the changes in associational life (fig- associated with respondents' perceptions of the ure N.4). All four Bank's projects are positively changes in associational life.14 Figure N.4: The Bank's CBD/CDD Projects and Social Capital Benin [N = 772] Madhya Pradesh [N = 1,068] 100 100 (%) (%) 80 # 80 increase increase an 60 an 60 # # # reporting 40 reporting 40 # # # 20 20 # Respondents Respondents 0 0 Trust life Trust life Associa- Politicalevents Friends Associa- Politicalevents Friends tional Traditional events tional Traditional events PAMR Comparator Project Comparator Brazil [N = 602] Uttar Pradesh [N = 1,157] 100 100 (%) (%) 80 80 increase increase # an 60 # an 60 # # reporting 40 # reporting 40 # # 20 20 Respondents Respondents 0 0 Trust life Trust life Associa- Politicalevents Friends Associa- Politicalevents Friends tional Traditional events tional Traditional events FUMAC Comparator Project Comparator # Differenceisstatisticallysignificant Note: Statistical significance based on the model without interactions. 1 2 4 ANNEX N Social Capital influence on social capital of the Bank's initiative Social capital refers to the norms and networks in Rio Grande do Norte. Third, a lengthy en- that enable collective action.15 By drawing peo- gagement with a consistent capacity-building ple together to collectively decide and manage strategy with the same communities is likely to project activities and outputs, these projects are yield better results than a brief one. This might expected to expand the depth and range of com- explain why the Borgou Pilot Project, which was munities' social networks. In order to assess the introduced in communities where the Bank's extent to which Bank-funded interventions have Village Level Participatory Approach had been succeeded in enhancing social capital at the implemented in the 1990s and shared its ap- community level, we draw on five variables, proach, outperformed the Social Fund (AgeFIB) which capture respondents' perceptions of the and the Food Security Project (PILSA).19 changes in (a) trust, (b) associational life, (c) par- ticipation in traditional events, (d) participation Bank's Projects and Members in non-traditional/political events, and (e) circle of Community Organizations of friends before and after subproject imple- As already mentioned above, new ad hoc com- mentation.16 It is important to bear in mind that munity organizations were created by the Bank's these variables capture only some of the multi- projects for their operationalization at the com- ple dimensions of social capital and that our munity level (see endnote 3). This section ex- analysis of the association between Bank-sup- plores the institutional development impact of ported projects and social capital was limited the CBD/CDD projects in Brazil, Madhya Pradesh, to the changes observed in these five dimensions. and Uttar Pradesh on the members of these or- Multivariate analysis indicated a statistically ganizations. In order to do so, a variable inter- significant and positive association between the acting membership in project organizations and projects in Uttar Pradesh, Madhya Pradesh, and the project dummy was included in the model. Benin and respondents' perceptions of the This variable estimates the association between changes in social capital.17 Results for the Brazil the dependent variable and membership in proj- projects are mixed. While respondents in project ect organizations relative to the project dummy. communities reported a significantly greater in- The multivariate analysis indicated a statisti- crease in associational life than did respondents cally significant and positive association between in comparator communities, the opposite holds the projects in Brazil, Madhya Pradesh, and Uttar for respondent's perceptions of changes in their Pradesh, and the changes in empowerment re- circle of friends (figure N.4).18 There are three rea- ported by members of community organizations sons for the different levels of change in social set up by these projects (table N.1). In Brazil, capital in the four project areas, which might members reported a greater increase in access to also explain the differences in the changes in information and community leaders' responsive- empowerment. First, communities in the four ness to community needs than non-members. In project areas are likely to have different capacity Madhya Pradesh, members reported a greater in- levels; therefore, the change that can be expected crease in access to information and mobilization as a result of exposure to a Bank intervention is skills than non-members, while in Uttar Pradesh, also likely to be different. Second, the socio-po- they reported a greater increase in mobilizations litical setting in which these initiatives are im- skills and communities' ability to reach an agree- plemented affects their impact on social capital ment. A statistically significant and positive asso- enhancement. The literature notes that the per- ciation is also found between the projects in vasive clientelism in the northeast of Brazil cre- Madhya Pradesh and Uttar Pradesh, and respon- ates a social system in which vertical ties of mutual dent's perceptions of the changes in social capi- dependence prevail, and hinder the develop- tal. These two Bank projects are positively ment of strong horizontal links of solidarity be- associated with four of the five dimensions of so- tween communities (Costa and others 1997; cial capital considered in this study (table N.2). A Tendler 2000). This probably explains the lack of weaker association is found between the Brazil 1 2 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.1: Empowering Members of Community Organizations Change in... Brazil Madhya Pradesh Uttar Pradesh Access to information positive positive Mobilization skills positive positive Ability to reach agreement positive Community leaders' responsiveness to community needs positive Table N.2: Enhancing the Social Capital of Members of Community Organizations Change in... Brazil Madhya Pradesh Uttar Pradesh Trust in individuals/organizations positive positive Associational life positive positive positive Participation in traditional events Participation in non-traditional/political events positive positive Circle of friends positive positive Note: Results for the dummy for members of project organizations in CBD/CDD communities estimated in the model with interactions (tables N.10­N.16). project and changes in social capital, with mem- in project communities relative to the project bers of Community Associations reporting a greater dummy. increase in only one of the five dimensions of so- The multivariate analysis indicated that the cial capital considered--associational life. poor in project communities in Madhya Pradesh While encouraging, these findings raise im- reported a significantly greater increase in two portant concerns, because, as pointed out ear- dimensions of social capital--trust and associa- lier (see endnote 7), respondents who were tional life--as well as a greater increase in their members of project-induced community or- mobilization skills than did the relatively better- ganizations had a higher socioeconomic profile, off in project areas. In Uttar Pradesh, the asso- including greater mobilization skills and a more ciation between changes in empowerment and extensive social network than non-members the poor in project communities is mixed. While prior to subproject implementation. the poor in project areas reported a greater in- crease in the community's ability to reach an Bank's Projects and Vulnerable Groups agreement than did the relatively better-off in Bank's CBD/CDD projects are aimed at em- CBD/CDD communities, they also reported a powering and enhancing the social capital of significantly smaller increase in their access to in- vulnerable groups, including women and the formation. In Brazil, no significant association is poor (endnote 1). Two interactive variables were found between changes in social capital and em- included in the regression model in order to powerment and the poor in project communi- explore the associations between the changes in ties. social capital and empowerment and the poor in The projects in Benin and Uttar Pradesh, project communities, on the one hand, and which explicitly targeted women, do not appear women in project communities, on the other. to have enhanced women's capacities over and The first variable interacts belonging to the bot- above other respondents in project communities. tom quartile of the index of economic status The only exceptions are women in project areas with the project dummy, while the second in- in Uttar Pradesh who reported a significantly teracts being a women with the project dummy. greater increase in their mobilization skills and These variables estimate the association between associational life than did men in project com- the dependent variable and women or the poor munities. 1 2 6 ANNEX N Table N.3: Benin: Bivariate Analysis of Variables Relevant to the Discussion on Empowerment and Social Capital (percent) PAMR, 566 Comparator, 206 observations observations Change in access to information Worse 1 2 ** Same 12 17 Better 86 80 Change in mobilization skills Same 10 16 ** Increase in 1 of 4 10 8 Increase in 2 of 4 16 14 Increase in 3 of 4 20 29 Increase in all 4 45 34 Change in ability to reach an agreement Worse 1 9 *** Same 14 22 Better 85 68 Change in community leaders' responsiveness to community needs Less 3 1 ** Same 25 35 Listen more 44 40 Listen and respond more 28 24 Change in trust in individuals and organizations Same 32 43 *** Increase in 1 of 4 11 12 Increase in 2 of 4 17 24 Increase in 3 of 4 26 12 Increase in all 4 14 8 Change in associational life Same 18 25 *** Increase in 1 of 2 43 50 Increase in 2 of 2 39 25 Change in participation in traditional events Less 9 8 Same 47 55 More 44 37 Change in participation in political eventsa Less 12 11 Same 54 50 More 35 38 Change in circle of friends Less 1 2 ** Same 25 32 More 74 66 Express grievances if this risks losing project funds or No 50 58 compromising relations with other villagers Yes 49 42 If unhappy with community leaders, villagers call a Disagree 6 9 *** meeting to discuss itb Somewhat disagree 5 6 Somewhat agree 15 23 Agree 74 62 If unhappy with community leaders, Disagree 19 27 *** villagers replace them c Somewhat disagree 11 13 Somewhat agree 13 13 Agree 57 47 (continued on following page) 1 2 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.3: Benin: Bivariate Analysis of Variables Relevant to the Discussion on Empowerment and Social Capital (percent) (continued) PAMR, 566 Comparator, 206 observations observations Participation at community meetings for subproject selection Unaware of meetings 4 Aware of meetings 96 Attended meetings 72 Attended and spoke at meetings 34 Note: Significance based on a test of proportion for binary variables and the Kruskall-Wallis test for categorical variables. * Significant at 10%; ** significant at 5%; ***significant at 1%. a. No. observations: project = 562 b. No. observations: project = 561 c. No. observations: project = 555; comparator = 202 Table N.4: Brazil: Bivariate Analysis of Variables Relevant to the Discussion on Empowerment and Social Capital (percent) FUMAC, 485 Comparator, 117 observations observations Change in access to information a Worse 16 3 *** Same 38 41 Better 46 56 Change in mobilization skills Same 60 53 Improve in 1 of 4 20 33 Improve in 2 or more of 4 20 14 Change in ability to reach an agreement Worse 13 2 *** Same 32 27 Better 54 69 Change in community leaders' responsiveness to community needsb Less 19 9 * Same 53 58 Listen more 28 32 Change in trust in individuals and organizations Same 66 59 Increase in 1 of 4 22 34 Increase in 2 and above of 4 12 7 Change in associational life Same 49 58 *** Increase in 1 of 2 16 25 Increase in 2 of 2 34 17 Change in participation in traditional events c Less 18 17 Same 73 76 More 9 7 Change in participation in political events d Less 13 2 Same 80 97 More 7 1 Change in circle of friends Less 4 3 * Same 51 44 More 44 54 1 2 8 APPENDIX H: RESULTS FROM THE MAP SELF-ADMINISTERED QUESTIONNAIRE Table N.4: Brazil: Bivariate Analysis of Variables Relevant To the Discussion on Empowerment and Social Capital (percent) (continued) FUMAC, 485 Comparator, 117 observations observations Express grievances if this risks losing project funds No 66 77 ** Yes 34 22 Express grievances if this risks compromising relations No 61 66 with other villagers Yes 39 33 If unhappy with community leaders, villagers call a meeting No 38 44 *** to discuss ite Yes 56 14 If unhappy with community leaders, villagers replace theme No 64 49 *** Yes 24 4 Participation at community meetings for subproject selection Unaware of meetings 38 Aware of meetings 62 Attended meetings 37 Attended and spoke at meetings 22 Note: Significance based on a test of proportion for binary variables and the Kruskall-Wallis test for categorical variables. * Significant at 10%; ** significant at 5%; ***significant at 1%. a. No. observations: project = 482; comparator = 115 b. No. observations: project = 452; comparator = 96 c. No. observations: project = 481; comparator = 117. d. No. observations: project = 474; comparator = 117 e. No. observations: project = 484; comparator = 111. Table N.5: Madhya Pradesh: Bivariate Analysis of Variables Relevant to the Discussion on Empowerment and Social Capital (percent) Project, 807 Comparator, 261 observations observations Change in access to information a Worse 16 21 *** Same 35 41 Better 49 38 Change in mobilization skills Same 35 40 * Improve in 1 of 3 31 34 Improve in 2 of 3 19 14 Improve in all 3 15 12 Change in ability to reach an agreement Worse 23 25 Same 29 31 Better 19 15 Change in community leaders' responsiveness to community needs b Less 17 16 Same 37 38 Listen more 20 27 Listen and respond more 19 15 Change in trust in individuals and organizations Same 62 67 * Increase in 1 of 5 15 12 (continued on following page) 1 2 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.5: Madhya Pradesh: Bivariate Analysis of Variables Relevant To the Discussion on Empowerment and Social Capital (percent) (continued) Project, 807 Comparator, 261 observations observations Increase in 2 of 5 8 9 Increase in 3 of 5 5 4 Increase in 4 of 5 5 3 Increase in all 5 6 4 Change in associational life Same 60 67 ** Increase in 1 of 2 15 13 Increase in 2 of 2 25 20 Change in participation in traditional eventsc Less 9 13 Same 65 63 More 26 23 Change in participation in non-traditional eventsd Less 11 8 Same 55 63 More 34 29 Change in circle of friendse Less 10 12 *** Same 47 54 More 43 34 Express grievances if this risks losing project funds No 48 46 Yes 43 47 Express grievances if this risks compromising relations No 40 34 * with other villagers Yes 52 59 If unhappy with community leaders, villagers call Disagree 14 17 a meeting to discuss itf Somewhat disagree 6 3 Somewhat agree 19 20 Agree 61 59 If unhappy with community leaders, villagers replace themg Disagree 23 26 Somewhat disagree 8 7 Somewhat agree 17 21 Agree 51 47 Participation in community organizations set up by the bank project Unaware of project organizations 8 Aware of project organizations 92 Member of project organizations 52 Member and attended 33 meetings regularly Note: Significance based on a test of proportion for binary variables and the Kruskall-Wallis test for categorical variables. * Significant at 10%; ** significant at 5%; ***significant at 1%. a. No. observations: project = 653; comparator = 218. b. No. observations: project = 581; comparator = 191. c. No. observations: project = 790; comparator = 260. d. No. observations: project = 787; comparator = 258. e. No. observations: project = 716; comparator = 249 f. No. observations: project = 688; comparator = 229. g. No. observations: project = 667; comparator = 227. 1 3 0 ANNEX N Table N.6: Uttar Pradesh: Bivariate Analysis of Variables Relevant to the Discussion on Empowerment and Social Capital (percent) Project, 757 Comparator, 400 observations observations Change in access to informationa Worse 8 8 *** Same 29 42 Better 63 50 Change in mobilization skills Same 22 33 *** Improve in 1 of 3 21 31 Improve in 2 of 3 27 21 Improve in all 3 30 16 Change in ability to reach an agreement Worse 24 46 *** Same 15 39 Better 56 8 Change in community leaders' responsiveness to community needs Less 26 Same 27 Listen more 23 Listen and respond more 22 Change in trust in individuals and organizations Same 33 51 *** Increase in 1 of 5 24 25 Increase in 2 of 5 17 15 Increase in 3 of 5 14 7 Increase in 4 of 5 8 3 Increase in all 5 4 0 Change in associational life Same 53 68 *** Increase in 1 of 2 13 11 Increase in 2 of 2 34 22 Change in participation in traditional events Less 12 20 *** Same 36 42 More 52 38 Change in participation in non-traditional Less 12 15 *** Same 40 52 More 47 34 Change in circle of friendsb Less 5 4 *** Same 32 42 More 63 54 Express grievances if this risks losing project funds No 45 36 *** Yes 54 64 Express grievances if this risks compromising relations No 33 30 with other villagers Yes 65 70 If unhappy with community leaders, villagers call a Disagree 11 11 * meeting to discuss itc Somewhat disagree 5 4 Somewhat agree 19 27 Agree 63 55 (continued on following page) 1 3 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.6: Uttar Pradesh: Bivariate Analysis of Variables Relevant to the Discussion on Empowerment and Social Capital (percent) (continued) Project, 757 Comparator, 400 observations observations If unhappy with community leaders, villagers replace themd Disagree 25 25 Somewhat disagree 7 7 Somewhat agree 21 28 Agree 42 37 Participation in community organizations set up by the bank project Unaware of project organizations 50 Aware of project organizations 50 Member of project organizations 16 Member and attended meetings regularly 13 Note: Significance based on a test of proportion for binary variables and the Kruskall-Wallis test for categorical variables. * Significant at 10%; ** significant at 5%; ***significant at 1%. a. No. observations: project = 686; comparator = 306. b. No. observations: project = 705; comparator = 386. c. No. observations: project = 728; comparator = 393. d. No. observations: project = 722; comparator = 393. Table N.7: Coefficients and Significance of Project Dummies in the Model without Interactive Terms: Change in Empowerment (Ordered probit) Change in Change in Change in Change in community leaders' access to mobilization ability to reach responsiveness to information skills agreement community needs (Coef.) (Coef.) (Coef.) (Coef.) Benin (PAMR) 0.19 0.25 0.73 *** 0.13 Brazil (PAC) ­0.04 ­0.44 ­0.53 *** ­0.42 * Brazil (FUMAC) ­0.40 *** ­0.11 ­0.50 *** ­0.38 * Brazil (FUMAC-P) ­0.37 ** ­0.42 *** ­0.83 *** ­0.65 ** Madhya Pradesh 0.27 ** 0.13 0.19 0.07 Uttar Pradesh 0.08 0.61 *** 0.93 *** -- Note: Weighted estimation (except for Benin) adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 2 ANNEX N Table N. 8: Coefficients and Significance of Project Dummies in the Model without Interactive Terms: Change in Social Capital (Ordered probit) Change in Change in Change in trust in Change in participation participation in Change in individuals & associational in traditional non-traditional/ circle of organizations life events political events friends (Coef.) (Coef.) (Coef.) (Coef.) (Coef.) Benin (PAMR) 0.35 *** 0.27 ** 0.17 0.08 0.35 *** Brazil (PAC) ­0.27 ­0.26 0.11 ­0.08 0.07 Brazil (FUMAC) ­0.09 0.27 * 0.13 ­0.10 ­0.27 *** Brazil (FUMAC-P) ­0.34 ** ­0.57 ** 0.01 ­0.32 ** ­0.45 *** Madhya Pradesh 0.36 *** 0.36 ** 0.34 ** 0.20 ** 0.26 *** Uttar Pradesh 0.55 *** 0.48 * 0.50 *** 0.25 ** 0.34 ** Note: Weighted estimation (except for Benin) adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. Table N.9: Benin: Change in Empowerment (Ordered probit) Change in Change in Change in Change in community leaders' access to mobilization ability to reach responsiveness to information skills agreement community needs (Coef.) (Coef.) (Coef.) (Coef.) Dummy for PAMR 0.19 0.16 0.73 *** 0.08 Dummy for AgeFIB 0.01 ­0.11 0.54 0.05 Dummy for PILSA ­0.46 * 0.37 ­0.19 ­0.11 Dummy for female in PAMR 0.01 0.26 ­0.01 0.16 Dummy for school construction subproject 0.32 *** 0.05 ­0.07 0.09 Dummy for training subproject 1.23 *** 0.07 1.43 *** 0.51 *** Dummy for difficult access to community ­0.05 ­0.05 ­0.16 ­0.07 Household size 0.02 0.01 * 0.00 0.02 *** Number of children ­0.03 ­0.01 0.01 ­0.01 Dummy for female ­0.22 ­0.36 ** ­0.15 ­0.14 * Schooling of the respondent ­0.07 0.09 ­0.11 0.01 Age 0.00 ­0.04 *** ­0.03 ­0.01 Age square 0.01 0.18 0.27 0.10 Number of leaders known 0.17 *** 0.11 ** 0.15 ** ­0.03 Participation in traditional events 0.21 *** ­0.03 0.13 * ­0.02 Participation in political events 0.00 0.05 ­0.02 0.12 *** Blue collar skills of respondent 0.13 ­0.05 ­0.10 0.10 Mobilization skills of the respondent ­0.11 *** 0.38 *** 0.01 ­0.02 Observations 1,028 1,028 1,028 1,028 Pseudo R-squared 0.08 0.12 0.09 0.02 Chi2 472.67 489.58 276.21 361.93 Note: Estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.10: Brazil: Change in Empowerment (Ordered probit) Change in Change in Change in Change in community leaders' access to mobilization ability to reach responsiveness to information skills agreement community needs (Coef.) (Coef.) (Coef.) (Coef.) Dummy for PAC ­0.25 ­0.44 ­0.62 *** ­0.44 * Dummy for FUMAC ­0.62 ** ­0.23 ­0.55 *** ­0.51 * Dummy for FUMACP ­0.62 ** ­0.51 ** ­0.95 *** ­0.69 ** Dummy for poor in PAC 0.43 ­0.09 0.17 ­0.10 Dummy for poor in FUMAC 0.11 0.25 ­0.04 0.11 Dummy for poor in FUMACP 0.31 0.20 0.21 ­0.27 Dummy for irrigation subproject ­0.05 0.04 ­0.05 ­0.39 Dummy for small bridge subproject 0.06 0.53 ** ­1.04 *** ­0.67 *** Dummy for Agreste region 0.08 ­0.02 ­0.34 ­0.43 *** Municipal Human Develop Index 1.35 ­2.44 ­6.33 ** 0.85 Score for community ­0.80 ­0.91 * ­0.65 ­0.31 Economic status 0.44 0.66 *** 0.37 0.07 Dummy for poor ­0.25 ­0.09 ­0.07 0.01 Household size 0.01 0.04 0.05 0.00 Number of children 0.01 ­0.04 ­0.04 0.01 Dummy for member of the CA 0.49 *** 0.13 0.19 0.26 ** Dummy for female 0.04 ­0.10 0.11 ­0.17 Schooling of the respondent 0.04 ­0.04 0.03 0.08 Dummy for agricultural laborer ­0.26 ** ­0.24 0.01 ­0.13 Age ­0.01 ­0.01 0.02 ­0.01 Age square 0.03 0.02 ­0.23 0.12 Participation in political events 0.10 ­0.22 0.11 ­0.02 Participation in traditional events 0.14 0.11 0.08 0.26 *** Mobilization skills of the respondent 0.01 ­0.09 ** 0.03 0.06 * Observations 916 925 915 771 Pseudo R-squared 0.05 0.03 0.06 0.07 Chi2 1,636.00 418.59 2,147.15 3,005.88 Note: Weighted estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 4 ANNEX N Table N.11: Madhya Pradesh: Change in Empowerment (Ordered probit) Change in Change in Change in Change in community leaders' access to mobilization ability to reach responsiveness to information skills agreement community needs (Coef.) (Coef.) (Coef.) (Coef.) Dummy for project village 0.22 0.01 0.21 0.05 Dummy for poor in project village 0.11 0.30 ** ­0.13 0.11 Dummy for Betul district 0.27 * 0.12 0.55 *** 0.38 ** Score for community ­0.25 ­0.48 ­1.42 *** ­1.20 *** Economic status 0.21 0.05 0.17 0.04 Dummy for poor ­0.08 ­0.22 * 0.00 ­0.09 Household size -0.03 0.02 0.01 ­0.02 Number of children 0.08 ** 0.00 ­0.01 0.06 Dummy for member of forest committee 0.28 *** 0.35 *** 0.06 ­0.02 Dummy for female ­0.14 ­0.11 * ­0.03 ­0.07 Age of respondents 0.01 0.00 0.04 * 0.00 Age square ­0.05 ­0.11 ­0.51 ** 0.07 Schooling of the respondent 0.34 *** 0.34 *** 0.18 0.10 Number of leaders known 0.00 0.04 0.00 0.03 Participation in traditional events 0.05 0.17 *** 0.31 *** 0.19 *** Participation in non-traditional events ­0.04 0.05 0.03 ­0.07 Mobilization skills of the respondent ­0.06 0.18 *** 0.02 ­0.08 Observations 859 1,046 756 712 Pseudo R-squared 0.05 0.07 0.10 0.05 Chi2 155.53 490.72 216.07 289.32 Note: Weighted estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.12: Uttar Pradesh: Change in Empowerment (Ordered probit) Change in Change in Change in Change in community leaders' access to mobilization ability to reach responsiveness to information skills agreement community needsa (Coef.) (Coef.) (Coef.) (Coef.) Dummy for project village ­0.09 0.34 ** 0.80 *** -- Dummy for poor in project village ­0.28 * 0.23 0.42 *** ­0.17 Dummy for female in project village 0.37 0.34 *** ­0.05 ­0.04 Dummy for Amawa district 0.25 0.08 0.14 0.49 *** Dummy for Maharajganj district ­0.17 ­0.29 ­0.25 0.22 *** Dummy for rural ­0.52 ** ­0.16 ­0.25 ­0.34 *** Score for community ­0.25 0.06 0.74 * ­0.17 Economic status 0.17 ­0.25 ­0.75 * 0.19 Dummy for poor ­0.11 ­0.48 ** ­0.66 ** -- Household size 0.02 0.01 0.00 0.01 Number of children ­0.08 ­0.05 ** ­0.02 ­0.04 Dummy for member of SIC 0.25 0.25 ** 0.32 ** 0.30 Dummy for female ­0.20 ­0.41 *** ­0.24 ** -- Age of the respondent ­0.03 ­0.01 0.02 ­0.07 *** Age square 0.20 0.02 ­0.29 0.71 *** Schooling of respondent 0.29 ** 0.37 *** 0.00 ­0.01 Number of leaders known 0.19 *** 0.35 *** 0.07 0.19 *** Participation in traditional events 0.07 ­0.05 0.03 -- Participation in non-traditional events 0.17 ** 0.24 *** ­0.03 -- Blue collar skills of respondent ­0.02 0.18 0.40 *** -- Mobilization skills of the respondent 0.14 0.18 *** 0.12 ­0.04 Observations 986 1,148 1,082 747 Pseudo R-squared 0.09 0.10 0.13 0.03 Chi2 757.04 2,022.38 2,407.09 408.07 Note: Weighted estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. a. This question was only asked in project villages. Three variables were dropped from this regression because in order to be estimated adjusting for cluster effects, the model requires the number of clusters to be greater than the number of constraints. The variables dropped were found not significant in the model estimated without adjusting for cluster effects. 1 3 6 ANNEX N Table N.13: Benin: Change in Social Capital (Ordered probit) Change in Change in trust in Change in participation Change in Change in individuals & associational in traditional participation in circle of organizations life events political events friends (Coef.) (Coef.) (Coef.) (Coef.) (Coef.) Dummy for PAMR 0.33 *** 0.29 ** 0.12 0.01 0.34 *** Dummy for AgeFIB 0.28 ** 0.05 0.07 ­-0.04 ­0.14 Dummy for PILSA 0.30 *** 0.32 * ­0.08 0.03 ­0.06 Dummy for female in PAMR 0.07 ­0.06 0.13 0.23 0.03 Dummy for school construction subproject 0.43 *** 1.32 *** ­0.12 ­0.01 0.19 Dummy for training subproject 0.16 0.03 ­0.01 ­0.36 *** 0.45 Dummy for difficult access to community ­0.10 ­0.25 ­0.09 ­0.18 * ­0.34 ** Household size 0.02 *** 0.00 ­0.02 ** ­0.01 0.01 Number of children ­0.02 ­0.01 0.04 *** 0.02 0.00 Dummy for female ­0.16 ­0.17 ­0.01 ­0.11 ­0.13 Schooling of the respondent ­0.08 0.03 ­0.01 0.11 0.08 Age ­0.04 ** ­0.01 0.00 0.01 ­0.01 Age square 0.34 ** 0.10 0.00 ­0.18 0.11 Number of leaders known 0.13 * 0.20 *** 0.04 0.08 0.14 ** Participation in traditional events ­0.05 0.11 ** 0.06 ­0.03 0.14 *** Participation in political events 0.11 ** ­0.01 0.10 * 0.39 *** ­0.06 Blue collar skills of respondent 0.06 0.15 ­0.25 ** ­0.35 *** 0.02 Mobilization skills of the respondent 0.05 0.02 ­0.05 ­0.03 ­0.01 Observations 1,028 1,028 1,028 1,026 1,028 Pseudo R-squared 0.03 0.15 0.02 0.08 0.04 Chi2 151.03 629.10 491.85 303.81 768.10 Note: Estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.14: Brazil: Change in Social Capital (Ordered probit) Change in Change in trust in Change in participation Change in Change in individuals & associational in traditional participation in circle of organizations life events political events friends (Coef.) (Coef.) (Coef.) (Coef.) (Coef.) Dummy for PAC ­0.21 ­0.23 0.07 ­0.17 0.20 Dummy for FUMAC ­0.10 0.24 0.15 ­0.14 ­0.37 *** Dummy for FUMACP ­0.28 ­0.70 ** ­0.05 ­0.39 ** ­0.61 *** Dummy for poor in PAC ­0.23 ­0.35 0.18 0.28 ­0.49 ** Dummy for poor in FUMAC 0.03 ­0.37 0.02 0.11 0.14 Dummy for poor in FUMACP ­0.22 0.01 0.28 0.23 0.38 * Dummy for irrigation subproject 0.01 0.08 ­0.25 ­0.36 ­0.25 Dummy for small bridge subproject 0.35 ­0.69 *** ­1.19 *** ­0.54 ­0.81 *** Dummy for Agreste region 0.00 ­0.55 ** ­0.21 ­0.04 ­0.23 Municipal Human Develop Index ­2.37 ­8.18 ** ­4.90 ** ­1.73 ­4.65 Score for community ­0.68 0.17 0.22 0.11 ­0.90 * Economic status 0.33 0.55 ** ­0.47 ** 0.05 0.42 Dummy for poor 0.11 0.34 ­0.31 ­0.17 ­0.10 Household size 0.05 * 0.05 0.05 * 0.02 0.02 Number of children ­0.04 ­0.07 ­0.09 ** ­0.02 ­0.03 Dummy for member of the CA 0.00 0.38 ** ­0.06 0.02 0.17 Dummy for female ­0.13 0.02 ­0.11 ­0.16 0.08 Schooling of the respondent ­0.05 * 0.01 0.08 * 0.05 ­0.07 ** Dummy for agricultural laborer ­0.43 ** 0.16 0.00 ­0.10 ­0.02 Age ­0.01 ­0.01 ­0.01 ­0.02 0.01 Age square ­0.04 0.00 0.02 0.20 ­0.10 Participation in political events ­0.22 0.04 0.00 0.00 0.12 Participation in traditional events 0.12 0.17 0.20 *** 0.17 *** 0.20 *** Mobilization skills of the respondent 0.03 0.00 0.11 *** 0.06 ** 0.06 ** Observations 925 925 919 909 917 Pseudo R­squared 0.03 0.07 0.05 0.03 0.05 Chi2 1,043.50 986.26 1,945.22 283.91 874.42 Note: Weighted estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 8 ANNEX N Table N.15: Madhya Pradesh: Change in Social Capital (Ordered probit) Change in Change in Change in trust in Change in participation participation in Change in individuals & associational in traditional non-traditional circle of organizations life events events friends (Coef.) (Coef.) (Coef.) (Coef.) (Coef.) Dummy for project village 0.25 * 0.20 0.41 *** 0.11 0.20 * Dummy for poor in project village 0.33 * 0.54 ** ­0.25 0.22 0.11 Dummy for Betul district 0.13 0.18 ­0.01 0.26 ** ­0.09 Score for community ­0.83 ** ­0.50 * 0.33 0.05 0.21 Economic status 0.17 ­0.03 0.37 * ­0.13 ­0.27 * Dummy for poor ­0.38 * ­0.50 0.31 ­0.22 ­0.37 ** Household size ­0.02 ­0.01 0.00 ­0.02 0.01 Number of children 0.03 0.01 0.05 ** 0.04 * 0.03 Dummy for member of forest committee 0.33 *** 0.29 *** 0.13 0.30 *** 0.22 ** Dummy for female ­0.18 ** ­0.21 * ­0.15 * ­0.20 ** ­0.11 Age of respondents 0.00 ­0.01 0.07 ** 0.04 ­0.02 Age square ­0.06 0.12 ­0.76 *** ­0.43 * 0.18 Schooling of the respondent 0.45 *** 0.26 0.00 0.19 0.17 Number of leaders known 0.00 ­0.02 ­0.03 ­0.03 0.03 Participation in traditional events 0.00 0.11 * ­0.04 0.00 0.03 Participation in non-traditional events 0.02 0.03 ­0.02 0.00 0.05 Mobilization skills of the respondent 0.05 0.12 *** 0.00 ­0.14 ** ­0.11 ** Observations 1,046 1,046 1,045 1,042 956 Pseudo R-squared 0.06 0.06 0.03 0.04 0.04 Chi2 721.66 113.39 80.62 100.59 327.13 Note: Weighted estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 3 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Table N.16: Uttar Pradesh: Change in Social Capital (Ordered probit) Change in Change in Change in trust in Change in participation participation in Change in individuals & associational in traditional non-traditional circle of organizations life events events friends (Coef.) (Coef.) (Coef.) (Coef.) (Coef.) Dummy for project village 0.44 *** 0.18 0.33 ** 0.09 0.15 Dummy for poor in project village 0.19 0.26 ­0.10 0.05 0.22 Dummy for female in project village ­0.01 0.31 * 0.31 0.17 0.10 Dummy for Amawa district 0.24 ** 0.15 0.09 ­0.12 ­0.09 Dummy for Maharajganj district ­0.13 ­0.31 * 0.04 ­0.24 ­0.09 Dummy for rural ­0.09 ­0.08 0.14 ­0.09 0.15 Score for community ­0.29 0.65 1.09 *** 0.10 0.31 Economic status ­0.11 ­0.10 ­0.13 ­0.25 0.17 Dummy for poor ­0.24 ­0.31 ** ­0.06 ­0.30 * ­0.28 *** Household size 0.01 0.00 ­0.01 0.01 0.02 Number of children ­0.02 ­0.05 * ­0.01 ­0.04 ­0.04 Dummy for member of SIC 0.37 ** 0.45 *** 0.25 0.35 *** 0.61 *** Dummy for female ­0.05 ­0.31 * 0.08 0.10 0.08 Age of the respondent ­0.02 ­0.02 ­0.03 ­0.01 ­0.02 Age square 0.26 0.18 0.27 0.11 0.13 Schooling of respondent 0.44 *** 0.30 *** 0.04 0.03 0.03 Number of leaders known 0.18 *** 0.14 0.14 0.11 0.21 *** Participation in traditional events 0.04 ­0.13 ** 0.04 0.02 ­0.03 Participation in non­traditional events 0.01 ­0.08 0.10 0.16 0.14 *** Blue collar skills of respondent 0.13 ** ­0.09 0.01 0.07 0.28 *** Mobilization skills of the respondent 0.07 0.12 ­0.03 ­0.03 0.08 Observations 1,148 1,148 1,148 1,144 1,083 Pseudo R-squared 0.06 0.07 0.03 0.03 0.06 Chi2 1,120.56 218.88 194.52 511.74 2,091.91 Note: Weighted estimation adjusted for cluster effects. * Significant at 10%; ** significant at 5%; ***significant at 1%. 1 4 0 ANNEX O: POVERTY TARGETING Poverty Targeting Mechanisms through sustainable increases in productivity Bank projects use a variety of methods to target and better food security. Clearly, project objec- project beneficiaries. The most popular method tives required geographic targeting to identify is geographic targeting. A project can focus on marginal areas, social targeting to focus on small- a poor region or province, usually using gov- holders, and self-targeting to focus on subpro- ernment figures/criteria, or at a more local level jects that define basic needs of the poor (food such as a municipality or community, often using security). The project used a combination of all the results of participatory planning processes. three poverty-targeting mechanisms. Over time, Another mechanism that has been increasingly Bank-supported CBD/CDD projects are em- used, especially in social funds, is self-targeting. ploying increasingly sophisticated mechanisms In this method the project supports basic infra- to target beneficiaries.1 structure and services that are likely to be among the priority needs of poor communities or house- Poverty Targeting for CBD/CDD Projects holds, such as basic schools, health posts, water Covered by the Fieldwork pumps, and similar infrastructure. A third mech- The evaluation studied in depth four targeted anism becoming popular with CDD projects is CBD/CDD projects, one each in Benin and Brazil, social targeting, whereby the project targets par- and two in India. One or more targeting mech- ticular social groups (women, handicapped, dis- anisms were adopted in all of them. advantaged, small and marginal farmers, herders, and others). · The Borgou Pilot Project in Benin applied ge- The approach to poverty targeting for any ographic targeting in 250 villages using a par- project is largely based on the project objec- ticipatory diagnosis. The eligibility criteria for tives, availability of data, and institutional con- beneficiaries included the definition of clear siderations. For example, the development priorities, as established by the Comité Villa- objective of the Natural Resources Management geois de Concertation, and the capacity of the and Poverty Reduction Project for Armenia (ap- community to contribute financially to cer- proved in fiscal 2002) was to adopt sustainable tain types of investments. Specific emphasis natural resource management practices and to was to be placed on women's groups and Fu- alleviate rural poverty in mountainous areas lani herders.2 where degradation is now reaching a critical · The RPAP project in Brazil's Rio Grande do point. Provincial-level geographic targeting alone Norte was to apply three targeting methods: (a) was sufficient given the nature of the project: the geographic by poverty level and other charac- project selected two marzes in the poorest moun- teristics of the municipality; (b) geographic tainous regions of Armenia. In contrast, the de- within municipalities to target rural settle- velopment objective of the Rural Development ments and communities; and (c) community- in Marginal Areas Project in Mexico (fiscal 1998) based selection of poor beneficiaries and was to improve the well-being and the income particularly vulnerable groups by the project of smallholders in about 24 targeted marginal Municipal Councils. The communities them- areas--among the poorest of the country-- selves, through their majority participation in 1 4 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT the Councils, were to determine where proj- geted the poorest and most disadvantaged. In ect resources would best be applied. Brazil, although the project was a targeted in- · In the Forestry Project in Madhya Pradesh, tervention, 136 of the 166 municipalities in the India, the target group consisted of tribal peo- state were covered. The justification for the vast ples and forest fringe villagers. The group was coverage was that all of the rural areas of the state to be to be identified by the government for- were deemed sufficiently poor to warrant in- est department. The project incorporated spe- clusion in the project. Only the state capital and cific measures to safeguard the interests of the its surrounding area were considered ineligible. landless and women. Using a Municipal Human Development Index · The Sodic Land Reclamation project in Uttar (MHDI), the evaluation attempted to assess Pradesh, India, was to target small and marginal whether municipalities with lower MHDI re- farmers or previously landless allottees in the ceived more Bank funds compared to munici- sodic land area identified by the government palities with higher MHDI. There was no relation agency. Women were to be targeted and sup- between the level of MHDI and per capita in- ported as a special group. vestment in the municipality as a whole. In other words, there appears to be no concerted effort Disaggregated data were not available for to target greater resources (subprojects) to most projects (except for Brazil) to allow com- poorer municipalities. ment on the success with which the project tar- 1 4 2 ANNEX P: EVIDENCE FROM FIELDWORK ON SUSTAINABILITY This annex draws on the household surveys and ernment responsibility. These interventions have the qualitative data gathered in four project areas put a resource burden on the communities, in the context of this evaluation to explore issues which they may not have anticipated and may relevant to sustainability of subprojects funded find difficult to sustain. by Bank interventions. There is also the issue of the quality of educa- tion services imparted, which requires coordina- Benin tion with the education department in terms of Both the PAMR and AgeFIB projects provided adherence to a centrally planned curriculum, support for construction of small infrastructure, among other things. In the absence of certified a very large percentage of which was primary schoolteachers, it is not clear how education in schools, in hundreds of communities. Typically, these schools will conform to a national standard. the government pays the salaries of certified Poor communities may not be able to pay adequate schoolteachers for village primary schools. How- salaries and benefits to attract qualified teachers. ever, the fieldwork in the Borgou region reveals that the government has not been in a position Brazil to provide for paid certified teachers in the nu- In Brazil, project communities benefited from merous schools that have been constructed three types of investments: water supply (20 under both the projects. Interviews with village communities), irrigation (three communities), leaders revealed that over 50 percent of PAMR and small bridges (two communities); while the schools and 80 percent of AgeFIB schoolteach- three comparator communities benefited from ers were community teachers, in comparison a government water pipeline. The comparison with comparator villages, where only a third of between project and comparator communities the teachers were community teachers. Because is restricted for the analysis on sustainability to of a shortage of government teachers, the com- those project communities that received water munities have been forced to hire teachers and supply investments. pay their salaries from their own resources. The fieldwork was undertaken shortly after the proj- Water supply investments: The RPAP financed ects closed, and new infrastructure generally a variety of water supply systems, including wells, does not entail significant maintenance costs. It cisterns, and small dams. While the majority of is not clear whether poor communities will be these systems are community-based, and hence able to bear the cost burden of maintenance require the community to collectively organize and teachers' salaries from their own resources for its O&M, three communities benefited from over the long run. The expectation among the household-based water supply systems, such as communities is that the government will be re- household water tanks and boxes, whose O&M sponsible for the salary portion of the provision falls solely on the individual household. These of education services. The majority of the house- three project communities were dropped from hold survey respondents considered repairs to the comparative analysis between the project be a responsibility of parent-teacher associa- and the comparator group. As figure P.1 shows, tions, but considered staff salaries a central gov- a larger share of respondents in comparator 1 4 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure P.1: User-Fee Payments and Satisfaction Rates with Water Services in Project and Comparator Communities in Rio Grande do Norte Project [N = 560] Comparator [N = 117] 100 95% *** 89% *** 89% *** 80 (%) 60 59% 59% 57% Respondents 40 20 0 Pay user fees Satisfied w/services Satisfied w/O&M Note: Significance level based on test of proportion. * Significant at 10%; ** significant at 5%; ***significant at 1%. communities than in project communities paid Irrigation investments: Fewer than half of user fees, and were satisfied with the services pro- the respondents who are members of the three vided and with O&M of their water system. How- community associations (CA) that benefited from ever, these aggregated figures hide pronounced irrigation investments pay user fees.1 Only 37 per- differences among project communities. As the cent of them rated the services provided as qualitative data reveal, while almost half of them good, while they hold divergent opinions on collect monthly fees to cover the cost of the O&M, with 42 percent rating it as poor and an- electricity, and in some cases maintenance of other 42 percent rating it as good. These ag- the equipment, a few have yet to set up adequate gregate figures, however, hide pronounced O&M systems, and others have transferred the differences among the three CAs. In one of them, system to the state water company, which is now the subproject is paralyzed by the high cost of responsible for O&M. The quantitative data re- electricity to operate the pump, while another flect this great variation among project com- is temporarily suspended by a shortage of water. munities. The percentage of respondents that pay A large share of the CA members interviewed user fees varied widely, from 100 percent in four where the irrigation system is functional and communities to zero percent in six communities. where it has been temporarily suspended rated Similarly, satisfaction with the services provided the service provided and O&M as good. While the and with O&M varied, respectively, between 8 majority in the former pay user fees, half in the and 94 percent and 5 and 100 percent. Con- latter do so. versely, variation between comparator commu- Small bridges: Fifty-seven percent of the re- nities is very limited, and all display similarly spondents in the two communities that benefited high level user-fee payment and satisfaction with from the construction of a small bridge are sat- the services provided and with O&M. isfied with the service provided by the infra- 1 4 4 ANNEX P Table P.1: Declining Forest Cover hold data indicate that Bank JFM villages seem (percent) to have experienced less of a decline in forest cover than comparator communities (table P.1), Forest Less than Same as More than cover before before before this gain has not been sufficient to provide ad- Bank-JFM 35 10 22 equate returns to the communities. At the time Govt.-JFM 62 14 16 of the survey only a small percentage of the ben- eficiaries from Bank JFM communities reported Non-JFM 90 3 1 collection of forest products, a large percent- age reported having less access to forest prod- structure, while 52 percent rated O&M as poor. ucts than previously, and a very small percentage None of the respondents pays any kind of fees reported collection of the forest products for in- for the upkeep of the small bridges. As the qual- come (table P.2). Focus group sessions reveal itative data reveal, the choice of these invest- increasing hardship and lack of income-gener- ments, which did not result from a process of ating activities in the Bank JFM villages. Further, broad community participation, is not perceived communities that have received support from the as a solution to one of their main priorities. Bank appear to be receiving less support from the government for other development activities. Madhya Pradesh Nearly two-thirds of the Bank beneficiaries report The World Bank project aimed to assist imple- no micro-project in their village, compared with mentation of the government strategy for de- a third of government beneficiaries. velopment of the forest sector using joint forest management (JFM). The Bank project also pro- Uttar Pradesh vided improved information, extension services, The World Bank project aimed to remove sodic- and complementary investments in communal ity of land. The project also provided gypsum and infrastructure to give villagers an incentive to co- other inputs, helped construct boring for irriga- operate. The appraisal document identified the tion, set up formal extension services, and intro- need for the Bank to support the forest sector duced loans for cropping. Maintenance of drains, in the state for about 10 years with investments especially the main drains, is critical to the sus- that could total more than US$200 million. Ac- tainability of reclaimed lands. Most respondents tual support provided was about a quarter of this from the Bank project communities character- amount over a period of four years. The Bank did ized maintenance of the drains as "bad," and said not follow through with a second intervention that O&M of the drains had deteriorated over in the state. time (figures P.2 and P.3). Focus group sessions re- It appears that this is the main reason why it iterated these findings, emphasizing dysfunctional has not been possible to build the elements of main drains. An interesting fact revealed by the sustainability in this short time. While house- open-ended discussion was that most farmers no Table P.2: Forest Product Use Percent reporting collection of forest products Percent reporting decline in collection Forest product Bank Government Bank Government Fuelwood 65 94 47 52 Fodder 16 22 15 13 Tendu 47 54 39 34 Mahua 36 43 29 26 Amla 8 15 13 16 Grass 6 11 10 15 1 4 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Figure P.2: O&M for Drains Is Bad Figure P.3: O&M for Drains Deteriorating Field Field drains drains of Link of Link peyT peyT Main Main 0 25 50 75 100 0 25 50 75 100 Percentage of respondents [N = 757] Percentage of respondents [N = 757] Good Fair Bad Do not know Improved Same Deteriorated Do not know longer have field drains. Farmers have used that of O&M of main drains lies with the Irrigation De- part of the land for cropping and are using irri- partment, and that responsibility of O&M of link gation channels for drainage purposes. drains lies with the village organization (table P.3). The Irrigation Department is responsible for The project constructed many borings, and maintaining the main drains, the village organi- most respondents agree that O&M of that bor- zation is responsible for link drains using inter- ing is the responsibility of the pump owner. How- nal funds, and farmers are responsible for field ever, most water pump owners indicated that drains. The OED assessment notes that the im- the money received from farmers for water rights plementing agency believed that political pres- was insufficient to maintain the pump. At the sure from farmers would ensure that the same time, a majority of the respondents who pay government provides sufficient resources to the the pump owner for water for their fields be- Irrigation Department for this activity. However, lieve they pay enough to cover the O&M of the most beneficiaries are not even aware of the pump. Eight of the 18 village leader interviews in- critical importance of drainage for containing dicated that boreholes have not helped improve sodicity, or that it is the responsibility of the Ir- irrigation; and the other 10 raised concerns about rigation Department to maintain the drains. Over the continued flow of service from the borings 80 percent of the respondents thought that con- because of erratic water supply, prohibitive costs tinuous application of gypsum and water supply of diesel, temperament of the pump owners, for irrigation will prevent the land from becom- and deteriorating conditions of some boreholes. ing sodic. Further, fewer than 4 percent of the The project committee (site implementation respondents were aware that the responsibility committee) setup is functional in only one of the project villages, according to interviews with proj- ect committee members in each of the project vil- Table P.3: Many Unaware of Who Is Responsible lages. Focus group sessions revealed that most for Drains (percent) respondents were also unaware of the existence UPBSN/ Bene- of any MK/MMK. Irrigation Village ficiaries/ Do not Department leaders villagers know Main drains 15 20 14 36 Link drains 9 17 28 32 Field drains 5 9 51 24 1 4 6 ANNEX Q: SAFEGUARD THEMATIC STUDY: A SUMMARY The Safeguards Policy Review is one of two subprojects. Such subprojects may be multi- thematic studies conducted for the OED evalu- sectoral or may be limited to a single sector, ation of the World Bank's support for commu- such as health or education. nity-based and community-driven development. · Other Projects (CBD/CDD-NS): Projects The study reviewed project appraisal, supervi- that have CBD/CDD aspects or components but sion, and completion documents for a sample of do not fit the definition above. 84 projects to assess their compliance with the Bank's safeguard policies.1 The desk reviews Quality at Entry were supplemented by interviews with task team The study found that the EA category was cor- leaders, the Quality Assurance and Compliance rectly assigned for 80 percent of the sample proj- Unit team, and Regional safeguards coordinators ects but, given the nature and extent of potential in selected cases. Selected items of direct rele- impacts, it was judged that 9 percent of Category vance from the literature on safeguard policies Bs should have been As and 38 percent of Cs and CBD/CDD projects were also reviewed. should have been Bs (see box 5.1 in Chapter 5 Detailed findings on each project were con- for definitions of these categories). The quality densed into a set of ratings on quality of com- of appraisal was rated moderately satisfactory pliance and analyzed with respect to: and above for 70 percent of the total sample, with environmental assessment (EA) category, sec- newer projects scoring higher. The quality of tor, Region, project type, and age. Findings and EA documents was mixed: only two of the five recommendations were developed from this A projects and 74 percent of B projects were analysis. Examples of best practice and missed rated moderately satisfactory or above. The spe- opportunities were also identified. A special re- cial requirements for IDA B projects with a sep- view was made of the 10 CBD/CDD projects in arate EA report were generally observed. The Benin as part of a country study. Finally, 473 quality of Resettlement Action Plans and In- headquarters and field staff were sent a ques- digenous Peoples Development Plans was gen- tionnaire that included questions on safeguard erally high. The number of cases where the issues to assess Bank performance in the area of potential applicability of one or more of the safe- CBD/CDD projects and how Bank capacity to un- guards policies should have been discussed but dertake CBD/CDD interventions has evolved.2 was not was high--about equal to the cases Because of the broad definition of CBD/CDD where policies were triggered. Compliance with used, which includes some projects with only the public disclosure and consultation require- minor CBD/CDD aspects, the sample projects are ments of the safeguard policies was good for heterogeneous. Therefore, the projects were di- resettlement and indigenous peoples issues, but vided into two broad groups: less so for EAs. In contrast, provisions for capacity building were well developed, with monitoring · CBD/CDD with subprojects (CBD/CDD-S) somewhat less so. (76 percent of the sample): CBD/CDD proj- Overall, quality at entry was rated moderately ects for which the majority of investment fund- satisfactory and above for 70 percent of the sam- ing is for a large number of small and scattered ple. The small group of FI (Financial Intermedi- 1 4 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT ary) projects were rated much better than aver- 35 percent of cases, with A projects at 40 percent. age, while As were distinctly worse than average. While newer projects score much better than Newer projects are markedly better than the older older ones, at 44 percent moderately satisfactory group--81 percent versus 54 percent moderately and above, they are still far from meeting Bank satisfactory and above. CBD/CDD-S projects also standards. As at appraisal, CBD/CDD-S projects score better than CBD/CDD-NS--77 percent ver- are distinctly better than CBD/CDD-NS (38 per- sus 50 percent moderately satisfactory and above. cent versus 25 percent). In contrast to the qual- Adjustable Program Loans (APLs) scored some- ity at entry ratings, East Asia and the Pacific and what better than conventional projects. In terms South Asia score highest for quality of supervi- of Regions, Africa, Europe and Central Asia, and sion of safeguard issues, with Latin America and the Middle East and North Africa have the high- the Caribbean and the Middle East and North est percentages of moderately satisfactory and Africa scoring lowest. Sectorally, water supply above, while Latin America and the Caribbean and sanitation and transport had the best record, and South Asia have the lowest. Among sectors, with energy, mining & private sector, social, and transport, social, and environment had the best education scoring lowest. results, while the ratings for the rural and urban sectors were well below average. Overall Project Quality The study found that internal guidance on the When the ratings for entry and implementation use of the FI category and, consequently, the are combined, the overall proportion of proj- practices of the Regions have not been entirely ects moderately satisfactory and above is 70 per- consistent since that category was introduced in cent, the same outcome as for quality at entry. January 1999. Discussion about the assignment However, this disguises the fact that the proj- of the FI category for most CBD/CDD projects ects rated (fully) satisfactory and above slip from is ongoing, but full guidance has yet to be issued. 52 percent at entry to 17 percent when imple- mentation is considered, and those rated unsat- Quality during Implementation isfactory and highly unsatisfactory climb from 20 Despite format changes in the Project Status Re- percent to 33 percent. Regardless of statistical port (PSR) that encourage detailed reporting quirks, the result of 70 percent falls well below on the implementation of safeguard measures, Bank expectations. Between EA categories, the such reporting remains sparse and inadequate. result for the small FI sample is 100 percent mod- This is true especially for Category A projects, erately satisfactory and above, while Bs and Cs are which should receive particular scrutiny during close to the average. The main concern is the very implementation, and FIs, where the real work of low percentage of A projects (40 percent) that is screening subprojects and designing mitigation being handled well. Nevertheless, newer proj- measures falls into the project implementation ects are closer to compliance than the older phase. There was no specialist follow up for group--87 percent to 45 percent. While 87 per- cases where the dam safety and pest manage- cent is an encouraging result, it includes 65 per- ment policies were triggered. There was almost cent in the moderately satisfactory category, no reporting on capacity building or monitoring indicating considerable room for improvement. systems. Most of the Implementation Comple- The Europe and Central Asia and Middle East tion Reports (ICRs) were also less than satisfac- and North Africa Regions achieved the best results tory on reporting safeguard compliance, with overall, with the other Regions close to each the majority containing no discussion at all. Of other at a lower level. Among the sectors, trans- the four Project Performance Assessment Re- port, social, and environment scored highest, ports (PPARs) available for this sample, two pro- and urban lowest. vided good analysis of safeguard issues, while the The review also found that difficulties exist in other two said nothing.3 applying the safeguard policies to multicompo- The overall quality of implementation was nent projects and that the potential for cumu- rated moderately satisfactory and above for only lative impacts from large numbers of small 1 4 8 ANNEX Q subprojects is sometimes overlooked. The im- practice" examples, of which this Review has portance of adequate collection and disposal of identified some. medical waste was not recognized in some ear- · A thematic study of the environmental and so- lier health projects, but recent practice has im- cial implications of changes in land use may be proved. warranted. The review found that 6 of the 11 projects · In any planned revision of Operational Policy rated unsatisfactory on overall quality were in the 4.01 (and/or the other safeguard policies), spe- Bank's largest borrower countries. cial attention should be given inter alia to: The survey of Bank staff showed that only a streamlining the IDA B with separate EA re- quarter of respondents agreed that resources port procedures; defining "financial interme- for addressing safeguard issues were sufficient, diary"; dealing with multicomponent A projects; though about half felt that current policies were defining standards for supervision and com- relevant for CBD/CDD projects. pletion reporting on safeguards compliance; and a possible mandated role for the Regional Conclusions environmental and social units in the supervi- Four broad themes emerge from the analysis: sion of A projects. · The experience of the Poland: Rural Develop- · Although there has been clear improvement, ment Project should be thoroughly reviewed safeguard compliance in CBD/CDD projects for examples of the issues that may arise from does not yet fully meet Bank standards. the use of country systems for safeguards com- · While quality at entry needs improvement, pliance. safeguards compliance during implementa- tion warrants much greater attention by the At the level of Regional safeguards compli- Bank and borrowers, and may indicate the ance assurance: need for greater allocation of supervision re- sources. · Evaluating the recent transfer of sign-off · Gaps in the compliance system may be lead- authority for Category B and FI projects in ing to significant environmental and social im- light of the above findings and those of other pacts, which may not be caught by the OED studies, including staffing and budgeting monitoring and reporting systems typically issues. used. · Ensuring full compliance at entry with safe- · The Bank appears to have particular difficulty guard policies, especially in the Bank's largest in ensuring safeguard compliance in its largest borrower countries. borrowers. · Ensuring that policies other than Operational Policy 4.01 are triggered in appropriate cases Based on its findings and conclusions, the and necessary follow up actions taken. Review makes the following recommendations. · Obtaining resources for and carrying out spe- cial reviews of safeguard compliance for At the level of policy development, Regional CBD/CDD projects under supervision, with coordination, staff guidance, and training: special attention to the adequacy of agreed provisions, the effectiveness of their imple- · Guidance is urgently needed on the appro- mentation, and the success of capacity build- priate EA categorization of CBD/CDD projects, ing and monitoring activities. especially on the use of the FI category and on · Developing standard document packages the special requirements for IDA B projects (cf. procurement documents) for safeguard with a separate EA report. instruments such as EAs, Environmental · Training of task teams in the application of the Management Plans, Resettlement Action safeguard policies to CBD/CDD projects should Plans, and Indigenous Peoples Development be intensified and should rely heavily on "best Plans. 1 4 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT · Reviewing the potential for delegation of safe- consultation requirements of Categories A and guard management authority to national agen- B (and agreeing with management on com- cies. monsense waivers where process requirements may impede project quality or timeliness). At the level of project development, · Obtaining sufficient financial and staff re- approval and supervision: sources to allow adequate supervision of the implementation of agreed safeguard meas- · Identification of potential safeguard issues, for ures, especially for As and Bs with an EMP, example, by use of Strategic Environmental including periodic review of a sample of sub- Assessment. projects. · Mainstreaming environmental and social safe- · Using the comment boxes in the PSR form to guards into the preparation process for explain the reasoning behind the ratings given, CBD/CDD projects, for example, in developing, the progress of capacity building or monitor- planning, programming, and monitoring pro- ing programs, and any unforeseen problems grams, as well as staff training. encountered, with special attention to Category · Collaborating closely with the Regional envi- A projects. ronmental and social units in assigning · Using the Midterm Review to look in greater EA categories appropriately and in using depth at safeguard compliance, with the as- the Integrated Safeguards Data Sheet as a sistance of environmental and/or social spe- "contract" for actions needed between the cialists. project concept development and appraisal · Following the guidelines for the ICR in re- stages. porting on safeguard compliance at project · Being sensitive to the special disclosure and completion. 1 5 0 ANNEX R: ADVISORY COMMITTEE COMMENTS Robert Chambers 1. Picking winners. This is not a criticism of the The OED team deserves congratulations on the selection of the large sample of projects where effort put into the monumental task of this eval- I agree with the points about this made by Nor- uation. The subject matter is vast, scattered and man Uphoff. There are two other points. difficult to assess. The documents made available to the Advisory Committee and our discussions First, success with "indigenously matured or- in December 2004 made it clear that the re- ganisations" is the result of no doubt rational search was carried out with conventional rigour cherry-picking by the Bank. These were out- and care, and that the conclusions are credible, standing organisations with exceptionally high based on and emerging from careful and bal- calibre, continuity and commitment of man- anced analysis of the evidence. The extensive and agement with charismatic and inspiring leaders valuable literature review also drew on and col- and which had existed for a decade or more and lated much other relevant experience. Other aid already successfully gone to very large scale: for agencies would do well to conduct evaluations example, AKRSP Pakistan, the NDDB in India, and similar in their independence, breadth and depth SEWA in Gujarat. They were highly successful be- as those of the OED. The conclusions, as far as fore the Bank became involved. They are cor- they go, resonate with and are confirmed by my rectly distinguished as a separate category from own experience. The recommendations, how- other CBD/CDD. Their performance is irrele- ever, fall short of what the evidence implies. vant to the evaluation of other CBD/CDD proj- I note that CBD/CDD approaches with ects, which are by far the majority. In earlier HIV/AIDS are not included and are the subject stages of their development they did not need, of a separate evaluation. Given the delicacy and and might have been hampered by, support from complexity of AIDS-related issues, and my par- the World Bank. ticipation in a Bank-led workshop on CDD and HIV/AIDS in Africa, I expect its findings to be even Second, the Matrouh Project in Egypt is world- more negative than those presented in this pres- renowned as perhaps the most famous Bank ent evaluation. flagship participatory project. It is several times There is much in the report which merits en- mentioned. We know that it benefited from ex- dorsement, presenting aspects which are both ceptional continuity of exceptional staff and positive and negative. Rather than list such high-level support from James Wolfensohn. But points, let me highlight four issues which qual- even it, one of the most favoured and best ex- ify the conclusions of the evaluation. I do not amples that could be found, is noted in this eval- make these comments with any pleasure, or uation for its serious downsides, including the lightly, but given what I have experienced, and effects of creating a parallel organisation and given the Bank's commitment to professionalism overlooking changes in land tenure which in the service of poor people, I have to make harmed the Bedouin. If one of the very best them. All four suggest that this evaluation is cherries has such flaws, one may wonder about over-favourable: the rest. 1 5 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 2. Positive bias. Any evaluation of Bank projects show success. Where government staff and re- involving interviews with Government staff and current funds are, as so often, limiting, this will NGO beneficiaries of Bank funding is vulnerable deprive poor communities that are less ac- to positive biases. The power and prestige of cessible. the Bank, the careful respect with which it is · Risks and costs associated with top-down time- treated, and the tendency to try to please with bound disbursement-driven capital projects. favourable feedback, present systemic difficulties These include scope for petty and not so petty in knowing what is really happening. All power corruption, and the proliferation of oppor- deceives (see Chapter 5 of my book Whose Re- tunistic NGOs as noted on page 35. ality Counts?). However careful the research, · The long-term disempowering effects of de- there will always be questions about prudent, def- pendence and disillusion created at the com- erential and self-serving responses. It is as much munity level (see, e.g., box 4.2). Communities as some officials' jobs are worth to say anything become, as so many have, less self-reliant and negative about the Bank. more inclined to lobby, beg, and wait 3. Hidden negative externalities. The fol- These are general tendencies. There will be lowing negative effects are either not mentioned exceptions. But together on balance they mean or understated. The fact that they are half-hid- that the findings of the evaluation should be den to conventional research does not make more negative. Given the goodwill, energy and them any less plausible or less real. Some of commitment of many Bank staff, I regret having them are part of another and more inclusive re- to say this. But it is quite possible that, overall, search agenda. the Bank's CBD/CDD initiatives do more harm than good to poor communities and people. In · Diversion of progressive NGOs (both INGOs addition there are the opportunity costs of al- and NNGOs) from rights-based and empow- ternatives foregone. ering activities which would do more for poor people than the provision of infrastructure 4. The comparative disadvantages of the which does less, and/or may even be negative, Bank. While the conclusions of the report fol- and/or may drag the NGOs back into activities low from the evidence and analysis, they do not they were attempting to move on from. (36 per adequately confront the comparative disadvan- cent of projects had some form of NGO in- tage of the Bank with CBD/CDD, nor the full volvement). In pro-poor terms, this is likely to range of what would be required if performance reduce NGO additionality, leaving poor people were to justify continuing to try to support it. This net losers. is alarming, especially when CBD/CDD-related · Undermining other more participatory, less lending, far from prudently diminishing, is in- target- and disbursement-driven, less infra- creasing beyond its already remarkably high structure-focused, and more sustainable pro- level. grammes supported by other organisations in The comparative disadvantages are institu- neighbouring areas. ("Why should we do it tional and paradigmatic and related to: ourselves when our neighbours are getting so much done for them or for free?") · Reliance on loans, the future repayment of · Diversion of government recurrent funds, which may impact adversely on government staff and materials from other places and ser- services and so on poor people. If loan-based vices (schools, clinics etc) to the new infra- rather than grant-based projects are to be jus- structure, with hidden costs to services in tified, repayments will be at the cost of other those other places. With schools and clinics, government expenditure. The bar, therefore, for example, resources are most likely to be has to be higher. diverted to communities which are accessible · Disbursement pressures and the typical one- to government and Bank staff inspection, to year sub-project cycle (pp. 20­21) with top- 1 5 2 ANNEX R down one-shot interventions, an approach an- The Recommendations tithetical to participation and to assuring ben- The recommendations in this final version of efits to those who are poorer. We know and do the evaluation fall far short of what is demanded not need to learn again how badly this works. by the evidence. If the Bank is serious about · The management and staff-intensity of em- poverty and empowerment, more radical action powering and participatory development. The is required. In effect, the recommendations as extra cost of preparation of CBD/CDD projects they stand leave the door open to going on with is only 10 percent higher than non-CBD/CDD. more of much the same. The evidence of this For effective pro-poor participation, it would OED evaluation, combined with other studies need to be far higher than this. and insights, shows the CBD/CDD initiatives of · The staff incentive system of the Bank which the Bank to be of such questionable value that rewards high and fast disbursements. This the approach now should be damage limitation, was a major factor which emerged from a par- intensive learning and finding out whether ticipatory workshop for task managers which change is possible in the Bank. I hesitate to say I facilitated a few years ago. Nothing I have what I believe the Bank should do, but the stakes heard suggests that this has changed signifi- for poor people and communities are so high and cantly. on such a scale that it would be wrong for me not · Inability to learn and change. That the ratings to do so. I have agonised over this. And I recog- of CBD/CDD projects are stagnating suggests nise that there is no way I can assess fully the im- that institutional learning and change are not plications or modalities for what follows. But on taking place. A likely reason is that the Bank is the basis of the evidence of this evaluation and not looking hard enough at itself or is simply of other experience, my own best judgement is unable to perceive, learn, and change. that it the Bank should now, and decisively: These factors combine to disable the Bank, · Rein back on and/or slow down existing making it inherently difficult for it to do well CBD/CDD projects, where this is legally and with CBD/CDD. The question then is whether ethically feasible. they can be changed. · Impose a moratorium on new ones. The report correctly points to the need for · Learn more about what happens and what radical institutional change in government bu- might be made to happen by selecting on- reaucracies: "The literature shows that the in- going projects for intensive learning through stitutionalization of a CBD/CDD approach action research, including investigating hid- requires a radical reorientation in the way gov- den externalities. ernments and bureaucracies operate." This ap- · As part of this examine the Bank itself--its cul- plies if anything more to the Bank itself, as the ture, procedures, norms, incentives and be- dominant partner, than to governments and haviours--and its impact on governments, their bureaucracies. This is not rocket science. NGOs and communities, and analyse the con- It is common sense and common experience. In tradictions between these and empowering, practice, the disabling culture, incentives, pro- pro-poor community development and how cedures and imperatives of the Bank are passed these play out. on "downwards" to governments and NGOs. It · And then review how the Bank must change is no good saying "Do as I say but not as I do." if the short and long-term effects for poor peo- The Bank may not be able to become more par- ple, communities and countries of Bank-driven ticipatory. But unless it does, it cannot expect the CBD/CDD are to be positive and to justify the CBD/CDD it funds to be cost-effective in em- costs, and whether and how such change could powering and benefiting poor people. This be achieved. means that the Bank itself must walk the talk, and take on board "physician heal thyself" and "do Robert Chambers no harm." 5 September 2005 1 5 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Norman Uphoff implementation, did not perform much better, This evaluation is originally planned to evaluate and sometimes less well, than conventional proj- World Bank support for community-driven ects with similar objectives and environments. development (CDD) projects. However, when This could be because there is nothing inherently reviewing earlier drafts of this report, this and the superior in CDD-type project, or what were other reviewers saw some problem with char- called CDD projects were not planned and car- acterizing these projects "community-driven." ried out in ways that gave them any advantage. This terminology had been was introduced From my experience of over 30 years working on within the Bank to distinguish newer, more par- participatory development, I would think the ticipatory project initiatives from what were latter explanation is more apt, and indeed, the being called "community-based" development evaluation team identified many ways in which projects. The term "CDD" seemed rather the way the Bank operates "got in the way" of car- grandiose to the reviewers for what was actually rying out the projects included in this study so happening in these projects. They were hardly that they did not perform noticeably better. Pos- "community-driven" when the project design, sibly if they had been carried out properly, they what kinds of things could be done, within what might still have shown no performance advan- time frame, on what financial terms, were all de- tage. But we can't know that until the CDD con- cided unilaterally by Bank staff. The projects cept has at least been properly introduced. Can themselves were not open to local participatory CDD deliver more benefits to the poor? More inputs, only subprojects. The main decisions cheaply? More reliably? We don't know and can't left to communities were whether or not they say because it hasn't really been properly tried. would make a proposal to gain access to Bank The overall hypothesis guiding this evaluation project funds for something they wanted to do was, essentially, that CDD projects (as supported (within the non-negotiable framework set by by the Bank) would produce more and better Bank or government personnel) and how they outcomes. The evidence assembled and assessed would carry out the work once it was approved. does not support the hypothesis; so we are left This issue of how these projects should be de- with the null hypothesis. However, as someone scribed does not make the evaluation that has who has worked on participatory development, been done less relevant or meritorious, since the several times as a consultant for the Bank on this purpose of the assessment was to learn more subject, I must say that I am not surprised, be- about the Bank's approach to this kind of de- cause I have seen the Bank's efforts, however velopment, no matter what it is called. well-intended, to be continually well behind the The projects have in common an aspiration "state-of-the-art." Although there are some ways (and enabling provisions) for delegating to com- in which some of the CDD projects have given munities (or their representatives) responsibil- superior results, there is no basis for concluding ity for taking initiative to plan and implement that on average, the Bank would get better re- certain improvements in infrastructure and/or sults by doing more of its projects in what it has services at community level. But "community- been considered and created as a CDD mode. driven" means only community-initiated, -im- This said, the converse is also true: there plemented and -managed, within externally-set would be no significant degradation of the Bank's parameters. The first point in the Executive Sum- portfolio by expanding CDD efforts even as cur- mary makes clear that only participation in "sub- rently supported by the Bank. The increased projects" is covered by the study. There is nothing cost associated with CDD project design and wrong with this, and it can be preferable to more sometimes in implementation can probably be conventional top-down efforts for local devel- justified by some subsequent cost reductions opment, depending on results. to the government as communities take more fi- The evaluation of results found that what nancial responsibility, or by more rapid imple- were called CDD projects, meeting certain cri- mentation once the planning has been teria for a community role in activity initiation and completed, or from better post-project utilization 1 5 4 ANNEX R and maintenance of project services and facili- decision to analyze internally heterogeneous ties, not all the time, but sometimes. Such a samples. But more "less rigorous" analysis could conclusion is based on the evidence provided in have revealed some operationally useful insights, the evaluation. It does not take into account the not being "shackled" by statistical analysis con- various external costs that Robert Chambers de- ventions. lineates very persuasively in his comments on the evaluation. Such broader considerations that go Ontological assumptions. In this analysis, as beyond the terms of reference and the frame- in most such evaluations these days, there is a work of this evaluation could justify his sugges- certain "reductionism" that obscures more than tion that there be a moratorium on CDD efforts it reveals. There is an implicit assumption that until the Bank figures out how to pursue them there is an "essence" of CDD that is the same in more effectively. all cases that are characterized as CDD, and that There is little to find fault with in the way this quality is rather equally represented in all that the evaluation itself has been done, or in the such cases. Then there is a concomitant as- way that it is reported. The evaluators have used sumption that this "essence" of CDD, whatever both quantitative methods and qualitative de- it is, has independent, rather than contingent, scription and summarization quite satisfactorily. causal effects. Such assumptions are common There is not much to critique on methodologi- throughout development studies, and indeed cal grounds. However, this does not make the in most social science analyses, which try to evaluation itself irreproachable, because the draw broad generalizations (about often ethereal state-of-the-art followed has some shortcomings characteristics) rather than stick to more disag- that should be noted. The following comments gregated, concretely specified assessments. This refer to the way in which formal evaluations are kind of reliance on abstractions is at the root of done these days by institutions like OED, rather many of the failures in development efforts, and than to the way that this particular evaluation was it is one reason why so much of our social sci- done. ence is so irrelevant to real-world decision mak- ing and action. This comment is not a critique Sample size and statistical significance. of this evaluation, but rather of the broader en- One area where the evaluation could have been terprise. more informative is to have disaggregated the The Bank's CDD approach has sought to bring CDD sample more in terms of the kinds of de- more participation and more flexibility into World velopment work being fostered. This was not Bank projects. This is commendable and moves done because that would have reduced sub- the Bank toward what can be thought of as the sample sizes and impaired the scope and valid- "state-of-the-art." This evaluation documents ity of statistical analysis. However, by lumping that the Bank's procedures, staff incentives and diverse experiences together in one pool, or orientation, borrower-country predispositions making only a few gross disaggregations, the va- and capabilities, and still other factors have kept lidity and value of the generalizations are di- "CDD" projects from achieving the degree of luted for the sake of being able to use statistical community assumption of responsibility that methods and offer assessments of (statistical) sig- was anticipated and hoped for, and the amount nificance. If the analysis had been done on of improvement in the lives of poor people that smaller but more homogeneous subsamples, was expected. What is evaluated in this report is, what could be said with statistical confidence at most, "partial CDD," and some of the case would have been reduced--but we would know studies suggest this was even "minimal CDD." more about whether CDD approaches (as the When local people say that the project did not Bank has been supporting them) would be more address their priority needs (see 3.9) or they productive and sustainable in some sectors than are not willing to maintain the facilities or ser- in others, in some Regions, within certain time vices beyond life-of-project (4.30), it does not ap- periods, etc. The OED team made a defensible pear that this is an evaluation of truly 1 5 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT "community-driven" processes. Such processes prove prospects for better CDD outcomes. My are directed to meeting priority needs, and if own experience with a USAID project introduc- what is created is of value to people, they will find ing participatory irrigation management in Sri ways to maintain the facilities or services some- Lanka in the early 1980s has demonstrated that how, anyhow. there can be significant measurable benefits An earlier draft of the report described the from truly community-driven development. The Bank's Adaptable Program Loans (APLs) and incremental rice production possible in a single Learning and Innovation Loans (LILs) as "allow- season, the 1997 dry season, alone covered 4­5 ing (!) communities greater choice in the selec- times the total cost of the participatory compo- tion of activities" and as "providing them nent, for example (see article by Uphoff and Wi- opportunities to control investment decisions jayaratna in World Development, November, and resources during project implementation." 2000). And such investments in social infra- This underscores the extent to which the process structure and social capital can be sustainable; is in Bank or borrower government hands. These the community organizations established in 1981- are community-proposed (not community-se- 85 are still functioning well 25 years after end of lected) projects, and they are community-man- project, and they became the model for a national aged (within often complicated and burdensome program improving management of the whole terms).Few people would understand this to be sector. (On the strategy and implementation of "community-driven development." this effort, see Uphoff, Learning from Gal Oya, The concept of "community" that under- Intermediate Technology Publications, 1996.) lies this effort is also questionable. Endnote 1 to The Bank has approached the introduction of Chapter 1 in the report says: in a World Bank proj- CDD in a rather self-referential manner, trying to ect, a community is considered a "unified, organic learn mostly from its own experience, which is whole." This is dubious thinking, suggesting that a limited "slice' of what is known and what has the Bank has learned little from the hundreds of been done, rather than look beyond its institu- social science assessments of "community" writ- tional boundaries. ten over the past half century. These have One of the most interesting and relevant find- stressed the pitfalls of internal divisions, con- ings of the evaluation concerns the value of rec- flicting interests, etc. among any given set of ognizing and working with informal households that is delineated on the basis of organizations (pp. 39-40, and the very in- residential area. Even when there are ethnic or structive Annex K), not just with formal organi- other kinds of homogeneity, village residents zations. We saw this clearly in the Sri Lanka often find personalistic or other bases on which experience mentioned above. One of the "hy- to factionalize. Some communities will have a potheses" we tested and confirmed was that it high degree of solidarity and significant poten- would be most effective to begin with informal tial for collective action. But this is a variable organization, creating a demand for formal rather than a defining characteristic. (The Bank's organization rather than begin with supply own efforts to bring some rigor to the concept and then try to create a local demand for it, the of "social capital" have help to clarify this mat- usual approach. Our strategy was: work first, or- ter, because this concept does not take village sol- ganize second. More could be said and done idarity for granted, but rather examines different along these lines, but this last paragraph "flags" levels, kinds, and activation.) this issue as one where the Bank could usefully Assuming that the Bank is serious about focus some attention. poverty reduction and empowerment, two of the most attractive justifications for its existence Norman Uphoff and for its support by member governments, Cornell University there is much more that could be done to im- 1 5 6 ANNEX R Paiboon Wattanasiritham therefore, may be more or less effective de- The Revised report has taken into account pending, to a significant extent, on the nature of the Advisory Panel's concern about the the programs' interrelationships with the many World Bank's use of the term "Community relevant factors just mentioned. Driven Development" (CDD) and has made Utilizing a "systems thinking" approach changes which make the report read bet- should be useful in undertaking such an ter on the whole. evaluation as well as in drawing conclu- In particular, the reference to the participa- sions and making recommendations. tion of communities in development as either A country or society--comprising communi- "community-based development" (CBD) or ties, institutions ,organizations, etc.--is a highly "community-driven development"(CDD) ap- complex and dynamic "system." World Bank pro- pears more appropriate and more easily under- grams, therefore, are but a small "part" or "com- stood. ponent" of a very big, complex "whole", especially Realizing the many difficulties in eval- when considering that "sustainable poverty re- uating the effectiveness of World Bank sup- duction" is the ultimate objective of those World port for community development, the Bank programs. In this context of "systems think- evaluation attempt has done well in gath- ing", it should be added, CBD/CDD is more than ering information from several angles, "poverty reduction", and "sustainable poverty making logical analyses and interpreta- reduction" is more than CBD/CDD. Further- tions, and finally coming up with reason- more, the differentiation and comparison be- able and credible conclusions and tween CBD/CDD and non-CBD/CDD programs recommendations. cannot be too clear-cut since both, as well as The difficulties in evaluating CBD/CDD ef- the many other relevant factors, are very much fectiveness in the context of World Bank pro- inter-related and intertwined in a complex and grams stem partly, as pointed out in the report, dynamic manner. Indeed, CBD/CDD and non- from the fact that the Bank has not , until recently, CBD/CDD programs can, and perhaps should systematically identified and tracked its portfo- also, be complementary. Although they may be lio of CBD/CDD projects and therefore has lacked different in nature and may aim at different out- a comprehensive understanding of the evolution puts and even outcomes, the ultimate objec- and scope of its work on community develop- tive, or final impact, should be the same, that is, ment. Further, the Bank has not been sufficiently sustainable development which includes poverty clear about the objectives of using CBD/CDD reduction and improvement in people's quality approaches, criteria for choosing between dif- of living. ferent community development approaches, or The purpose of evaluation is (or should how to measure the results. But evaluating be) "to learn and improve." This OED eval- CBD/CDD effectiveness in World Bank programs uation should be a useful point of departure, is also difficult because, for CBD/CDD, and es- or point of reference, for both learning and pecially for CDD, to be effective , a complex set improvement efforts, both of which are mu- of factors have to be at work. Government poli- tually reinforcing. cies and programs, national and local adminis- Much "learning" can be gained from such trative structures, relevant laws and regulations, processes as "knowledge management" (KM) attitudes and capacities of officials who can have which in turn may be organized under one or impact on CBD/CDD effectiveness, the eco- more, or all, of the following contexts : nomic, social , cultural and human capacity set- tings of communities in the country, all · KM within a World Bank program contribute to how easy or difficult it is for · KM among World Bank programs CBD/CDD programs to be effective, as well as · KM within a country how long it would take for the cumulative effects · KM among countries to be realized. World Bank projects or programs, · KM within the World Bank 1 5 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT · KM among a number of organizations includ- agenda of the country, not just the World Bank ing the World Bank program. In addition, adequate time will be · KM under any other contexts or combinations. needed for the many relevant parts and factors to be in place and become rooted firmly enough For the World Bank in particular, appropriate to make the empowerment or CDD sustainable. knowledge management, or some other learn- In this context, therefore, it may be useful for the ing systems, should be useful for executives and World Bank to be clear from the beginning about staffs, not only in understanding and appreciat- the purpose, nature, scope, etc., of its program ing the true essence of CBD/CDD, but also in in a country, particularly whether the program being able to come up with innovations and / or aims at CDD as the main thrust or only a sup- creative developments that will be beneficial for plementary feature, and so on. all concerned, including the World Bank itself. The concept of CBD/CDD, especially in As a "development partner" of the its the form of CDD, logically points to the member countries (particularly developing principle of "holistic country-driven devel- countries), the World Bank may find it con- opment " (HCDD), about which an in-depth structive and beneficial, both to the mem- study as well as a knowledge management ber countries and to the Bank , to have an process should be undertaken, which could assessment study undertaken as to the rel- lead to a significant change in the way World evant situations and factors that have bear- Bank programs, especially country pro- ing on each country's potentials for higher grams, are conceptualized, designed, and degrees of community empowerment, in- managed. cluding community-driven development. A number of possible outcomes of such a Such a study would be particularly relevant in study (and / or knowledge management process) the case where the World Bank is to have a de- can be envisaged. For example, more emphasis velopment lending program in a country, be it a may be given to "holistic country-driven devel- CBD/CDD-related programs or a non-CBD/CDD opment" programs as opposed to ordinary pro- program. The outcome of the study should then grams or the so-called "country assistance" be useful is helping shape the concept and the programs. In such a "holistic country-driven design of the program in such a way that it is con- development" (HCDD) program, "community- ducive to the improvement in community em- driven development" (CDD) should automati- powerment or community-driven development cally be a significant part, or even a crucial part, (CDD) efforts. For this purpose, it should be of the total package. borne in mind also that true and effective com- munity empowerment or CDD need to involve Paiboon Wattanasiritham a comprehensive, integrated development 2 September 2005 1 5 8 ANNEX S: MANAGEMENT RESPONSE Introduction denced by a growing demand. The OED review Management welcomes a review of the effec- talks about "Bank projects" and the "Bank- tiveness of Bank support for community-driven subproject cycle" and contrasts "Bank proj- development (CDD)1 and projects that include ects" with government programs, not noting community participation (covering CBD by that CDD operations are all government pro- OED's definition).2 Management is encouraged grams. The Bank's role is to provide support. by the evidence that CDD and CBD operations b. The analysis and findings related to safeguards have in aggregate, higher development outcome and fiduciary aspects of CDD that lead OED to ratings than non-CBD and non-CDD operations recommend more guidance and oversight. and have proven to be an effective tool for client Data from OED's own reviews of compliance countries. Management notes that this review re- with safeguard policies have never highlighted inforces findings from previous OED reports in- CDD or "CBD/CDD" operations as having more cluding the OED Review of Social Development3 problems than other types of projects. On the that community participation contributes to contrary several other types of projects were overall project success and sustainability. Given highlighted. Data from the Quality Assurance the demand from client governments, these Group (QAG) indicate that CDD operations do types of operations are expected to remain an im- better in assuring compliance with Bank poli- portant component of the Bank's assistance to cies than the average for all other operations, client countries. both during preparation and implementation. This is not surprising, given the extensive train- Areas of Agreement. Management concurs ing programs, thematic reviews, and opera- with the OED Review that sustainability, moni- tional guidance already available to staff. toring and evaluation, and local leadership are c. The suggestion that the Bank should do more issues of particular importance to the Bank's up-front analysis and move more slowly on work on community development. Accordingly, "CDD/CBD" operations; while great care is im- for some time management has been devoting portant in working with client countries on increasing resources to address these issues to the preparation of all operations, Management improve the quality of its support for community does not see evidence of a need to slow down development interventions. These issues are its processing of CDD operations, given their discussed in more detail below. better quality at entry and better outcomes that the average for all other projects. Areas of Divergence. Management, however, has questions regarding the relevance, rigor, This remaining divergence likely results from and clarity of some aspects of the OED Review. the scope and methodology of the OED review. In particular, management would like to note: Management tracks CDD operations and oper- ations that involve participation broadly, in- a. The potential for misunderstanding the Bank's cluding community participation. For its review, role in CDD operations and borrowing coun- OED created a set of operations as discussed in tries' ownership of the CDD agenda, as evi- Annex E of the report. The methodology in- 1 5 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT cludes text searches and OED staff judgment. agement has been directing attention to these is- That set of operations differs significantly from sues in CDD operations. The impetus for the es- either of the sets that Management tracks. Much tablishment of a Bank-wide CDD Steering Group of the evidence OED presents for "CDD" con- and a corporate Anchor unit in FY01 was to draw cerns operations that do not meet the definition on global experiences in order to enhance the of CDD used broadly, including by Management.4 quality and effectiveness of community-driven Further, Management has questions concerning development operations. Client Governments the methodology used, including the data have embraced CDD as a valuable approach to de- sources, fieldwork, and surveys (for more details liver public services to poor people, to strengthen see Annex I). the capacity of people's organizations to partic- ipate in development, and to build the assets The Approach Paper. As outlined in the Ap- and capabilities of poor women and men so as proach Paper,5 the original focus of the review to improve their well-being. Given the heightened was on CDD. It stated that the objective of the interest of client governments, management has review was to "assess the relevance, efficacy, ef- invested significantly in ensuring that ongoing and ficiency, institutional development impact, and new programs benefit from global lessons learned sustainability of the World Bank's CDD inter- through improved guidance and technical sup- ventions" (para. 6).6 While the Approach Paper port. clearly stated that the evaluation would also look at other forms of community participation, the Key Area of Agreement: Sustainability three primary evaluation questions and the six The review attests to overall improvements in sub-questions included in the approach paper fo- sustainability of the "CBD/CDD" portfolio, al- cused on CDD.7 In management's view, a review beit with scope for further improvement (para. with a focus on CDD would have been particu- 4.27). When CDD operations and programs are larly useful, given the growth in the CDD port- carefully designed and well implemented, with folio in recent years and the increased corporate clear exit strategies, the likelihood of sustain- attention to CDD over the past five years.8 Man- ability increases. OED's Social Development re- agement believes that a review of the now sub- view found that "giving the participants the stantial set of closed CDD projects (56 operations responsibility for structuring their involvement at the end of fiscal year 2005) would have been in the project increases the likelihood of success more appropriate. When invited by OED to com- and sustainability."9 The Social Development ment early in the review process, management evaluation emphasized the value of quality par- expressed its concerns with how the Approach ticipation in the success of Bank-supported op- Paper defined the scope of OED's inquiry and erations and noted that "best practice" projects with the proposed methodology. In retrospect, were likely to have attributes such as involvement management should have shared those con- of beneficiaries in project design and imple- cerns officially in writing with OED and with ex- mentation and securing of community contri- ecutive directors. butions to the project and its future maintenance (Annex D). All are attributes found in most CDD Management Comments operations. A combination of top-down and bot- The management comments that follow refer to tom-up measures is critical for sustainability of the main issues of agreement and provide an ex- any type of project. These typically include line panded set of comments on the evolution of ministry budgetary resources and inputs, ap- Bank community-driven development opera- propriate technical standards, community-led tions. As noted above, the OED review highlights maintenance and local government involve- three areas that, while important for all opera- ment.10 tions, are particularly pertinent to effective CDD operations: sustainability, monitoring and evalu- Community-Level Sustainability. At the com- ation, and local leadership. For some time man- munity level, sustainability efforts and project exit 1 6 0 ANNEX S strategies revolve around dynamics of linkages cessful. Given that borrowing governments in- between communities and external support in- creasingly rely on CDD approaches to address stitutions, most notably local governments. De- community empowerment objectives, Manage- sign characteristics of CDD operations place ment has for some time been strengthening considerable emphasis on embedding commu- monitoring and evaluation of the CDD portfolio. nity initiatives in permanent institutional frame- works, including local government systems. Guidance on M&E. Consistent with OED rec- Evidence from the field suggests that CDD ini- ommendations to promote learning by doing, tiatives with these design characteristics are Management has been encouraging clients im- more likely to be sustainable; for example, plementing CDD programs to test out different schools built through a CDD approach have approaches, to build robust systems to monitor more and better teachers than schools built implementation for rapid and operationally rel- without (Zambia) and child mortality declined evant feedback, and to emulate best practice. The more in areas that have involved communities operational manuals that guide client imple- than in areas that have not (Bolivia).11 mentation give practical guidance for setting up Management Information Systems to track results Aid Dependency and Sustainability. Despite of decentralized local initiatives. Innovative ben- this evidence of trends showing increasing sus- eficiary assessment approaches collect real time tainability, the OED review mentions lower than feedback from those directly involved in com- average sustainability ratings for "CBD/CDD" munity initiatives, so that information about suc- operations. This rating covers a number of issues cesses and challenges can be quickly relayed to beyond sustainability of the service and infra- implementing agencies. structure created. Specifically, for several oper- ations rated as non-sustainable, OED noted in Evaluating the Impact of CDD Approaches. their rationale for the rating the issue of whether As OED notes, while real-time information flow the implementing agency will be able to continue is crucial to success, it is also important to eval- absent donor funding. While these issues of aid uate the impact of CDD approaches. As part of dependency are valid and important for all donor its efforts to improve impact evaluation across assistance, they are different from the issue of the Bank portfolio, management has placed par- sustainability of outputs and outcomes at the ticular emphasis on CDD. Given the institutional community level. development objectives and decentralized im- plementation of CDD approaches, it is particu- Constant Attention to Sustainability Issues. larly challenging to evaluate CDD impact. Overall, what is required is constant attention Nonetheless more than 50 impact evaluations during design and implementation to sustain- have been carried out for 36 CDD operations (see ability of community impacts. To this end, Man- Annex II).12 In addition several evaluations have agement has augmented its internal quality been done that use the most sophisticated tech- control functions and scaled up its technical niques available, that are more robust than those guidance through project Quality Enhancement used in the past by essentially any development Reviews, CDD quality clinics and peer reviewing. evaluators, including OED. They use randomized treatment and control sampling and differences Key Area of Agreement: Monitoring in difference techniques (for example, the ran- and Evaluation domized studies of the Bolivia Social Fund and Management agrees with OED that monitoring the Indonesia Kecamatan Development Pro- and evaluation (M&E) is of utmost importance. gram). Several additional sophisticated evalua- Management's approach to monitoring results tions are currently underway. To measure the emphasizes the identification and tracking of institutional effects of CDD operations, Man- the impact of Bank-financed operations and agement has developed and supported extensive learning from approaches that are most suc- quantitative and qualitative techniques for un- 1 6 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT derstanding changes in social capital, such as development, promote inclusive decision-mak- those applied recently in the Philippines, In- ing and collective action and strive for the type donesia, and Thailand. of institutional change that underlies participa- tory local governance. These programs are ex- Cost-Benefit Analysis. In addition to impact tensively engaged in dealing with the challenge evaluation studies, Management also agrees with of how to most effectively work with local lead- OED that it is important to do cost-benefit analy- ership. The degree to which they have succeeded sis of CDD operations--where feasible and at rea- is reflected in part by assessments of percep- sonable cost. Besides provisional ex-ante tions of choice and relevance of investments. cost-benefit analysis (such as that carried out in Evidence suggests that most benefits accrue to the Philippines), many operations undertake poor people; reviews of beneficiary assessments some ex-post cost-benefit analyses; some ex- from eight countries "were uniform in their find- amples include Bank support to the Cambodia ing that beneficiaries consistently felt that...proj- SEILA program and the Indonesia Kecamatan ects reflected priority needs of the community."14 Development Project. Operational manuals for The quality of program design, including the CDD operations contain guidance on how to role of community facilitators and local trans- carry out economic analysis. As would be ex- parency and accountability measures, have a pected when communities are able identify and major bearing on a program's impact on com- support operations that are their highest prior- munity inclusion and community empowerment. ities, the internal rates of return for such oper- ations are high. For example, in the Indonesia Evolution of CDD Programs case, two independent cost-benefit analyses The CDD approach builds on a rich history of in- point to internal rates of returns in excess of 60 novations in participatory development, pre- percent for rural roads built using CDD ap- dominantly accumulated from outside the Bank. proaches. That said, ex-ante cost-benefit analy- The composite of CDD programs also reflect sis of social sector operations is rare (and not lessons from earlier efforts by the Bank in sup- required by Bank procedures), given the intrin- port of community-based development (CBD) sic difficulties, and social sector activities sup- initiatives of client countries. There are several ported by CDD operations are no exception. CDD operating principles that are markedly dif- OED is correct in noting that for the majority of ferent from CBD program mechanisms; these interventions in client "CBD/CDD" operations were conceived directly in response to perceived supported by the Bank there was no ex-ante shortcomings in earlier CBD efforts. Of particu- cost-benefit analysis of the project itself. OED fails lar relevance are current CDD design features for to note, however, that individual sub-projects institutional arrangements, the role of local gov- were subjected to economic analysis.13 ernment, and broader governance linkages. Key Area of Agreement: Local Leadership The CDD Portfolio Today. The current CDD Management agrees with OED on the impor- portfolio comprises iterative and evolving pro- tance of supporting client countries in working grams that build on lessons from across the port- effectively with local leadership. Communities are folio--from prior operations in-country, similar not homogenous. As management has learned operations in comparable environments (for ex- from extensive experience in support of com- ample in post-conflict countries), and opera- munity initiatives, it is critical to ensure buy-in tions that address related technical and from diverse social groups within a given com- operational issues, such as decentralization re- munity. The role of elites needs to be carefully form and public-private partnerships. The CDD considered, but practice suggests that they can portfolio includes a host of long-standing, trans- act benevolently and play a positive leadership formative CDD programs which have evolved role, as the OED review points out (para. 3.19, over time. In these programs in particular, the footnote 27). CDD programs, through capacity areas of sustainability, local leadership challenges, 1 6 2 ANNEX S and M&E have been of critical interest. Man- other sources of finance has been key to ensur- agement will continue to put resources into ing sustainability of investments. learning in these areas and continue to exercise vigilance in enhancing operational guidance to Fiduciary Aspects. The APDPIP project has in- staff. Some examples of client operations sup- vested in the development of adequate fiduciary ported by the Bank that have contributed to and auditing systems to ensure accountability and and benefited from lessons learned include the transparency in a program of this scale, includ- following. ing building up auditing and financial manage- ment capacity among 800 federations of self-help Andhra Pradesh (AP) CDD Operations, India groups. These federations have developed ca- The AP District Poverty Initiatives Project pacity to train village-level bookkeepers in fi- (APDPIP) and AP Rural Poverty Reduction Pro- nancial management and provide auditing ject are two statewide community-driven rural services. The project has initiated the develop- poverty reduction projects under implementa- ment of a rating system jointly with financial in- tion in Southern India with World Bank support stitutions to provide performance data on village since the year 2000. These programs build on the groups to banks and agencies investing in com- AP Government's cumulative investments in munities. The project commissions independent women's self-help groups over the last 10 years. agencies to undertake process monitoring to The APDPIP supports the development of self- give feedback on process, institutional and em- managed grass-roots level institutions of poor powerment aspects. rural women and their federations; it has sup- ported half a million groups and 800 federations Zambia Social Investment Fund (ZAMSIF) in 29,000 villages covering 80 percent of all rural This fund was conceived as a two-phase pro- poor households (6.4 million) over last five years. gram to support, over a ten-year period, two The key project investments include institution main strategic objectives of the Government of building, capacity building and local leadership Zambia (a) decentralization and empowering development of community organizations, an local authorities to improve governance and ef- investment risk fund in key livelihood sectors and ficiency in service delivery; and (b) increasing ac- livelihood support services for the poor. The cess to basic social services through direct project uses a learning-by-doing approach and poverty interventions. It followed on two social employs process monitoring and other M&E recovery projects (SRP) designed in the late mechanisms to develop mid-course corrections 1980s, and is a classic example of a CDD program and make adjustments in project design and that has evolved over time, based on country con- procedures. The current design features reflect text and lessons learned. The earlier SRP projects this evolution. worked directly with communities (and decon- centrated technical staff of central ministries) Scaled-Up Investments in the Poor. An out- to deliver quick impacts and open up space for come of institutional development and the anticipated macro-level reforms. ZAMSIF also empowerment and skill enhancement of com- strove to support the implementation of critical munities has been the willingness of public and aspects of the decentralization process and in- private sector agencies, including commercial troduced a process for strengthening local gov- banks, to scale up their investments in the poor. ernment capacity and performance. Annual credit to poor households and household groups has increased twelve-fold from $23 mil- Learning and Design Evolution during Im- lion in 2000 to $276 million in 2005. The total plementation. The project devised a process of credit flow from commercial banks to these graduation that would align a local government's groups is expected to reach $1 billion by the capabilities with its responsibilities in each of two time the project closes. World Bank investment ZAMSIF components: a community investment has proved to be catalytic and mobilization of fund financing community sub-projects and a 1 6 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT district investment fund financing capacity build- up to permit financial flows to a large number of ing and providing funds for district-level sub- poor areas. Among the most innovative aspects projects serving multiple communities managed of the program were mechanisms for trans- by local governments. Over time, the project parency (such as access to public records, media was restructured and its development objec- involvement, and NGO scrutiny) and to en- tives modified in the face several structural chal- courage institutions to respond quickly to pro- lenges, including delays by the government in posals and be responsive to community needs. enacting supportive decentralization policies-- The first KDP project covered about 30 percent notably new laws providing for elected local gov- of the rural sub-districts in the country and it has ernments and policies to increase the limited benefited more than 15,000 villages. Ongoing administrative, technical and fiscal capacity of evaluations have determined that KDP invest- nominated local governments. ments are more cost-effective than other mech- anisms for delivering similar services. Kecamatan Development Project (KDP), Indonesia Learning During Implementation. As an ex- This project is an example of a CDD program ample of learning by doing, Management en- built on rigorous social and institutional analy- couraged, supported, and financed (along with sis, adapted to country context, which includes DFID) a path-breaking study on monitoring pos- mechanisms for poverty targeting and inclusion. sible corruption (as measured by price versus In 1996, prior to the Asian economic crisis, a cost indicators) across different models of KDP local level institutions study portrayed the rich interventions. The study found that increasing variety of organizations and associations at that grass roots participation in monitoring village- level capable of planning and managing a broad level KDP interventions altered the method of range of development projects. It also illustrated possible corruption (it substantially reduced the that development resources rarely reached local theft of villagers' wages) but had relatively min- groups, and instead were channeled through imal effects on the overall level of possible cor- public sector institutions that crowded out local ruption. However, KDP interventions that also initiatives in favor of government and elite con- included an announced increased probability of trolled "user groups." With the onset of the eco- a government audit substantially reduced the nomic crisis, economic gains vanished, poverty level of possible corruption.15 Because there are worsened, and both government policies and in- no other such careful analyses of corruption for stitutions were discredited. In this context, a non-CDD operations it is not possible to bench- rapid response to poverty was required that mark whether the amount of corruption ob- would by-pass weak and ineffective line agencies, served in the CDD operations is greater or less establish transparent systems to deliver financial that in other operations. resources to communities to be used for high pri- ority social and infrastructure projects, and de- OED Recommendations velop institutional mechanisms at the local level The following paragraphs provide Management's that were more inclusive and community driven. specific comments on the review's recommen- dations. Targeting Financial Resources. To achieve its objectives, the project's financial resources were Recommendation 1. The Bank should targeted to the poorest sub-districts in rural provide operational guidance for the ap- areas where poverty was most highly concen- plication of Bank safeguard policies and trated, and mechanisms were built in to ensure fiduciary oversight of CBD/CDD projects that poor people, and especially poor women, and for the strengthening of cost-benefit were incorporated into decision making about analysis and M&E [monitoring and eval- priorities and project proposals. Although orig- uation] systems; and should commission inally intended to be small, the project was scaled an audit of the fiduciary aspects of a rep- 1 6 4 ANNEX S resentative sample of CDD projects for ations and continues to work with implement- submission to the Board within a year. ing agencies in borrowing countries to improve such analysis. There are fundamental challenges Operational Guidance on CDD Safeguards in undertaking ex-ante cost-benefit analysis for and Fiduciary Oversight. Management notes operations that involve decentralized community that it has been engaged for some time in pro- activities. These are principally related to the viding additional operational guidance and fi- difficulties in profiling with any degree of exac- duciary oversight and that CDD and "CBD/CDD" titude in advance the type and proportion of a operations have benefited as a result. The Qual- large number of dispersed sub-projects. How- ity Assurance and Compliance Unit (QACU), the ever, all project manuals, the client-produced Rural Development anchor, and the CDD an- documents that the World Bank appraises, in- chor have been conducting specialized training clude guidance on how to carry out cost-bene- programs on safeguards in CDD operations. Ad- fit analysis where appropriate. Management notes ditionally, the Regional safeguard teams rou- that QAG reviews and OED reviewed imple- tinely carry out thematic reviews of CDD mentation completion reviews tend to rate Bank operations to assess the level of compliance. performance in project preparation and super- The results of these studies have been used for vision higher for CBD and CDD operations than development of staff guidelines for application for overall Bank-supported operations.17 of safeguard policies in CDD operations. The CDD anchor will continue to collaborate with Monitoring and Evaluation. Management QACU and Regional teams in providing opera- notes that through its results agenda, much ef- tional guidance for the application of Bank safe- fort is being directed at assisting countries in im- guard policies during the preparation and proving M&E systems across its portfolio. implementation of CDD operations. There is no Concerning impact evaluation of CDD opera- evidence from OED's evaluations of environ- tions, Management notes significant commit- mental and social safeguard compliance, nor ment to conduct cutting edge research to from work by QACU or QAG that indicate that understand the impact of these initiatives and is those operations perform less well than the rest pleased to note that there are several impact of the World Bank's portfolio in terms of safe- evaluation efforts underway. The Bank's Devel- guard implementation. The "CBD/CDD" opera- opment Economics (DEC) vice presidency in- tions sampled by OED for this study and tends to produce a Policy Research Report in reviewed by QAG do better on safeguards and fi- FY07 on CDD operations, once a number of im- duciary aspects than the average in QAG reviews pact analyses of sufficient quality have been un- for all operations.16 The OED review's findings dertaken to provide accurate and operationally on medical waste do point out difficulties with relevant evidence. the application of the Environmental Assess- ment policy in health projects. However this is Fiduciary Aspects of CDD Projects. All CDD an issue for health projects in general and not operations have audits built into them, most specifically for CDD or "CDD/CBD" projects. using both the borrower's regular independent Similarly the report raises the issue of dams. audit system as well as special systems for inde- However, all the dams financed were below the pendent audits of sub-projects. However, given height that triggers the Operational Policy on the concerns raised in the OED report Manage- Dam Safety. ment will conduct a review of audits of a repre- sentative sample of CDD operations by the end Cost-Benefit Analysis. As indicated above, Man- of the year, with independent input into the agement agrees with OED's recommendation choice of the sample, and on the basis of the re- that it is important to do good economic analy- view findings will determine what course of ac- sis (including but not limited to cost-benefit tion to take. That said, Management notes that analysis) of all operations including CDD oper- IAD's extensive risk analysis in setting up its 1 6 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT FY06 work program did not raise CDD as a pri- operations, will contribute to achieving CAS out- ority concern for auditing. QAG data indicate comes. no special issue with fiduciary aspects in CDD op- erations; in fact the "CBD/CDD" operations sam- Local Government Capacity Development. pled by OED tend to score higher on fiduciary One of the mandatory issues of discussion in the aspects than other operations both in QAG qual- context of the review of the new generation of ity at entry and quality of supervision assess- results-based CASs (along with governance and ments.18 results) is capacity development, including local government capacity development. There is Recommendation 2. Future CASs should growing evidence that greater community par- show how they have analyzed and ad- ticipation improves local government capacity, dressed linkages not only between vari- particularly when operations are designed as ous CBD/CDD projects to be undertaken in part of decentralization efforts. In the mid-1990s, the country but also between CBD/CDD there was a concern that support for decentral- and relevant non-CBD/CDD projects. In ized initiatives could undermine local govern- particular, the analysis should address ment capacity. As a result of this concern, CDD whether arrangements for CBD/CDD proj- operations have evolved to include the explicit ect implementation come at the expense of objective to empower communities so that they local government capacity development. work with local government and serve as ac- countability mechanisms to improve local gov- Results-based CAS Methodology and Use of ernance (see examples from Zambia and Existing Guidelines. Management believes that Indonesia above). As evidenced by an increasing its recent guidelines for results-based CASs and number of CASs focused on governance that the process of corporate review of CASs and rely on combinations of CDD operations and CAS Completion Reports have now established decentralization support, Management believes a strong basis to learn lessons of past interven- that current guidance meets the objectives set tions and set out the Bank's proposed inter- out by OED. ventions within a medium-term results framework. All CASs are now required to be Recommendation 3. For any new based on a CAS Completion Report that evalu- CBD/CDD project, the Bank should ana- ates the development impact of past interven- lyze (using existing processes, such as so- tions and derives lessons for the design of future cial assessments) whether it is building interventions. Moreover, all CASs are required to on indigenously matured initiatives or identify country-owned development goals that attempting to begin a CDD program in a the Bank will support. In the CAS, country teams country and then tailor the intervention choose what they view as the most effective mix to local capacity; and the Bank should of instruments to deliver CAS outcomes. In turn, also selectively undertake rigorous im- these outcomes are expected to contribute to pact assessments upon completion of its achieving country's development goals. In the ongoing CBD/CDD projects to learn for new format, teams must adequately demonstrate the future. why they have chosen a given set of instruments. In addition, CASs go through an extensive cor- Social Assessments. Management agrees that porate review process that provides guidance to it is always important to understand the local in- CAS teams on possible weaknesses and defi- stitutional and social context, though that un- ciencies in analyzing and diagnosing develop- derstanding may not always need to arise from ment challenges and the choice of instruments a formal social assessment. Management notes to address these challenges. As part of this review, that social assessments are undertaken for many CAS teams will be asked to explain how opera- projects and supports their strategic use where tions and instruments, notably including CDD there are important knowledge gaps. However, 1 6 6 ANNEX S in certain situations such as natural disasters or Thus, Management believes that it is important post-conflict settings, speed of response is crit- to offer to support client governments that re- ical and these knowledge gaps would need to be quest assistance with community development, filled in parallel with project implementation. even when the client has little experience in In addition as part of the implementation of the working with communities. Further, in conflict social development strategy recently reviewed by or recent post-conflict settings where local in- the Bank's Board of Executive Directors, social stitutions are very weak--including both for- analysis is increasingly moving upstream to the mal government structures and informal policy and CAS level, which heightens the impact non-government institutions--Bank efforts to and lowers the cost. On the specific issue of support reconstruction and fight poverty may identifying whether a country (not the Bank-- still best be served by CDD operations and evi- the Bank supports the client country) is build- dence suggests that these types of operations do ing on an indigenously matured initiative, that better in fragile environments than alternative information is normally available as part of ap- approaches. praisal documentation. Rigorous Impact Assessments. Management Indigenously Matured Initiatives. It is gen- notes that, as outlined in Annex II, it has sup- erally true for all projects that they do better ported or facilitated impact assessments for a when building on indigenously matured initia- large number of client-country CDD operations. tives and when design reflects local capacity. Further, through the auspices of its Develop- The key operational question is what type of ment Impact Evaluation (DIME) initiative, it is intervention to support in low capacity countries, promoting more rigorous impact evaluation traditional top-down interventions that rely on across the portfolio. The Office of the Chief ministries for implementation or projects that Economist is particularly committed to increas- also build on local community participation and ing the number of operations that are subject to community control over decisions? Manage- careful impact evaluation and DEC resources ment, therefore, does not interpret OED's rec- support a number of such studies, including ommendation on tailoring to local capacity to several ongoing impact evaluations of CDD op- mean that CDD or "CBD/CDD" operations erations. Management is committed to expand- should necessarily be the exception in these ing further its evaluation work, especially of settings. An important aspect of the Bank's role social impacts related to enhanced social capi- is to support innovation in our client countries. tal, enhanced local capacity and empowerment. 1 6 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Management Action Record OED Recommendation Management Response The Bank should provide operational guidance for the application of Management believes that current operational guidance is adequate. Bank safeguard policies and fiduciary oversight of CBD/CDD projects Concerning the specific sub-set of CDD operations, Management is al- and for the strengthening of cost-benefit analysis and M&E systems; ready providing operational guidance on safeguard and fiduciary sys- and should commission an audit of the fiduciary aspects of a repre- tems, along with training and special reviews. sentative sample of CDD projects for submission to the Board within a year. Management is committed to a review of its control framework under IDA14 and considers this as the highest priority use of its resources in this area. However, Management will conduct a review of the au- dits of an independently selected representative sample of CDD op- erations by the end of the year following which it will determine what course of action to take, including, as needed, updating current fidu- ciary guidelines for CDD operations Management concurs with regard to the importance of M&E systems; however, its commitment for action is related to impact evaluation (see Recommendation 3 below). Future CASs should show how they have analyzed and addressed With the mainstreaming of the results-based CAS during FY05, all CASs linkages not only between various CBD/CDD projects to be undertaken are now required to follow a results-based approach. CASs are to be in the country but also between CBD/CDD and relevant non-CBD/CDD based on a CAS Completion Report that evaluates the development projects. In particular, the analysis should address whether arrange- impact of past interventions and derives lessons for the design of fu- ments for CBD/CDD project implementation come at the expense of ture interventions. The Bank analyses the most effective mix of in- local government capacity development. struments to deliver CAS outcomes and through that analysis demonstrates why the proposed set of instruments was chosen. CDD operations need to meet this test. In addition, CASs go through an ex- tensive corporate review process that provides guidance to CAS teams on possible weaknesses and deficiencies in analyzing and diagnosing development challenges and the choice of instruments to address these challenges. Hence, Management believes that its program of mainstreaming results-based CASs fulfills this recommendation. Management finds that many of its most innovative new operations are designed to combine decentralization initiatives with CDD opera- tions to improve local governance. These programs are based on the operational lessons that empowered local communities that are able to hold local government accountable for service delivery, improve local government capacity to deliver effective and demand-responsive ser- vices. Management believes that current guidance and assistance to staff meet the objectives of the OED recommendation and does not plan further steps. 1 6 8 ANNEX S Management Action Record (continued) OED Recommendation Management Response For any new CBD/CDD project, the Bank should analyze (using exist- Management agrees that social assessments are valuable tools that ing processes, such as social assessments) whether it is building on are widely employed in CDD operations. It also agrees that CDD de- indigenously matured initiatives or attempting to begin a CDD program sign will depend on whether the operation is building on an indige- in a country and then tailor the intervention to local capacity; and the nously matured initiative or is responding to a request from a client Bank should also selectively undertake rigorous impact assessments that does not have such an initiative. Management believes that this upon completion of its ongoing CBD/CDD projects to learn for the fu- is already the case--the importance of tailoring support for CDD op- ture. erations to local capacity is already a priority. Therefore, Management does not plan further follow-up with regard to this recommendation. (Management notes that CDD is an important approach in post-con- flict settings where "indigenously matured initiatives" are often ab- sent and is an important tool in implementing its operational policy on Development Cooperation and Conflict--OP2.30.) Management notes that borrower governments supported by the Bank are already undertaking many impact evaluations of CDD projects. Sev- eral sophisticated evaluations that meet the highest academic stan- dards are currently underway. Given the technical challenges of doing those evaluations well, Management is pleased to note that the Chief Economist and DEC will lead the effort to conduct a meta-evaluation of these impact assessments that result. That assessment is planned for FY07. 1 6 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Annex S.I: Management Concerns with Scope and Methodology Management's concerns over the conclusions Bank supports in Rio Grande do Norte. The con- and recommendations of this review result from trol group included municipalities and commu- issues around the scope of the OED inquiry. nity associations that had been involved in these Though Management tracks CDD operations previously Bank-financed CDD-based rural de- and those that involve community participation, velopment projects in Rio Grande do Norte. OED chose to establish its own portfolio of op- OED's "control" communities had specific mu- erations for this review. As a result of this deci- nicipal councils. Only those municipalities that sion, operations that OED examined, both in its had participated in CDD operations had these portfolio review and its specific cases, differ from municipal councils. Second, the household sur- those that Management tracks and guides. Specif- vey methodology, on which most of the con- ically, operations OED cites when drawing con- clusions are based, had an important built-in clusions about "CDD" are not CDD operations negative bias concerning the beneficiary com- in Management's view. OED based its evalua- munities. The beneficiary groups defined by tion on a word-search in project documents and OED included households that were not bene- refined the set using OED staff judgment as to ficiaries of the operation. The OED study used whether an operation should be included in the towns and villages to sample beneficiaries and portfolio it examined. Management's tracking these generally do not coincide with beneficiary of CDD operations is based on Task Team Lead- associations. Third, the OED review excluded key ers designating their projects as CDD, subject to non-income measures of poverty (notably, health review by Regional Management. Given their ex- benefits, savings in time because of more ac- tensive, in-the-field experience working with cessible and reliable water supply, and reduction client governments to identify and appraise op- to vulnerability to drought) and asset accumu- erations, Management believes that operational lation from its analysis of program impact. It staff are best placed to identify operations that also introduced negative biases in the measure take a CDD approach. of impacts by excluding from the analysis the kind Beyond these difficulties of scope and defi- of activities more directly linked to income gen- nition, Management also questions OED's eval- eration (the productive subprojects). Fourth, uative methodology, particularly concerning its the program OED selected for comparison pur- field work. An important illustration is its treat- poses differed fundamentally in design and ob- ment of a Brazil operation in Rio Grande do jectives (top-down, large infrastructure Norte, which is the only Management-recog- investments with no empowerment objective) nized CDD operation for which OED conducted from the RPAP whose beneficiaries were the sub- field work. First, there are important questions ject of OED's study. Finally, the OED review regarding the control group in the OED study. could have drawn more broadly on the literature The OED selection of control municipalities and covering the whole time span of the program. community associations was based on their not This issue is important because a crucial ele- having benefited from the Rural Poverty Allevi- ment of Brazil CDD program is its evolution ation Project I (RPAP). However, RPAP is only one over time, learning from experience.19 of a series of CDD rural development projects the 1 7 0 ANNEX S Annex S.II: List of CDD Operations with Impact Evaluations (completed and ongoing) 1. Albania Development Fund 21. Nicaragua Emergency Social Investment 2. Armenia Social Investment Fund Fund (FISE1) 3. Armenia Social Investment Fund 2 22. Pakistan Aga Khan Rural Support Project 4. Benin Social Fund (AKRSP-Kwaja Study) 5. Bolivia Social Investment Fund 23. Pakistan National Rural Support Program 6. Brazil Rural Poverty Reduction Project (DEC Study) 7. Cambodia Seila/Rural Investment and Local 24. Panama Rural Poverty and National Governance Project Resources Project 8. Ethiopia Women's Development Initiative 25. Peru Social Fund (FONCODES) Project 26. Philippines KALAHI-CIDSS 9. El Salvador Community-Managed Schools 27. Philippines ARMM Social Fund Program (EDUCO) 28. Senegal PNIR (Projet National d'Infrastruc- 10. Honduras Social Investment Fund tures Rurales) 11. India Andhra Pradesh District Poverty 29. Burkina Faso 2nd National Services Initiative Project Development Project (PNDSA2) and 12. Indonesia Support for Conflict-Ridden Senegal PSAOP (Programme d'Appui au Areas (SCRAP) Secteur Agricole et aux Organisations 13. Indonesia Urban Poverty Project 2 (UPP2) Paysannes) 14. Indonesia Kecamatan Development Project 30. Sierra Leone GoBIFO and IRCBP (KDP1, KDP2, KDP3) 31. St. Lucia Poverty Reduction Fund 15. Indonesia KDP and Conflict 32. Thailand Social Investment Fund 16. Indonesia Decentralized Agriculture and 33. Tanzania Social Action Fund Forestry Extension Project (DAFEP) 34. Vietnam Northern Mountains Poverty 17. Laos Poverty Reduction Fund Project Reduction Project (NMPRP) 18. Malawi Social Action Fund (MASAF) 35. Vietnam Community Based Rural 19. Moldova Social Investment Fund Infrastructure Project (CBRIP) 20. Nepal Rural Water Supply Project 36. Zambia Social Investment Fund (ZAMSIF) 1 7 1 ANNEX T: CHAIRMAN'S SUMMARY: COMMITTEE ON DEVELOPMENT EFFECTIVENESS (CODE) On August 31, 2005 the Committee on Devel- before completing the discussion on this topic. opment Effectiveness (CODE) reviewed the re- To remove any ambiguity regarding the scope of vised draft Management Response to The the review, OED changed the title of the report Effectiveness of World Bank Support for Com- to The Effectiveness of World Bank Support for munity -Based and -Driven Development: An Community-Based and ­Driven Development. OED Evaluation as agreed at the first CODE meeting (June 22, 2005) on this topic. On July Revised Draft Management Response (MR). 27, 2005 a CODE informal roundtable meeting Management concurred with the findings that was held so CODE members may better under- CDD and CBD operations achieve better results stand some of the issues raised in the OED eval- than other operations that do not involve com- uation, taking into consideration more recent munity participation. Management also agreed experience. Written statements by some mem- with OED that sustainability, monitoring and bers were circulated before the meeting. evaluation, and local leadership are important is- sues for the Bank's work on CDD, as highlighted Background. On June 22, 2005 CODE reviewed in the revised MR. The revised MR elaborated on The Effectiveness of World Bank Support for Com- how some concerns raised by OED are being ad- munity Development: An OED Evaluation (the dressed in recent CDD operations while com- Report) and draft Management Response (MR), menting on other issues including: (i) some of for which an interim "Green Sheet" was issued the analysis concerning safeguards and fiduciary on July 13, 2005. Generally, the Committee per- aspects of CDD; (ii) interpreting the OED rec- ceived that community development (CD) op- ommendation to approach CDD with greater erations had the potential to effectively help care; and (iii) potential misunderstanding of the and empower the poor, but it also took note of Bank's role in CDD operations, which are owned the report's cautionary findings. At the same and implemented by countries and not by the time, several members and speakers had difficulty Bank. Management also continued to express drawing a clear position on the report and MR concerns about the definitions used and the given the disagreements between OED and Man- scope and methodology of the evaluation. Given agement on some points including the defini- the lengthy process to review this OED evalua- tions used and the scope and methodology of the tion, Management highlighted some key lessons evaluation, as well as some of the conclusions. learned including: (i) importance of discussing Accordingly, as suggested by a member, an in- the issue of scope and definition at an early formal roundtable meeting was organized on stage when the Approach Paper of evaluations July 27, 2005 to enable CODE members to gain are presented; (ii) consider a standing commit- a better understanding of the issues raised in the tee to discuss methodological issues; and (iii) OED evaluation, based on current experience, need for care in ensuring balanced communi- both positive and negative, of community-driven cation of both OED and Management views. development (CDD) operations. At the earlier CODE meeting, it was agreed that the MR needed OED Comments. OED welcomed the revised to be revised and further considered by CODE MR. OED was happy to note Management's af- 1 7 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT firmations that it was already acting on many of focused evaluation on CDD operations as pro- the issues raised in the evaluation, which it will posed by some members. Finally, given the monitor and report on results. OED restated lengthy review process for this OED evaluation, the importance of a fiduciary audit of a sample Management and OED were requested to ex- of projects, which should be representative and plore possible procedural improvements to be independently selected. On the issue of scope considered by the Committee in due course. of the evaluation and specific interest in a review The following main issues were raised during of only CDD operations, OED stated that it is very the meeting: difficult to separate CBD and CDD elements in a project to enable an evaluation of only CDD ini- Substantive Issues Raised in the OED Eval- tiatives. OED expressed confidence in its method- uation. Members commented on some of the ology and the rigor of its work, which had been issues raised in the OED evaluation, which had reviewed by internal and external experts. been discussed at previous CODE meetings: Overall Conclusions and Next Steps. Fol- a. Sustainability, Aid Dependency, and Exit lowing extensive discussions, including those at Strategies. In considering sustainability be- the earlier CODE and informal roundtable meet- yond the issue of aid dependency, several ings, the Committee recognized the important speakers stressed the importance of incorpo- contribution of CDD operations to empowering rating exit strategies in the design of CDD proj- and helping the poor and it generally supported ects, and suggested that the MR address this scaling-up Bank assistance in response to coun- matter further. Aid dependency was considered try demand. At the same time, the Committee a general issue, not specific to CDDs. Man- noted the OED report's cautionary findings, agement noted that CDD projects include which called for greater vigilance in designing strategies for exiting communities and that it CDD operations. The points highlighted in- works closely with client Governments to learn cluded issues of sustainability and need for exit which are most effective. Management also strategies; fiduciary matters; environment and so- noted that OED's sustainability ratings for the cial safeguards; monitoring and evaluation; build- CBD/CDD portfolio review results not only on ing on indigenously matured initiatives; parallel community-level sustainability but also on the structures and local government and community sustainability of funding mechanisms for capacities; and cost-benefit analysis. The Com- CBD/CDD approaches, mechanisms that are mittee welcomed the revised MR and was gen- susceptible to classic problems of aid de- erally satisfied with the changes. Members noted pendency. the more positive tone of the MR, although some b. Fiduciary Audit. Ensuring appropriate uti- members commented that it could be further ad- lization of funds, especially in the context of justed and also made suggestions to clarify spe- local initiatives and systems was considered cific aspects, which Management agreed to critical in the design of CDD operations. OED consider in the final MR. recommended a fiduciary audit of a repre- Members appreciated Management's accept- sentative sample of CDD operations be un- ance to undertake a fiduciary review of a repre- dertaken. Management assured the Committee sentative sample of CDDs that are selected that under the Bank's operational framework, independently. In proceeding with disclosure all operations supported by the Bank are sub- of the OED evaluation report according to the ject to audits by independent auditors. It pro- approved OED disclosure policy, the Committee posed to undertake a review of the audits of a emphasized the importance of balanced com- representative sample of CDD operations se- munication, including the possibility of placing lected independently; initially, the revised MR the summary of CODE discussions upfront in- proposed a fiduciary review of five large CDD stead of as annex to the OED report. OED will operations by the end of the year, and on the consider in a few years the possibility of a more basis of the review findings determine whether 1 7 4 ANNEX T to recommend to Internal Auditing Depart- strengthen impact evaluation, track quantita- ment (IAD) an audit of CDD operations. Mem- tive and qualitative results, and to support bers welcomed Management agreement to countries' M&E capacities. Management noted consider a representative sample of CDD op- that many quantitative and qualitative evalua- erations, which are independently selected. tions are already being carried out and agreed Management noted QAG data that indicate that ensuring government and Bank capacity that CDD does better than other operations on in this area is important. financial management. g. Cost-Benefit Analysis. While cost-benefit c. Environment Safeguards. Some members analysis may not be possible in some cases, a sought assurance that adequate guidelines few members stressed the need for stronger in- were in place, referring to Annex Q of the OED stitutional commitment in this area. They sug- report. A member noted the difficulty of com- gested that Management could provide further pliance in conflict and post-conflict situations guidance in this area as well as clarify in the MR where country systems may be very weak. the Bank's role given clients' capacity con- Management clarified that Annex Q referred to straints. Management agreed to address this CBD and CDD projects, and not just CDD op- issue more fully in the MR, noting that all op- erations, including issues such as medical waste erational manuals for CDD operations contain that are not normally relevant for CDD oper- guidance on how to carry out economic analy- ations. sis. Management will continue to monitor to en- d. Building on Indigenously Matured Ini- sure that guidance remains appropriate and its tiatives. A few members stated that the Man- use will be covered under M&E. agement response should clarify particular circumstances where Bank support to new ini- Communication and Disclosure. Speakers tiatives may be merited, given OED's findings acknowledged the importance of OED inde- of greater success with indigenously matured pendence, but also emphasized that there should initiatives. Some members noted that in post- be balanced communication of all views, in- conflict countries and LICUS, there may be cluding of Management's response. In disclosing merit in supporting new initiatives. Manage- evaluations in accordance with the approved ment noted that there is evidence that CDD OED disclosure policy, some speakers proposed does better than other types of operations in that the summary of CODE discussions be placed these difficult circumstances. at the beginning rather than as annex, particu- e. Building Local Government and Com- larly for potentially controversial reports. munity Capacities. Several members high- lighted the importance of strengthening local Lessons Learned. Some members noted that government and community capacities, and Management continued to have concerns about requested elaboration on how to prevent par- the definition, scope, and methodology of the allel structures detracting from this aim. A OED evaluation. A member commented that member supported the combination of "bot- there should be Management and OED agree- tom-up" and "top-down" approach to devel- ment on the objectives of the evaluation up- opment, but suggested that the MR clarify how front, while OED, as an independent evaluation CDD operations should embody the two ap- entity, may be best placed to make the final de- proaches. Management agreed to clarify further cision on methodology. It was pointed out that in the MR how CDD operations currently work the scope of this evaluation was explicitly out- on strengthening community and local and lined in the Approach Paper. Management noted national government capacity, where relevant. that, while it had communicated its difficulties f. Monitoring and Evaluation (M&E). Not- with the scope and methodology to OED infor- ing Management and OED agreement on im- mally, it had erred in not formally commenting portance of M&E, some members sought in the Approach Paper. In this connection, speak- assurance that staff is adequately equipped to ers agreed with Management that one key lesson 1 7 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT emerging from reviewing this OED evaluation shared with CODE members as and when nec- was the need to pay more careful attention to the essary. OED independence was considered im- Approach Paper for each evaluation. They stated portant, but the need to ensure OED's that any significant divergence between Man- accountability was also raised. Some members agement and OED should be brought to CODE'S suggested a follow-up evaluation more narrowly attention at an early stage. A few speakers also focused on CDD operations in the near future; supported Management's suggestion to explore OED agreed to consider the possibility in a few the possibility of a standing committee to discuss years time. methodology issues; OED stated that it already has in place a thorough system for vetting its Chander Mohan Vasudev, methodology by external experts which could be Chairman 1 7 6 ENDNOTES Executive Summary OED notes: The evaluation draws from this docu- 1. Management notes that CDD operations are clearly ment (listed in references as World Bank 2003d), the distinguished by their giving communities control over full title of which is: "Evaluating Social Funds: A Cross- decision making and resource allocation. Country Analysis of Community Investments"; also 2. Management notes that it tracks: (1) participa- known as "Social Funds 2000." This study is based on tion in operations; and (2) community-driven devel- data collected more than 5 years ago and reviews opment operations. It finds these two categories as only the experience of social funds--a specific subset being more operationally useful than the very diverse of CBD/CDD projects. sample that OED has put together for the purpose of 5. Management notes that Rawlings, Sherburne- this review. Benz, and van Domelen 2004, cited above by man- OED notes: The Bank's tracking of participation is agement, provides rigorous evidence that community not limited to community participation. Since it also management of investments offers significant po- covers participation of all stakeholders at both the proj- tential for cost savings. That report also provides ect and macro levels, it does not permit the separate careful analysis of the benefits of these types of proj- tracking of all CBD/CDD nor does it allow for an analy- ects. sis of participatory issues relevant at the community OED notes: The study cited by management refers level that could help inform the design of future to social funds only--a specific subset of CBD/CDD CBD/CDD operations (in each situation where the projects--and is based on field work in six countries. Bank considers fostering community development, it needs to draw from relevant experience from both Chapter 1 CDD and CBD projects). 1. The "community" in a Bank-supported project 3. Management notes that, when discussing weak- is considered a "unified, organic whole." Since the nesses, the table does not compare its set of CDD/CBD group of people in a "community" live in a particular operations with other Bank operations. area, share a common interest (water users associa- 4. Management notes that there have been a num- tions, herders, and the like), and are governed by a ber of rigorous evaluations done of operations that in- set of norms, its members are assumed to be in the volve community participation, which is close to the best position to identify their most pressing needs and definition OED created for this review. For example, problems. The assumption suggests that there are Rawlings, Sherburne-Benz and Van Domelen 2004, common problems that can be solved through com- A Cross-Country Analysis of Community Investments, munity consensus. While this may be true, it neglects Washington, D.C, The World Bank, synthesizes six community members' differences and power rela- careful empirical analyses of social funds, each of tionships, the conflicts, and the diversity of interests which was published in the World Bank Economic Re- that determine day-to-day behavior and that affect view. In addition, management notes that a large pro- the effectiveness of participatory approaches. The portion of the most rigorous evaluation work under poor themselves are rarely a homogenous group; way concerns CDD impact. Given the relatively recent they live in different geographic areas and face different Bank operational support to CDD, there has been kinds of deprivations, and each seeks a personalized little opportunity for careful before and after evalua- way of reducing poverty (Schneider 1999). For more tion to be completed. on community see Annex A. 1 7 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 2. Initially, six "ways" of involving communities in 2. This finding is supported by a Bank study on the Bank-supported interventions were defined, though performance of community-level user groups carried later the Bank's Social Development Department out in the context of three Bank-supported inter- adopted a more intuitive fourfold classification (in- ventions in India. The study questioned whether user formation sharing, consultation, collaboration, em- groups as currently designed and implemented can powerment). Although it is not stated, presumably achieve long-term sustainability as independent or- both categorizations were derived from the work of ganizational entities. According to the study, past ex- Arnstein (1969), who created an eight-rung ladder of perience with various government and NGO initiatives citizen participation. has habituated user groups to let final responsibility 3. The CDD Anchor in its recently established CDD for the groups' management rest with an external au- database (see Chapter 2), also includes projects that thority (Alsop and others 2002). involve community control in the development and 3. Moore (2001) examines the use of the concept implementation of their donor-supported develop- of empowerment in multilateral development agen- ment plans, as well as projects that focus primarily on cies, including the World Bank, and finds that while information sharing, consultation, and collaboration. the notion of empowering the poor was important in 4. Management notes again that the definition framing the discussion on poverty reduction in these used by OED is closest to that of participation and that institutions, the lack of a clear definition could lead there are benchmarks for evaluating participation, to different interpretations of the concept of em- specifically those laid out in the Participation Source- powerment. The review finds two distinct implicit book. definitions of empowerment within the reports in 5. Management notes that it tracks: (1) participa- these institutions: the underlying proposition of one tion in operations, and(2) community-driven devel- definition is that improving the material status of opment operations. For operational relevance, poor people is empowering because it weakens the management believes that the review would have been social, economic, and political dependence and pro- more useful had it been based on either or both of the vides the poor with greater freedom and autonomy; categories that management tracks. the underlying proposition behind the second defi- 6. Management notes that, given the Review's def- nition is that insofar as empowerment contributes to inition of CBD/CDD, there is a larger set of impact eval- collective organization of the poor, it can also con- uation evidence to rely on. For example, Rawlings, tribute to political action and may imply political con- Sherburne-Benz and van Domelen 2004, cited by man- frontation. agement above, reviews a set of high-quality empiri- 4. It is important to acknowledge that it is much cal studies on community participation and its impact. more difficult to meet and measure the qualitative goals than the quantitative goals. For example, house- Chapter 2 hold survey data analysis from Benin found that com- 1. The Anchor's CDD database also includes in- munities' access to school infrastructure, supplies, formation on CBD projects. and equipment increased because of the Bank-sup- 2. Over two-thirds of the projects in the sample of ported projects, though this did not necessarily mean 84 devote more than 75 percent of project cost to improvement in access to education services, which CBD/CDD. Since community participation is a process is also determined by, among other things, availabil- issue, it has the potential to affect project outcomes ity of teachers. (See Annex P and section on sustain- even when it forms a small part of the project cost. ability in Chapter 4 for an explanation.) Moreover, 3. Management notes that there is often a detailed part of the improvement in capacity is expected to description of participatory processes in the project come about through the participation of the com- appraisal documents. In addition, these processes munities in the planning, design, and construction of are discussed in great detail in project and field the infrastructure. As a result, in the short duration manuals. of a Bank-supported intervention, when achieving visible results is seen as an indicator of progress on Chapter 3 implementation, progress toward qualitative goals 1. Using the Bank's primary sector coding. tends to receive less attention, not only from the 1 7 8 ENDNOTES Bank, but also from the country. trust, association life, and mobilization skills than the Management notes that rigorous cross-country relatively better-off (see Annex N). studies, not cited in this review, found that schools 13. The Bank staff survey found that only 40 per- funded with community inputs have more and better cent of the respondents were satisfied or very satis- teachers than those that were funded in the traditional fied with their Regional management's understanding top-down manner. Further, in the case of Benin, all of the objectives and design of CBD/CDD interven- schools funded by the project had teachers. tions. This suggests that Bank management may not 5. "Participation can make development assistance completely understand what is needed for CBD/CDD more effective. But it works best for groups that are to succeed. This may also explain why Bank proce- already able to help themselves" (Da Cunha and oth- dures have not yet been fully adapted to CBD/CDD. ers 1997). 14. It is worth noting here that the literature sug- 6. The literature and earlier OED evaluations also gests--and earlier OED work on social fund projects consider the dairy experience in India among the few confirms--that having the elite and the local leaders examples of successful scaling up of sustainable rural in the decision-making position does not necessarily development (Krishna and others 1997; Uphoff and constitute a problem in and of itself. In fact, the lit- others 1998; Candler and Kumar 1998). erature shows that the involvement of the elite could 7. SEWA has had some Ford Foundation support also be helpful in some ways. The important issue is but no major external assistance. The Orangi Pilot not so much how to avoid elite domination, but how project received support from several donors, in- best to use the power and energy of the elite to serve cluding the World Bank (see Uphoff and others 1998; the poor (Narayan 1995). One way of doing so, per- Krishna and others 1997 for details). haps, would be to create appropriate mechanisms to 8. Requested and funded by the Aga Khan Foun- ensure, before a subproject is funded, "that the ideas dation. of the leader are also the most important ones for the 9. Since in Uttar Pradesh the comparator commu- community as a whole" (OED 2002b, p. xxvii). As Plat- nities had no program, whereas in Madhya Pradesh the teau and Gaspar (2003) argue, so long as the inter- comparator was communities with the same activity, vention of the elite leads to an improvement in the also supported through a participatory approach, but situation of the poor, the latter are likely to be thank- by the government, it is not surprising that results are ful to their leaders. more significant in Uttar Pradesh than in Madhya 15. Management notes that it did not concur with Pradesh. the analysis and conclusions of OED's social funds 10. Management notes that other reviews have evaluation. found quite different and more positive results from 16. "The danger is not going far enough, and being those cited here. satisfied with any partial progress," as Ostrom (1999) 11. "But certain cultural settings are better suited puts it, "creating dependent citizens rather than en- to local participation and collective action than oth- trepreneurial citizens reduces the capacity of citizens ers. Participation works best when it is based in, rather to produce capital." The costs of development assis- than in opposition to, existing organizations. In North- tance will also inevitably increase--it is not costless east Brazil, regional tradition and existing social, eco- to establish new organizations." The "stages" theory nomic, and political structures pose strong challenges also notes that "progression is not taken to be in- to horizontal social organization and thus to popular evitable, with outcomes being regression (going back mobilization and participatory development. Although to the previous stage), stagnation or arrested devel- certain democratizing trends have weakened tradi- opment (remaining at one stage), and extinction (or- tional authorities, prior organization still poses chal- ganizations may fail or terminate)" (Pretty and Ward lenges, which must be recognized and systematically 2001). addressed in policy planning and in project design and 17. A recent Bank study (Alsop and others 2002) implementation" (Costa and others 1997). based on fieldwork in three states of India also found 12. It is worth noting that household data also that members' perceptions of the purpose of user show that the poor in project communities in Madhya groups differ from the perceptions of project de- Pradesh reported a significantly greater increase in signers and implementers. 1 7 9 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 18. In his study of 60 villages in the state of Ra- explain, poverty tends to shrink the time horizon, jasthan in India, Krishna (2001) shows that no matter because it forces people to attach considerable im- how high the level of social capital, it needs to be mar- portance to their present income opportunities. Con- shaled strategically and directed toward incentives sequently, the poor are likely to resist any type of available within the broader institutional and envi- collective activity that requires them to forgo present ronment. income opportunities--even if it permanently in- 19. Management notes that rigorous evaluations on creases future incomes. This argument is echoed by which Rawling, Sherburne-Benz, and Van Domelen, Weinberger and Jutting's (2001) quantitative analysis. 2004, cited above, is based show that the bottom two 24. The literature review carried out for this eval- deciles of the income distribution benefit more than uation distinguishes between formal inclusion, which higher deciles. concerns the extent to which community members are OED notes: Of the five social funds for which data able to enter decision-making arenas, and substantive are presented in the study cited by management, four inclusion, which captures the extent to which differ- social funds (Honduras, Nicaragua, Peru, Zambia) had ent participants are able to exert influence over de- a neutral or only mildly progressive distribution of ben- cisions. For example, poor people may opt not to eficiaries at the household level, and the fifth social speak up against the views and positions put forward fund (Armenia) had a regressive distribution (see by more powerful members of the community but, pages 64­65, table 3.3 and figure 3.3 of the manage- rather, conform to them (Kolavalli and Kerr 2002, ment-cited study); this lack of progressivity is of con- p. 225). As Linden (1997) points out, keeping a low cern because social funds aim to reach poor profile is an essential survival strategy of poor people. communities and households using a variety of tar- In the context of patronage, the poor are highly de- geting mechanisms. pendent on their leaders, and are hence unwilling to 20. A recent study by the Institute for Social and antagonize them (Kumar 2002). Economic Change in India, which carried out fieldwork 25. Rajasekhar and others (2004) found that the ac- in the States of Karnataka and Uttaranchal, found sig- tivity preference across the local organizations was to nificant elite capture of benefits in participatory wa- a large extent influenced by the level of contribu- tershed projects (Rajasekhar and others 2004). tions. The study reports: "The well-off farmers needed 21. In the context of irrigation schemes in Tanza- certain type of activities to be undertaken. Under the nia, Koopman and others (2001) find that while framework of beneficiary contribution having impor- landowners were the main beneficiaries, they did not tant say in the activity, the well-off farmers contributed shoulder a greater share of the costs. Instead, the net more and also obtained more from the groups. This costs borne by tenants were significantly higher, for was, thus a case of `elite capture,' which had negative they were not only required to put in as much free impact on the processes that the [local organizations] labor as landowners, but also ended up paying higher had to engage in." rents, because the project increased the value of 26. The literature also shows that inadequate at- land. tention to issues such as inequality within a commu- 22. The expression "participatory spaces" (Corn- nity can also affect targeting outcomes of CDD wall 2002) is used here to identify the various types interventions, because economic and social hetero- of community organizations, formal and informal, geneity can increase the risk of elite capture (Elbers created to enable the participation of project benefi- and others 2004) Governments rarely have informa- ciaries in the process of making decisions. tion on the level of inequality in communities. 23. The literature also notes reasons for the lack 27. For example, in the context of the Indian Pan- of participation of the poor. Participation places ad- chayat Raj system in Karnataka, Vyasulu and Vyasulu ditional demands on community members, which are (1999) found that women attained high levels of for- likely to be particularly problematic for poorer house- mal inclusion. At the gram panchayat level, well over holds (Pantoja 2000; Garcia and Way 2003). As Baland 40 percent of elected representatives were women. and Platteau (2002) point out, the poor often lack in- However, many elected women were surrogates for centives to take part in collective undertakings, as husbands and fathers who could not contest because these violate their survival constraints. As the authors of the reservation, while others had been put in place 1 8 0 ENDNOTES by the wealthy and powerful for their malleability. found that the important issue of what impact the on- 28. For example, a socioeconomic impact study was going adjudication of Bedouin lands is going to have carried out in connection with the Borgou Pilot proj- on the poor and on rangeland management systems ect in Benin. The OED assessment for the project was ignored. Further, the Egypt country study found questioned the credibility of the ERR rates reported that in the Sohag Project, while the community process by that study. In Brazil, a review of an often-quoted im- elements were quite well covered, QAG noted that the pact study by van Zyl and others (2000) revealed that larger policy issue of subsidized interest rates was in- the study had not compared project communities adequately addressed, related fiscal issues were not against controls. In addition, the study report provided well covered, institutions had not been adequately ap- no information on the number of respondents in the praised for capacity, and the risk that the project three different kinds of project communities, and no could undermine the development of alternative indication of how many subprojects had been financed credit sources had not been adequately assessed. per community. Further, there are several shortcom- 3. Bank management notes in response that the ings in the cost-benefit analysis reported. new national CDD project in Benin (approved by the 29. That report notes that, "Although social funds Board in October 2005) is fully taking the decentral- were typically more efficient than other national pro- ization framework into account and intends to build grams in term of overhead expenditures, their in- local government's capacity to implement the CDD vestment unit costs tended to be more efficient only approach. where there was significant input and control by com- 4. Management notes that communes did not exist munities." (World Bank 2003d) until one year after the Borgou Project became ef- 30. Kent and Rimarachin's (1994) study on public fective. works in Peru found that community contribution 5. The majority of the councilors interviewed in Rio ranged from 7 to 47 percent, and was on average 20 Grande do Norte had not received any training by the percent of total project costs. Isham and Kahkonen RPAP or the follow-on and ongoing RPRP. From the (2002) found that community contribution to water NGO focus group session it emerged that a training service projects in Sri Lanka was 43 percent of total event for municipal councilors had been organized in construction costs--well above the required 20 per- the state capital in the past. Some NGO representa- cent. tives criticized this form of training for it only allowed 31. Management notes that the background work the participation of few councilors per municipality for Rawlings, Sherburne-Benz and Van Domelen 2004, and was a one-time event, and not a systematic train- cited above by management, found that community ing program. management of investments offers significant poten- 6. It is worth noting that the latest ongoing CDD tial for cost savings, often on the order of 25­40 per- projects in Northeast Brazil are attempting to promote cent, even after taking into account the full value of greater integration of existing programs to improve community contributions. the impact of public resources available for poverty 32. Econometric analysis by Kerr and others (2000) reduction. However, it is too early to say how suc- finds that participatory watershed projects in India cessful this effort will be. performed better than technocratic and top-down 7. Three models of implementation are being used projects. The authors also find that a combination of in community-based projects in Philippines: The gov- participation and sound technical input performed ernment agency model has a line ministry or local gov- best of all. Similarly, Kahkonen's (1999) review of the ernment as the executing agency for the project. It is literature maintains that community-managed water responsible for the design and implementation, pro- and sanitation projects worked better than govern- curement, financial management, monitoring, and ment-managed schemes. evaluation. The project is often carried out by a proj- ect management unit within the agency, and executed Chapter 4 by local offices of the ministry in conjunction with local 1. The capacity of the Bank to support a CBD/CDD authorities and communities. In the social fund model, approach is addressed in Chapter 5. an autonomous agency of government makes grants 2. The OED assessment of the Matrouh Project directly to beneficiary communities or individuals. In 1 8 1 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT the community-driven development model, the com- munities have less resources to contribute in com- munities themselves decide which projects to un- parison with richer communities. Further, a review dertake and are responsible for their execution. The of community support for Basic Education in Sub- Comprehensive Integrated Delivery of Social Services Saharan Africa also found that support for recurrent (CIDSS) component of SEMP 1 and the proposed expenditures poses sustainability problems. Accord- SEMP 2 and the proposed "Kalahi" Project are exam- ing to the report, "Schools need a predictable flow of ples of this model. (Source: "Community-Driven De- income to meet their recurrent expenditures, yet velopment: A Case study of the Philippines." East Asia communities are often unable to provide this. This is Region Workshop June 12­13 2002.) especially the case among poor communities, who are 8. The rapid multiplication of national NGOs in re- often vulnerable to external shocks, and whose income sponse to the availability of funds from the interna- is irregular" (Watt 2001, p. 29). tional community is also an issue that has been raised 14. See Annex I for an explanation on project-sup- in the literature (Platteau and Gaspart 2003; Chabal ported municipal councils in northeast Brazil. and Daloz 1999). 15. It is worth noting that only 18 percent of the 9. It is important here to distinguish between respondents to the Bank Staff Survey agreed with the NGOs and grassroots community organizations, which statement that community maintenance contribu- may have different motivations for working with com- tions are sufficient to sustain infrastructure invest- munities. However, the country studies were not able ment for Bank-funded CBD/CDD projects (Annex L). to research the role of these organizations. 16. The Ethiopia Pastoral Community Develop- 10. Van de Walle and Johnston (1996) note in the ment Project (fiscal 2003) and the Nigeria Community- context of lack of aid coordination, "a conservative es- Based Poverty Reduction Project (fiscal 2001) are timate for a typical African country is that 600 projects examples. translates into 2,400 quarterly reports a year submit- 17. Among the infrastructures implemented in a vil- ted to different overseeing entities; and more than lage in Kalale commune of Benin's Borgou region 1,000 annual missions to appraise, monitor and eval- was an agricultural storage facility financed through uate. Each mission asks to meet with key officials, the Borgou Pilot Project. Ten percent of the total proj- and each will ask the government to comment on its ect cost was to be put into an account for repairs or reports. Is it any wonder that the most common com- other maintenance. When asked whether the funds plaint voiced by officials interviewed in the case stud- needed to be replenished when they were used, the ies is that aid imposes too many administrative village leader told OED they did not. When asked if burdens?" there was anything left of the 10 percent funds, he also 11. For northeast Brazil, specifically, the van Zyl said no, though it had only been two years since the study reports that 89 percent of the sample of 3,633 subproject had been completed. When asked what the subprojects funded by the RPAP in 1997­98 were community would do now if something were to hap- found to be operational in the year 2000. pen to the storage facility he was not at all worried and 12. Management notes the evidence presented in said that they would seek new funds. In this particu- the background research for Rawlings, Sherburne- lar village several Bank projects and other donor in- Benz and Van Domelen 2004, cited earlier by man- terventions had been implemented, so it would be agement. That work found that "impact evidence reasonable for the village committee to assume that showed that the facilities in which social funds invest they could just go to another donor when the need can be at least as sustainable as similar facilities if not arose. more so. The majority of the infrastructure appeared 18. A recent study on the role of local organizations to be well constructed and operating adequately and in water supply and sanitation in the states of Kar- levels of maintenance were equivalent or better than nataka and Uttaranchal notes that the possibility of ob- comparators." taining resources from the government toward the 13. OED's Impact Evaluation of Basic Education in maintenance of water supply projects could mean Ghana (OED 2004b) also found that the downside of that local organizations do not have an incentive to community financing of schools is that it leads to dis- mobilize resources from users (Rajasekhar and parities in resource availability, because poorer com- Veerashekhararappa 2004). 1 8 2 ENDNOTES 19. Further, OED's social fund evaluation found the mentioned at appraisal and no effort was made to set technical quality of social fund infrastructure to be vari- down guidelines for dam safety and to study the cu- able across countries and between sectors. The find- mulative input of a large number of small dams. How- ings of the recent self evaluation of social funds were ever, a Social Development Note (World Bank 2001c) similar in that respect (World Bank 2003d). mentioned that over a thousand such dams were built 20. "Ensuring that positions are filled, that staff under this program. report for work, and that they are responsive to all their 3. For example, in Nigeria the internal controls clients is a major challenge. ...Incentive payments are reported to be weak or inoperative and provide might encourage professionals to work in remote negligible assurance that the funds are being used en- areas, but they can be expensive....Even when posi- tirely for their intended purposes. In Senegal, even tions are filled, staff absence rates can be high" (World though the scope of NGO involvement has increased Bank 2004d, pp. 22­23). substantially, the CFAA notes that the supervisory ca- 21. Thus the ICR for the Cambodia Social Fund ac- pacity of NGOs' oversight bodies is very weak. Fidu- knowledges "sustainability was not set up as one of ciary sector work in Pakistan has reported that the main objectives because of the project's primary compliance with existing internal controls and pro- concern with the rapid and cost effective delivery of cedures have been inadequate and the financial re- subprojects. Another reason why sustainability con- ports and financial accounts have often been untimely cerns were not more integrated into project opera- or incomplete. tional activities was the assumption that a demand 4. There is considerable evidence in the literature driven identification process would ensure commu- on mismanagement of aid transfers that occur in class nity ownership and that line agencies would take over or caste-based societies (Platteau and Gaspart 2003; operations and maintenance of subprojects." Bardhan 2002; Conning and Kevane 2002). 22. The case study on Indonesia's Kecamatan De- 5. "As compared to other policy areas, performance velopment Program for the Shanghai Conference in developing countries is in general weakest in pub- (May 2004) points out: "Two limitations of the KDP lic sector governance. And within public sector gov- model are particularly worth underscoring since they ernance, it is weakest with respect to transparency, can to some extent be overcome. The first is that accountability, and control of corruption. And the technically difficult activities or activities that involve weaknesses are the most pervasive precisely in coun- recurrent costs are not easily addressed through the tries where stronger institutional capacities are needed KDP system as it is currently designed. Examples, of to manage development interventions that will spur such necessary activities include large-scale health progress toward the MDGs--poor countries" (World provision, providing teachers for schools, or any kind Bank-IMF 2004). of infrastructure network planning" (World Bank 6. At the Fiduciary Forum 04, managers recognized 2004b). If maintaining the services from schools the need to translate diagnostic into action. "Too often through CDD interventions is a challenge for In- we do excellent work on Country Financial Account- donesia, it is an even bigger challenge for countries ability Assessments and Country Procurement Assess- like Benin and Nepal. ment Reports, and then the country team checks these 23. A significant number of focus groups (50 per- off [its `to-do' list] and says, `Okay, that's done,' when cent of the female focus groups and 40 percent of the the real work actually starts afterwards" (Fiduciary male focus groups) in that state expressed an inter- Forum 04, Managing Risk Realizing Opportunity). est in receiving further training. 7. Internal documents from the midterm review of the Andhra Pradesh District Poverty Initiatives Project Chapter 5 noted that an independent review of financial and 1. Staff skills within the institution have also become procurement processes had been completed. The re- more diversified, and today Bank staff come from a va- view found that progress had been made, but it also riety of academic disciplines and varied institutional confirmed some recurring weaknesses. Specifically, experiences. compliance remained inadequate and the processes 2. For a CDD intervention covering eight states in in place provided limited assurance that the project northeast Brazil, construction of small dams was barely funds had been used for the intended purposes. 1 8 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 8. The fieldwork done for the Indonesia Keca- Kalahi - CIDSS Project (P077012); and Yemen: Third matan Development Project found that while the ma- Social Fund for Development Project (P082498). jority of people who lost small amounts of land to 2. These are undertaken to validate the findings of roads and other infrastructure did not seem overly project Implementation Completion Reports. concerned, a few were not happy. Since in virtually all 3. The literature on social funds is excluded from cases land values rose substantially because of the this review, as it has been the subject of a recent OED infrastructure in question, it would be difficult to evaluation (OED 2002b). argue that anyone ended up worse off than they were before the project. However, there is no clarity in the Annex E Bank's resettlement policy on whether loss of land re- 1. As of September 2004. A detailed explanation of quires compensation, even when land values go up the methodology for identifying the universe of more than the value of land lost--that is, whether rel- CBD/CDD is presented in the document "CDD: A ative loss or loss comparing the with- and without-proj- Study Methodology" available at http://www.world- ect scenario matters. In any event, the OED mission bank.org/oed/cbdcdd/documents/discussion_paper.pdf. found the appraisal text on compensation for lost This document was shared with workshop partici- land confusing and likely to put pressure on affected pants and was also posted on the above Web site for households not to seek compensation. comments. While there was discussion at the workshop 9. File:///C:/Documents%20and%20Settings/wb25 around the methodology adopted for identifying the 1042/Local%20Settings/Temporary%20Internet%20File universe, no alternative approaches were offered to s /Content.IE5/1WDNJU3F/300,29,World Bank Staff) identify the universe. Nor did any of the Regions pro- 10. "Compartmentalization within major aid or- vide an alternative list of projects. ganizations of the expertise and responsibilities to 2. The list of projects identified in the post-2000 support administrative reforms, sectoral assistance period by the word search was compared with the list programmes and community development projects, produced by the CDD Anchor. Some post-2000 proj- produces fragmented and competing interventions ects in OED's universe were absent from the CDD An- that do not address and even retard the systemic chor's list and some were additional to the list. All changes needed to advance decentralization" (Romeo these projects were reviewed for their appropriateness; 2003). some were dropped from the list of 833 projects that was available at the design stage and others were Annex B added to give a final total of 847 projects. 1. A detailed explanation is presented in the CDD evaluation discussion paper available at http://www. Annex F worldbank.org/oed/cbdcdd/documents/discussion_ 1. A detailed explanation of the methodology for paper.pdf. identifying the sample of CDD is presented in the document CDD: A Study Methodology: http://www. Annex C worldbank.org/oed/cbdcdd/documents/discussion_ 1. From the World Bank community-driven devel- paper. pdf. opment Web site accessed 1. OED ratings are based on OED reviews of ICRs, January 2005. 25 percent of which are subsequently revisited through OED field assessments. OED ratings are analyzed by Annex D exit year since OED rates each project on only on 1. The projects reviewed were: Albania: Second exit. Community Works Project (P077297); Cameroon: 2. OED rates outcome on a six-point scale: 6 = Community Development Program Support Project highly satisfactory, 5 = satisfactory, 4 = moderately sat- (P073629); Honduras: Community Based Education isfactory, 3 = moderately unsatisfactory, 2 = unsatis- Project (P007397); India: Andhra Pradesh District factory and 1 = highly unsatisfactory. These ratings are Poverty Initiatives Project (P045049); Philippines: presented in most reports, including the Annual Re- 1 8 4 ENDNOTES port on Portfolio Performance and Annual Report on from the project. The evaluation wanted to capture Development Effectiveness, on a two-point scale by this "negative" aspect as well for all projects-- summing the top three ratings (4 to 6) as satisfactory CBD/CDD and non-CBD/CDD projects. and summing the bottom three ratings (1-3) as un- 8. OED rates ID impact on a four-point scale: 4 = satisfactory. While a small percentage of projects is "not high, 3 = substantial, 2 = modest, and 1 = negligi- rated," to keep the denominator constant for all three ble. The percentage of projects with "substantial" ID ratings (outcome, sustainability, and institutional de- impact includes both "substantial" and "high" ratings. velopment impact), calculations for percent satisfac- tory projects are based on the denominator equaling Annex I all closed investment lending during the specified 1. In Vietnam, officials picked multiple choices, period. unlike the other three countries, where officials picked 3. Post-conflict countries with a CBD/CDD project one primary choice of central, regional/provincial, exiting during 1994­2003 include Angola, Burundi, municipal/local, communities, NGOs, other donors, Cambodia, Colombia, Congo, Democratic Republic of, others, and do not know. Djibouti, Eritrea, Ethiopia, Guatemala, Indonesia, 2. This was also raised in interviews with stake- Macedonia, former Yugoslav Republic of, Mozam- holders at various levels in Benin. bique, Philippines, Rwanda, Sierra Leone, Sri Lanka, Tajikistan, Timor-Leste, Uganda, and the West Bank and Annex J Gaza. 1. Note that we are not here talking of the costs of 4. Regional analysis is restricted to Regions with the investments but of the costs to all players of get- more than 25 projects. ting to the point of implementing and then supervising 5. The non-CBD/CDD portfolio has been per- that investment. forming below CBD/CDD portfolio for both Africa 2. It is reasonable to expect that these costs will de- and Latin America and the Caribbean, however, decline cline over time. in the performance of non-CBD/CDD portfolio in the 3. A Study of Rural Hill Potentials and Service De- Latin American and Caribbean Region over the two livery Systems, by SAPROS and IFAD, April 2002, IFAD. phases has been only 2 percentage points, and the in- 4. Sample sizes from 110 households up to 154 crease in the Africa Region portfolio has been 7 per- households. centage points. 5. Often there is little detailed evidence to back up 6. Sectoral analysis is restricted to Sector Boards the claims of savings. Making a fair comparison be- with more than 25 CBD/CDD projects. tween provision of infrastructure through different 7. OED rates sustainability on a four-point scale: means is a complex calculation calling for allocation 4 = highly likely, 3 = likely, 2 = unlikely, 1 = highly of a number of fixed costs that are difficult to allocate. unlikely, plus non-evaluable. Calculations for percent 6. The opportunity cost curve of household time "likely" or better (includes both "highly likely" and may be concave, with modest amounts of time spent "likely.") ratings in this section are based on all closed in meetings having quite small opportunity costs but, projects including projects rated non-evaluable and as the time increases, having quite substantial costs uncertain. Excluding "non-evaluable" pushes per- through impact on labor peaks related to the agri- centages upwards by 7 percentage points for cultural calendar. There is some anecdotal evidence CBD/CDD projects and by 5 percentage points for that the costs to the poor are greater because they can non-CBD/CDD projects in phase 2. The reasons to least afford the lost labor. keep the non-evaluable in the denominator are twofold: first, the evaluation wanted to keep the de- Annex L nominator constant for all three OED ratings, namely 1. Multivariate analysis indicates that respondents outcome, sustainability and Institutional impact; sec- from the East Asia and the Pacific and Latin America ond, non-evaluable rating is given to projects largely and Caribbean Regions were more likely to express dis- because of poor quality of the ICRs, which makes it satisfaction with coordination within the Bank across difficult for an evaluator to make any concrete as- sectors as compared to all other respondents (table sessment on the likely hood of sustainable benefits L.5). 1 8 5 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT 2. Multivariate analysis indicates that respondents reformulated NRDP. Only two significant differences from the Europe and Central Asia Region, and soci- were found. Respondents in the community that had ologist/anthropologists were less likely to agree with benefited from the reformulated NRDP reported a sig- the fact as were respondents who had participated in nificantly smaller increase in their participation in larger number of CBD/CDD projects. Interestingly, political events, and a significantly grater increase in managers (sector level and above) were more likely ownership of medium consumer durables, which was to agree that implementation costs per dollar lent driven by greater increase in ownership of satellite for CBD/CDD projects are higher than more traditional dishes. projects (table L.5). 6. The draft questionnaire is in available on the Web 3. Multivariate analysis indicates that economist site http://www.worldbank.org/oed/cbdcdd. were more likely to indicate disagreement; and re- 7. Benin was the only country for which weights spondents from Europe and Central Asia more likely were not used due to a lack of information on com- to indicate agreement (table L.5). munity population. 4. Multivariate analysis indicates that respondents 8. The interactive variable for women in project who had participated in larger number of CBD/CDD areas was included for the projects in Benin and Uttar projects were more likely to disagree with this state- Pradesh, as these targeted women explicitly (Annex ment (table L.5). N). 5. Multivariate analysis indicates that respondents 9. A likelihood ratio test was performed in order from Africa and Latin America and the Caribbean to test the validity of the restricted model and this were more likely to disagree with the fact that was rejected in favor of the unrestricted model, which NGO-supported interventions generally achieve a includes three separate dummy variables for the better CBD/CDD outcome than Bank interventions three types of implementation modalities. The three (table L.5). dummy variables were differently associated with changes in: (a) in associational life, (b) circle of Annex M friends, (c) access to information and (d) mobiliza- 1. AgeFIB became inactive in fiscal year 2004; and tion skills. the ICR presented it as a CDD intervention, though from the appraisal document it appeared to be a CBD Annex N project. 1. "CDD empowers poor people (...) Targeted 2. AgeFIB and PILSA were implemented in other community-driven approaches devolve control and de- regions of Benin also. cision making to poor women and men. This em- 3. While the first two types of investments were cho- powers them immediately and directly. (...) the speed sen because together they accounted for 60 percent and directness with which CDD empowers poor peo- of investments financed by the RPAP in Rio Grande do ple is rarely matched by other institutional frame- Norte, the second and third type were chosen because works for poverty reduction. (...) Control over eligible PAC communities had mainly these types of decisions and resources can also give communities investments the opportunity to build social capital (defined as 4. The on-going RPRP had restricted the number the ability of individuals to secure benefits as a result of municipalities targeted under the RPAP because of membership in social networks) by expanding some were financially able to meet the needs of their the depth and range of their networks." (http:// communities. These municipalities could not there- lnweb18.worldbank.org/ESSD/sdvext.nsf/09ByDoc- fore be appropriate comparator for this study. Name/BasicConceptsPrinciplesWhyCDD). 5. Differences between the comparator community 2. As already mentioned in Annex M, the bivariate that had benefited from the reformulated NRDP and analysis for the Brazil project reports only the response the other two comparator communities were tested rate of FUMAC communities, which account for 60 using the same model discussed on page 109, under percent of respondents in project communities. "Multivariate Analysis" (specification without inter- 3. In Madhya Pradesh, village forest committee actions). In this model the project dummy repre- (VFC) or the forest protection committee (FPC) were sented the community that had benefited from the created, which comprised of one male and one female 1 8 6 ENDNOTES member from each household in the village. In Uttar bers were better educated than non-members, and re- Pradesh the project created two distinct committees, ported greater mobilization skills and social network the Site Implementation Committee (SIC) and Water prior to subproject implementation than did non- User Group (WUG), which played distinct role. While members. A significantly greater number of SIC mem- the SIC decided on broader project management is- bers also owned oxen prior to subproject sues, the WUG decided on water usage and water implementation than did non-members. rates of a particular boring around which it was 8. As Kumar and Corbridge (2002) point out, vil- formed. In Brazil, a Community Association needed lage elites are likely to nominate themselves as rep- to be legally constituted for the community to take part resentatives in their role of gatekeepers of in the project. The Community Association was re- development interventions. In other cases, partici- sponsible for selecting the subproject, submitting a patory projects choose to work through village chiefs proposal, and if successful for implementing and or community leaders, for these are seen as legitimate maintaining the subproject. In Benin, a Comite de Con- and appropriate institutions of community repre- certation (CC) was set up in each community and sentation (Kumar and Corbridge 2002; Ribot 1998; Gib- was given primary responsibility for monitoring proj- son and Marks 1995). In addition, communities ect implementation at the field level. generally elect the most prominent members and 4. The idea that decision-making forums are neu- those with political connection with power-holders, tral and that by entering them people can meet on a as they are believed to be the only ones in the posi- level playing field has been criticized by a number of tion to attract benefits to the community (Linden scholars because it ignores that differences in the dis- 1997; Platteau and Gaspart 2003). tribution of power and resources among community 9. Around 40 percent of the 84 CBD/CDD projects members impinge on the process of collective deci- in the sample included an extensive dissemination sion-making (Leach and others 1999; White 1996; campaign on project information, while another 40 Molyneux 2002). percent had only some focus on dissemination, and 5. Attendance at meetings for project selection the remaining 20 percent had no major focus on dis- was lower in PAC and FUMAC-P communities, where semination. only 18 and 28 percent of the respondents respectively 10. The share of respondents in PAC and FUMAC- attended. P communities who were unaware of the cost of the 6. In PAC and FUMAC-P communities a much Bank-funded subproject was higher than in FUMAC smaller share of respondents reported speaking at the communities--respectively 92 and 91 percent. meetings for project selection, respectively 10 and 11. Based on her fieldwork in Northeast Brazil, 14 percent. Tendler (2000) writes that "information ... [was] sur- 7. In Brazil, a significantly greater number of CA prisingly low even in the Brazilian programs." members reported owning durable goods (car, mo- 12. The same is true for respondents in FUMAC-P torcycle, freezer, and satellite dish) and large animals communities. (ox and cow) prior to subproject implementation 13. The variable that captures changes in respon- than did non-members. CA members also reported sig- dents' mobilization skills is a composite variable. See nificantly greater mobilization skills and participation tables M.9­M.12 for details on how this variable was in political and socio-cultural events prior to subpro- created in each of the four project areas. ject implementation than did non-members. In Mad- 14. In Brazil, FUMAC-P communities in Brazil are hya Pradesh, a significantly greater number of negatively associated with changes in associational members of forest committees reported owning land life, while no significant association was found be- and large animals (ox and cow) prior to subproject im- tween the latter and PAC communities. plementation than did non-members. Committee 15. As defined by the World Bank Social Capital Web members were also better educated than non-mem- site http://www1.worldbank.org/prem/poverty/scap- bers, and reported greater mobilization skills, social ital/index.htm. network and participation in traditional and non-tra- 16. The variables that capture changes in trust and ditional events prior to subproject implementation associational life are composite variables. See tables than did non-members. In Uttar Pradesh, SIC mem- M.9­M.12 for details on how these two composite 1 8 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT variables were created in each of the four project and social concerns into development decision mak- areas. ing. 17. In Uttar Pradesh the positive association be- 2. The response rate on the questionnaire was 32 tween the project dummy and changes in trust is percent. driven by respondent's greater trust in village mem- 3. PPARs are prepared for selected projects by the bers and village organizations. Operations Evaluation Department, whereas ICRs are 18. In Brazil, although no association was found be- prepared for all projects by the Regions. tween the project dummies and changes in trust, re- spondents in all three types of project communities Annex S reported a significantly smaller increase in trust in the 1. The standard World Bank definition, available on municipal government than did respondents in com- the CDD Web site (http://www.worldbank.org/oed/ parator communities. cbdcdd), states that CDD is an approach that "gives 19. The Village-Level Participatory Approach in- control over planning decisions and investment re- cluded participatory rural appraisal and other part- sources to community groups and local governments." nership efforts that enable communities to coordinate 2. The Effectiveness of World Bank Support for and execute their own rural development project Community-Based and -Driven Development: An with assistance from extension agents and financial re- OED Evaluation, July 11, 2005. sources from a variety of programs. 3. An OED Review of Social Development in Bank Activities, Operations Evaluation Department, Annex O World Bank, February 17, 2004. 1. This information is based on an in-depth re- 4. OED notes that this is likely due to the fact that view of 13 projects from the random sample of 54 tar- management's CDD portfolio covers only projects geted projects in the sample of 84--two from fiscal approved from FY 00 onwards. years 1989­93, five from fiscal 1994­98, and six from 5. Approach Paper: Evaluation of the World fiscal 1999­2003. Bank's Support for Community Driven Development 2. In Benin, the Bank's social funds project (Age- (CDD) (CODE2003-0052), July 31, 2003. FIB) targeted poor rural and peri-urban communities, 6. OED notes that the Approach Paper (para. 6) also with a special focus on women and unemployed said that "Since most of the current CDD projects youth, and in the Community-based Food Security Pro- have evolved from the Bank's experience with CBD ject (PILSA) pre-identified problem areas were tar- projects, the evaluation will review both CBD and geted to ensure that assistance would be provided to CDD interventions." the most disadvantaged segments of the rural popu- 7. The title of the OED report first referred to lation. CDD, that was changed later to "community devel- opment," before changing it again recently to Annex P "CBD/CDD." 1. As productive subprojects are normally invest- 8. OED notes that it found in its investigation that ments that interest CA members in particular rather most projects evaluated contained both CBD and than communities as whole (though the latter can ben- CDD components and there are very few `pure' CDD efit indirectly form such investments), we only con- projects. Management notes that this is not surpris- sidered responses from CA members for the analysis ing given the fact that OED's sample was designed to of sustainability of irrigation subprojects. capture projects with both CBD and CDD compo- nents (see Annex E of the report). Annex Q 9. An OED Review of Social Development in Bank 1. The safeguard policies--covering environmen- Activities, Operations Evaluation Department, World tal assessment, natural habitats, pest management, in- Bank, February 17, 2004. voluntary resettlement, indigenous peoples, forests, 10. Evaluating Social Funds: A Cross-Country safety of dams, cultural property, projects on inter- Analysis of Community Investments, L.B. Rawlings, L. national waterways, and projects in disputed areas-- Sherburne-Benz, J. Van Domelen, World Bank, 2004. provide a mechanism for integrating environmental 11. Ibid. 1 8 8 ENDNOTES 12. OED notes that the list in Annex II includes im- 17. OED notes that its database shows no signifi- pact evaluations which are still ongoing and thus have cant difference in overall Bank performance between not been "carried out" yet. CBD/CDD operations and overall Bank-supported 13. OED notes that it did not find any evidence in operations. the documentation which it had access to that eco- 18. OED notes that its own reviews do not show nomic analysis was carried out for most individual that CDD operations do better on fiduciary aspects sub-projects. than other operations. Management notes that QAG 14. Getting an Earful: A Review of Beneficiary reviews of the sample of operations OED developed Assessments of Social Funds, D. Owen, J. van Dome- for this report highlight no particular concern about len, World Bank, 1998. CBD/CDD operations. 15. Olken, Benjamin, Monitoring Corruption: Ev- 19. OED emphasizes that it has reviewed the idence from a Field Experiment in Indonesia, NBER, methodology on the project very carefully and stands November 2004. by its conclusions. 16. OED notes that its own reviews do not show that CDD operations do better on safeguard and fi- duciary aspects than other operations. 1 8 9 REFERENCES Agarwal, B. 2001. "Participatory Exclusions, at the Inequality Cooperation and Sustain- Community Forestry, and Gender: An Analy- ability Workshop, Santa Fe Institute, Sep- sis of South Asia and a Conceptual Frame- tember 21­23, 2001. ------. 2000a. "Conceptualizing Environmen- accessed March 2005. tal Collective Action: Why Gender Matters." Bardhan, P. 2002. "Decentralization of Gover- Cambridge Journal of Economics nance and Development." Journal of Eco- 24(3):283­310. nomic Perspectives 16(4): 185­205. ------. 2000b. "Group Functioning and Com- Beck, Tony, and Cathy Nesmith. 2001. "Building munity Forestry in South Asia: A Gender Analy- on Poor People's Capacities: The Case of Com- sis and Conceptual Framework." WIDER mon Property Resources in India and West Working Paper 172, UNU/WIDER, Helsinki. Africa." World Development 29(1):119­33. Agrawal, A., and C.C. Gibson. 1999. "Enchant- Bennett, Lynn, Mike Goldberg, and Pamela ment and Disenchantment: The Role of Hunte. 1996. "Ownership and Sustainability: Community in Natural Resource Conserva- Lessons on Group Based Financial Services tion." World Development 27(4):629­49. from South Asia." Journal of International Alsop, Ruth, Disa Sjoblom, Ceema Namazie, and Development 8(2): 271­88. Pawan Patil. 2002. "Community-Level User Binswanger, H., and S. Aiyar. 2003. "Scaling up Groups in Three World Bank­Aided Projects: Community-Driven Development: Theoreti- Do they Perform as Expected?" Social Devel- cal Underpinnings and Program Design opment Papers 40. World Bank Environmen- Implications." World Bank Policy Research tally and Socially Sustainable Development Working Paper 3039. Washington D.C.: World (ESSD) Network, Washington, D.C. Bank. Anani, Kofi. 1999. "Sustainable Governance of Blair, Harry. 2003. "Civil Society and Pro-Poor Livelihoods in Rural Africa: A Place-Based Re- Initiatives at the Local Level in Bangladesh." sponse to Globalism in Africa." Development Draft, February 28. Department for Interna- 42(2):57­63(7). tional Development, UK. Association 35(4): 216­24. accessed March 2005. Arrow, K., R. Solow, P.R. Portney, E.E. Leamer, R. ------. 2000. "Participation and Accountability Radner, and H. Schuman. 1993. "Report of at the Periphery: Democratic Local Gover- the NOAA Panel on Contingent Valuation." nance in Six Countries." World Development Federal Register 58: 4601­14. 28(1): 21­39. Baland, J-M, and J-P Platteau. 2002. "Collective Ac- Brock, K., A. Cornwall, and J. Gaventa. 2001. tion on The Commons: The Role of Inequal- "Power, Knowledge and Political Spaces in ity." In J-M Baland, P. Bardhan, and S. Bowles, the Framing of Poverty Policy." Institute of eds., Inequality, Collective Action and Envi- Development Studies Working Paper 143. ronmental Sustainability, papers presented Brighton, U.K.: IDS. accessed March Costa, Alberto, C.G. Conrad, P. Kottak, and 2005. Rosane M. Prado. 1997. "The Sociopolitical Candler, W., and N. Kumar. 1998. The Dairy Rev- Context of Participatory Development in olution: The Impact of Dairy Development Northeastern Brazil." Human Organization in India and the World Bank's Contribution. 56(2). Operations Evaluation Department Study Se- da Cunha, Paulo Vieira, and Maria Valeria Junho ries. Washington, D.C.: World Bank. Pena. 1997. "The Limits and Merits of Partic- Chabal, P,. and J.P. Daloz. 1999. Africa Works: Dis- ipation." World Bank Policy Research Working order as Political Instrument. Oxford: James Paper 1838. Washington, D.C. Currey. Dahl-Ostergaard, T., D. Moore, V. Ramirez, M. Chambers, R. 1995. "Paradigm Shifts and the Wenner, and A. Bonde. 2003. "Community-Dri- Practice of Participatory Research and Devel- ven Rural Development: What Have We opment." In N. Nelson and S. Wright, eds., Learned?" Inter-American Development Bank Power and Participatory Development The- Sustainable Development Department Tech- ory and Practice. London: Intermediate Tech- nical Paper Series. Washington, D.C. nology. accessed March 2005. Questioning Participatory Approaches to De- Das Gupta, M., H. Grandvoinnet, and M. Ro- velopment." Journal of International De- mani. 2000. "State-Community Synergies in velopment 11(4): 597­612. Development: Laying the Basis for Collective ------. 1998. "There's a Right Way to Do It ­ Action." World Bank Policy Research Work- Informal Arrangements for Local Resource ing Paper 2439. Washington, D.C. Management in Zimbabwe." Waterlines 16(4). de Clercq, Francine. 1994. "Putting Community ------. 1997. "Gendered Incentives and Infor- Participation into Development Work: The mal Institutions: Women, Men and the Man- Difficult Case of Winterveld." Development agement of Water." Discussion Paper Series 2, South Africa 11(3). No. 3, Development and Project Planning Desai, V. 1996. "Access to Power and Participa- Center. University of Bradford, U.K. tion." Third World Planning Review 18(2): Coady, David, Margaret Grosh, and John Hod- 217­42. dinott. 2004. Targeting of Transfers in De- Development Committee. 2004. "Global Moni- veloping Countries Review of Lessons and toring Report 2004. Policies and Actions for Experience: Regional and Sectoral Studies. Achieving the MDGs and Related Outcomes." Washington, D.C.: World Bank and IFPRI. DC2004-0006. Joint Ministerial Committee of Collinson, M.P. 1982. Farming Systems Research the Boards of Governors of the Bank and the in Eastern Africa: The Experience of CIMMYT Fund on the Transfer of Real Resources to and Some National Agricultural Research Developing Countries. Washington, D.C. Services, 1976­81. International Wheat and Devereux, Stephen, and Kay Sharp. Forthcoming. Maize Improvement Center. Nairobi: CIMMYT. "Trends in Poverty and Destitution in Wollo, Conning, J., and M. Kevane. 2002. "Community- Ethiopia. Journal of Development Studies. Based Targeting Mechanisms for Social Safety Elbers, Chris, Peter F. Lanjouw, Johan A. Mistiaen, Nets: A Critical Review." World Development Berk Ozler, and Ken Simler. 2004. "On the 30(3): 375­94 Unequal Inequality of Poor Communities." Cornwall, A. 2003. "Whose Voices? Whose The World Bank Economic Review 18(3). Choices? Reflections on Gender and Partici- Einhorn, Jessica. 2001. "The World Bank's Mis- patory Development." World Development sion Creep." Foreign Affairs 80(5): 22. 31(8):1325­42. Estrella, Marisol, and John Gaventa. 1998. "Who ------. 2002. "Making Spaces, Changing Places: Counts Reality? Participatory Monitoring and Situating Participation in Development." IDS Evaluation A Literature Review." IDS Working Working Paper 170. Brighton, U.K. Paper 70. Brighton, U.K. 1 9 2 REFERENCES Forster, R. 1998. "GTZ's Experience with Main- Based Water Services: Evidence from Sri Lanka streaming Primary Stakeholder Participation." and India." Economic Development and Cul- Paper presented at the International Confer- tural Change 50(3): 667­91. ence on Mainstreaming and Up-Scaling of Pri- Isham, Jonathan, Deepa Narayan, and Lant mary Stakeholder Participation--Lessons Pritchett. 1995. "Does Participation Improve Learned and Ways Forward. Washington D.C., Performance? Establishing Causality with Sub- November 19­20, 1998. jective Data." The World Bank Economic Garcia, X.M., and S-A. Way. 2003. "Winning Review 9(2):175­200. Spaces: Participatory Methodologies in Rural Jenkins, R., and A.M. Goetz. 1999. "Accounts and Processes in Mexico." IDS Working Paper 180. Accountability: Theoretical Implications of Brighton, U.K. the Right-to-Information Movement in India." Gibson, C., and S.A. Marks. 1995. "Transforming Third World Quarterly 20(3):603­22. Rural Hunters into Conservationists: An Jones, Emma, and SPEECH. 2001. " `Of Other Assessment of Community-Based Wildlife Spaces' Situating Participatory Practices: A Management Programs in Africa." World De- Case Study from South India." IDS Working velopment 23(6):941­57. Paper 137. Brighton Sussex, U.K. Greiner, L.E. 1972. "Evolution and Revolution Kähkönen, S. 1999. "Does Social Capital Matter as Organizations Grow." Harvard Business in Water and Sanitation Delivery? A Review of Review 50 (July-August): 37­46. Literature." World Bank Social Capital Initia- Grootaert, C. 1998. "Social Capital: The Missing tive Working Paper 9. Washington, D.C. Link?" World Bank Social Capital Initiative Kent, B.R., and J. Rimarachin. 1994. "Rural Pub- Working Paper 3. Washington, D.C. lic Works Construction in the Andes of North- Gueye, B. 1999. "Whither Participation? Experi- ern Peru." Third World Planning Review ence from Francophone West Africa." Dry- 16(4):358­74. lands Issue Paper 87. International Institute Kerr, J., with G. Pangare, V.L. Pangare, and P. for Environment and Development. Dryland George. 2000. "An Evaluation of Dryland Wa- Program accessed March Discussion Paper 68. Washington, D.C.: IFPRI. 2005. Khwaja, A.I. 2003. "A Note on Measuring Em- Gugerty, Mary Kay, and Michael Kremer. 2000. powerment--An Economist's Perspective." "Outside Funding of Community Organiza- Kennedy School of Government Harvard Uni- tions: Benefiting or Displacing the Poor?" Na- versity. Paper presented at the Workshop on tional Bureau of Economic Research Working "Measuring Empowerment: Cross-Discipli- Paper 7896. Cambridge, MA. nary Perspectives," World Bank, Washington, Handy, C. 1985. Understanding Organizations. D.C., February 4­5, 2003. Harmondsworth: Penguin. ------. 2000. "Can Good Projects Succeed in Bad Hentschel, J. 1994. "Does Participation Cost the Communities? Collective Action in Public World Bank More? Emerging Evidence." World Good Provision" Harvard University. Paper Bank Human Resource Development and Op- presented at the Annual Bank Conference on erations Policy (HRO) Working Paper 31. Wash- Development Economics, May 2, 2001. ington, D.C. Kleemeier, Elizabeth. 2000. "The Impact of Par- IBGE (Instituto Brasileiro de Geografia e Es- ticipation on Sustainability: An Analysis of the tadística). 2003. Perfil dos Municípios Malawi Rural Piped Scheme Program. World Brasileiros: Gestão Publica 2001, Rio de Development 28(5): 929­44. Janeiro. accessed March 2005. Development and Change 33(2): 213­35. Isham, J., and S. Kähkönen. 2002. "Institutional Koopman, J., and R. Kweka, M. Mboya, and S.W. Determinants of the Impact of Community- Wangwe. 2001. "Community Participation in 1 9 3 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Traditional Irrigation Scheme Rehabilitation Review." OED Working Paper Series. Wash- Projects in Tanzania: Report of a Collaborative ington, D.C. Research Project." Ministry of Agriculture and Kumar, N., N. Saxena, Y. Alagh, and K. Mitra. Cooperatives, Irrigation Section, Dar es 2000. India: Alleviating Poverty through For- Salaam, Tanzania. est Development. Evaluation Country Case Korten, D.C. 1980. "Community Organization Study Series, Operations Evaluation Depart- and Rural Development: A Learning Process ment. Washington, D.C.: World Bank. Approach." Public Administration Review Kumar, Sanjay, and Stuart Corbridge. 2002. "Pro- 40(5): 480­511. grammed to Fail? Development Projects Krishna, A. 2004. "Partnership Between Elected and the Politics of Participation." The Journal Local Governments and Community-Based of Development Studies 39(2) 830­51. Organizations: Exploring the Scope for Syn- Leach, M., R. Mearns, and I. Scoones. 1999. "En- ergy." Social Development Papers, Community vironmental Entitlement: Dynamics and In- Driven Development, Number 52/February. stitutions in Community-Based Natural ------. 2003. "Falling into Poverty: The Other Resource Management." World Development Side of Poverty Reduction." Special Article, 27(2): 225­47. Economic and Political Weekly. Hitkari Leipziger, Danny, Marianne Fay, Quentin Woden, House, Mumbai, India. and Tito Yepes. 2003. "Achieving the Millen- ------. 2002. Active Social Capital. Tracing the nium Development Goals--A Multi-Sectoral Roots of Development and Democracy. New Approach Revisited." World Bank Policy York: Columbia University Press Research Working Paper 3163. Washington, ------. 2001. "Moving from the Stock of Social D.C. Capital to the Flow of Benefits: The Role of Linden van der, J. 1997, "On Popular Participa- Agency." World Development 29(6): 925­43. tion in a Culture of Patronage; Patrons and Krishna, A., N. Uphoff, and M. Esman (eds.). Grassroots Organization in a Sites and Services 1997. Reasons for Hope: Instructive Experi- Project in Hyderabad, Pakistan." Development ences in Rural Development. West Hartford, 9 (1): 81­90. CT: Kumarian. Lipton, Michael. n.d. "Rural Poverty Reduction: Krishna, A., M. Kapila, M. Porwal, and V. Singh. The Neglected Priority." Work at the Poverty 2003. "Falling into Poverty in a High-Growth Research Unit, Sussex University, for the In- State: Escaping Poverty and Becoming Poor in ternational Fund for Agriculture Develop- Gujarat Villages." Special Article, Economic ment's Report on Rural Poverty. and Political Weekly. Hitkari House, Mumbai, Manor, J. 1999. The Political Economy of De- India mocratic Decentralisation. Directions in De- Krishna, A., M. Kapila, S. Pathak, M. Porwal, K. velopment. Washington D.C.: World Bank. Singh, and V. Singh. 2004. "Falling into Poverty Mansuri, G., and V. Rao. 2004. "Community- in Villages of Andhra Pradesh: Why Poverty Based and -Driven Development: A Critical Re- Avoidance Policies are Needed." Special Arti- view." World Bank Research Observer 19(1): cle, Economic and Political Weekly. Hitkari 1­39. House, Mumbai, India Meinzen-Dick, R.S., K.V. Raju, and A. Gulati. 2002. Krishna, A., P. Kristjanson, M. Radeny, and W. "What Affects Organization and Collective Ac- Nindo. 2004. "Escaping Poverty and Becom- tion for Managing Resources? Evidence from ing Poor in 20 Kenyan Villages." Journal of Canal Irrigation Systems in India." World De- Human Development, 5(2): 211­26. velopment 30(4):649­66. ------. 2002. "The Challenges of Community Minot, N., and B. Baulch. 2004. "Poverty Mapping Participation in Forest Development in Nepal." with Aggregate Census Data." MSSD Discus- OED Working Paper Series. Washington, D.C.. sion Paper 49. Washington, D.C.: IFPRI. Kumar, N. 2003. "Community-Driven Develop- accessed March 2005. 1 9 4 REFERENCES Molyneux, M. 2002. "Gender and the Silences of ------. 2002c. The Next Ascent: An Evalua- Social Capital: Lessons from Latin America." tion of the Aga Khan Rural Support Pro- Development and Change 33(2):167­88. gram, Pakistan. Operations Evaluation De- Moore, Mick. 2001. "Empowerment at Last?" partment Study Series. Washington, D.C.: Journal of International Development World Bank. 13(3):321­29. ------. 2001. "The National Agricultural Re- Moore, Mick, and Anuradha Joshi. 1999. "Be- search Project, the National Agricultural Ex- tween Cant and Corporatism: Creating an En- tension Project, the National Livestock abling Environment for the Poor." IDS Bulletin Services Project, the Agricultural Diversifica- 30(4). Brighton, U.K. tion Project, and the Agricultural Sector In- Narayan, Deepa. 1995. The Contribution of vestment Project "(sector overview). Project People's Participation Evidence from 121 Performance Assessment Report. Report No. Rural Water Supply Projects. Environmen- 22439, Operations Evaluation Department. tally Sustainable Development Occasional Washington, D.C. Paper No. 1. Washington, D.C.: World Bank. ------. 1999a. Nongovernmental Organiza- Narayan, Deepa, and Lant Pritchett. 1997. "Cents tions in World Bank-Supported Projects. Op- and Sociability: Household Income and Social erations Evaluation Department. Report No. Capital in Rural Tanzania." World Bank Policy 19061. Washington, D.C.: World Bank. Research Working Paper 1796. Washington, ------. 1999b. Agricultural Extension Projects D.C. in Kenya--An Impact Evaluation. Report Nelson N., and S. Wright (eds.). 1995. Power No. 19523. Washington, D.C.: World Bank. and Participatory Development Theory and ------. 1998. The World Bank's Experience with Practice. London: Intermediate Technology. Post-Conflict Reconstruction. Operations Eval- North, D.C. 1991. Institutions, Institutional uation Department Study Series. Washing- Change and Economic Performance. Politi- ton, D.C.: World Bank. cal Economy of Institutions and Decisions. Ostrom, E. 1999. "Social Capital: A Fad or a Fun- Cambridge, U.K.: Cambridge University Press. damental Concept?" In P. Dasgupta and I. Ser- Oakley, Peter, and others. 1991. Projects with ageldin, eds., Social Capital: A Multifaceted People: The Practice of Participation in Rural Perspective. Washington, D.C.: World Bank. Development. Geneva: ILO. Overseas Development Institute. 1996. "The Im- OED (Operations Evaluation Department, World pact of NGO Development Projects." Briefing Bank). 2004a. Monitoring and Evaluation: Paper 2/96. accessed March ington, D.C.: World Bank. 2005. ------. 2004b. Books, Building and Learning Pantoja, E. 2000. "Exploring the Concept of Social Outcomes: An Impact Evaluation of World Capital and its Relevance for Community-Based Bank Support to Basic Education in Ghana. Development: The Case of Coal Mining Areas Operations Evaluation Department Study Se- in Orissa, India." World Bank Social Capital Ini- ries. Washington, D.C.: World Bank. tiative Working Paper 18. Washington D.C. ------. 2003c. An OED Review of Social Devel- Picciotto, Robert. 1992. "Participatory Develop- opment in Bank Activities. Operations Eval- ment Myths and Dilemmas." Participatory De- uation Department. Washington, D.C. velopment Working Paper 930. Washington ------. 2002a. IDA's Partnership for Poverty D.C.: World Bank. Reduction: An Independent Evaluation of Pimbert, M., V. Bainbridge, S. Foerster, K. Pasteur, Fiscal Years 1994­2000. Operations Evaluation G. Pratt, and I.Y. Arroyo. 2000. "Transform- Department. Washington, D.C.: World Bank. ing Bureaucracies: Institutionalizing Partici- ------. 2002b. Social Funds: Assessing Effec- pation and People Centered Processes in tiveness. Operations Evaluation Department Natural Resource Management: An Annotated Study Series. Washington, D.C.: World Bank. Bibliography." London: IIED. Economic and Political Weekly 31 (Special accessed March 2005. Number):2479­86. Platteau, Jean-Philippe. 2003. "Institutional and Renata, Elad L. and Jack E. Houston 2002. "Sea- Distributional Aspects of Sustainability in sonal Labor Constraints and Intra-household Community-Driven Development." Paper pre- Dynamics in the Female Fields of Southern pared for the Fifth Biennial World Bank Con- Cameroon." Agricultural Economics 27:23­32. ference on Evaluation and Development, Ribot, J.C. 2001. "Integral Local Development: `Ac- Evaluating Development Effectiveness: Chal- commodating Multiple Interests' through En- lenges and the Way Forward. Washington D.C., trustment and Accountable Representation." July 5­16, 2003. International Journal of Agricultural Re- Platteau, J-P, and F. Gaspart. 2003. "The Risk of sources, Governance and Ecology 1(3/4): Resource Misappropriation in Community- 327­50. Driven Development." World Development ------. 1998. "Decentralization, Participation 31(10):1687­1703. and Accountability in Sahelian Forestry: Legal Pretty, Jules. 1998. "Sustainable Agricultural In- Instruments of Political-Administrative Con- tensification: Farmer Participation, Social Cap- trol." Institute of International Studies. Berke- ital and Technology Design." Paper Prepared ley Workshop on Environmental Politics. for the World Bank's Rural Week, Washington. Working Paper WP00-5-Ribot. ac- Pretty, Jules, and Hugh Ward. 2001. "Social Cap- cessed March 2005. ital and the Environment." World Develop- ------. 1995. "From Exclusion to Participation: ment 29(2) 209-227. Turning Senegal's Forestry Policy around?" Pritchett, Lant, and Michael Woolcock. 2004. World Development 23(9):1587­99. "Solutions When the Solution is the Problem: Romeo, Leonardo G. 2003. "The Role of Exter- Arraying the Disarray in Development." World nal Assistance in Supporting Decentralization Development 32(2): 191­212. Reform." Public Administration and Devel- Pronk, Jan P. 2003. "Aid as a Catalyst: A Rejoinder." opment 23: 89­96. Development and Change 34(3):383­400. SAPROS & IFAD. 2002. Nepal--A Study of Rural Rajasekhar, D., and Veerashekhararappa. 2004. Hill Potentials and Service Delivery Systems. "Role of Local Organizations in Water Supply Sapros:Kathmandu, Nepal. and Sanitation Sector." A Study in Karnataka SAPROS & World Bank. 2000. Delivery of Rural and Uttaranchal States, India. Institute for Development Services, A Comparative As- Social and Economic Change. Bangalore, sessment of Alternative Institutional Arrange- India. ments with a Focus on the Terai. Sapros: Rajasekhar, D., D.V. Gopalappa, and Madhushree Kathmandu, Nepal. Sekhar. 2004. "Role of Local Organizations in Schady, Norbert R. 2000. "Picking the Poor: In- Watershed Development." A Study in Kar- dicators for Geographic Targeting in Peru." nataka and Uttaranchal States, India. Insti- World Bank Policy Research Working Paper tute for Social and Economic Change. 2477. Washington, D.C. Bangalore, India. Schneider, H. 1999. "Participatory Governance for Rao, V., and A.M. Ibáñez. 2003. "The Social Im- Poverty Reduction." Journal of International pact of Social Funds in Jamaica: A Mixed- Development 11(4):521­34. Methods Analysis of Participation, Targeting, Schou, Arild. 2002. "Popular Participation in De- and Collective Action in Community-Driven centralised Service Delivery and Social Funds." Development." World Bank Policy Research NIBR Working Paper 124. Oslo. Working Paper 2970. Washington, D.C. Schroeder, Larry. 2000. "Social Funds and Ravallion, Martin, and Gaurav Datt. 1996. "India's Local Government: The Case of Malawi." Checkered History in the Fight against Public Administration and Development Poverty: Are There Lessons for the Future?" 20:423­38. 1 9 6 REFERENCES Schultz, Theodore W. 1964. Transforming Tra- Sri Lanka." World Development 28(11): ditional Agriculture. New Haven, CT: Yale 1875­90. University Press. Uphoff, N., M. Esman, and A. Krishna. 1998. Sen, Amartya. 1999. Development as Freedom. Reasons for Success. Learning from Instruc- New York: Alfred A Knopf. tive Experiences in Rural Development. Serrano, R. 1996. "Who Knows What's Best for Bloomfield, CT: Kumarian. the Poor? Demand-Driven Policies and Rural van de Walle, N., and T.A. Johnston. 1996. "Im- Poverty in Northeast Brazil." Thesis, MIT. proving Aid to Africa." Policy Essay No. 21. Shepherd, A. 1998. "Participatory Environmen- Baltimore: Overseas Development Council. tal Management: Contradiction of Process, van der Linden, J. 1997. "On Popular Participa- Project and Bureaucracy in the Himalayan tion in a Culture of Patronage; Patrons and Foothills." In J. Blackburn and J. Holland, Grassroots Organization in a Sites and Services eds., Who Changes? Institutionalizing Par- Project in Hyderabad, Pakistan." Environ- ticipation in Development. London: Inter- ment and Urbanization 9(1):81­90. mediate Technology. van Donge, Jankees. 2003. Nurtured from above Tendler, Judith. 2000. "Why Are Social Funds So and Growing from the Roots: Social Funds Popular?" In Shahid Yusuf, Simon Evenett, and Decentralization in Zambia and and Weiping Wu, eds., Local Dynamics in an Malawi. The Hague: Institute of Social Stud- Era of Globalization: 21st Century Catalysts ies. versity Press. accessed March 2005. ------. 1997. Good Government in the Tropics. van Zyl, J., L. Sonn, and A. Costa. 2000. "Decen- Baltimore and London: The John Hopkins tralized Rural Development, Enhanced Com- University Press. munity Participation and Local Government ------. 1989. "What Ever Happened to Poverty Performance: Evidence from the Northeast Alleviation?" World Development 17(7): Brazil." Unpublished paper. 1033­44. Vyasulu, P., and V. Vyasulu. 1999. "Women in Pan- Thompson, John. 1995. "Participatory Ap- chayati Raj: Grassroots Democracy in India. Ex- proaches in Government Bureaucracies: Fa- perience from Malgudi." Paper presented cilitating the Process of Institutional Change." at the Meeting on Women and Political Par- World Development 23(9):1521­54. ticipation: 21st Century Challenges, UNDP, Tripp, Aili Mari. 2001. "Non-Formal Institutions, New Delhi, March 24­26, 1999. http://mag- Informal Economics, and the Politics of net.undp.org/events/gender/india/VYA- Inclusion." Discussion Paper No. 2001/108. SULU3.htm World Institute for Development Economics Wagel, S., and P. Shah. 2003. "Case Study 2-- Research/United Nations University. New Porto Alegre, Brazil: Participatory Approaches York. in Budgeting and Public Expenditure Turton, C., and J. Farrington. 1998. "Enhancing Management." Social Development Note 71, Rural Livelihoods through Participatory Wa- World Bank. accessed March source Perspectives 34, London: ODI. 2005. UNDP (United Nations Development Program). Walter, Ian, Rafael Del Cid, Fidel Ordoñez, and 2004. Human Development Report 2004: Cul- Florencia Rodríguez. 1999. "Ex-Post Evaluation tural Liberty in Today's Diverse World. of the Honduran Social Investment Fund." Report for the World Bank. accessed ductivity of Farmer Organizations in Gal Oya, March 2005. 1 9 7 THE EFFECTIVENESS OF WORLD BANK SUPPORT FOR COMMUNITY-BASED AND -DRIVEN DEVELOPMENT Wassenich, Paul, and Katherine Whiteside. 2004. IDA/R2003-0199. Washington, D.C. accessed Development Papers Number 51, February. March 2005. Watt, Patrick 2001. "Community Support for ------. 2003b. PRSP Sourcebook. Washington, Basic Education in Sub-Saharan Africa." World D.C. Bank Africa Region Human Development ------. 2003c. "Review of Bank Supervision of Working Paper Series. Washington, D.C. CDD Operations in the EAP Region" (Draft). Weinberger, K., and J.P. Jutting. 2001. "Women's EAP, VPU. ac- the Uses and Abuses of Participation." De- cessed March 2005. velopment in Practice 6(1):6­15. ------. 2003d. "Evaluating Social Funds: A Cross- Wilks, Alex. 2001. "Overstretched and Under- Country Analysis of Community Investments." loved: World Bank Faces Strategy Decisions." Regional and Sectoral Studies. Washington, Bretton Woods Project. Washington, D.C. D.C. Williams, G. 2003. "Toward a Re-politicisation of ------. 2003e. "Community-Driven Recon- Development: Political Capabilities and Space struction as an Instrument in War-to-Peace of Empowerment." Paper presented at the Transitions." CPR Working Paper No. 7, August. conference Participation: From Tyranny to Social Development Department ESSD Net- Transition? Exploring New Approaches to Par- work. accessed March 2005. ing Projects: Emerging Approaches in Africa." ------. 2002a. "A Tale of Two Projects: Commu- Social Development Notes No. 86 (Novem- nity-Based Reconstruction in East Timor and ber). Washington, D.C. Rwanda." Social Funds Innovation Update ------. 2004b. "Case Study Summaries for Re- 2(4). Human Development Network-Social ducing Poverty Sustaining Growth, Scaling Protection Unit, Social Funds Thematic Group. Up Poverty Reduction." A Global Learning Washington, D.C. Process and Conference, Shanghai, May 25­27, ------. 2002b. Empowerment and Poverty Re- 2004. duction: A Sourcebook. Ethnic Dimensions of Empowerment and So- accessed March 2005. cial Inclusion. Measuring Empowerment and ------. 2002c. "Sleeping on our Own Mats: An Social Inclusion Study: Phase I Report, Social Introductory Guide to Community Based Development." Draft. Monitoring and Evaluation." Africa Region, ------. 2004d. World Development Report 2004:. Rural Development II. Washington, D.C. Making Services Work for Poor People. Wash- ------. 2001a. "Decentralization and Gover- ington, D.C. nance: Does Decentralization Improve Public ------. 2004e. "Community-Driven Development Service Delivery?" PREM Note Number 55. in Urban Upgrading." Social Development Washington, D.C. Notes No. 85 (February). Washington, D.C. ------. 2001b. "World Bank Strategic Frame- ------. 2003a. "The Role of Selectivity in En- work" January 24. Washington, D.C. accessed March ing the MDGs and Related Outcomes." Back- 2005. ground Paper, Development Committee (Joint ------. 2001c. "Empowering the Poor through Ministerial Committee of the Boards of Gov- Decentralization: Brazil Rural Poverty Allevi- ernors of the Bank and the Fund) on the ation Program." Social Development Note Transfer of Real Resources to Developing No. 51. Washington, D.C. Countries. Washington, D.C. ------. 2000. "The Community Driven De- Wunsch, James S. 2001. "Decentralization, Local velopment Approach in the Africa Region: Governance and `Recentralization' in Africa." A Vision of Poverty Reduction Through Em- Public Administration and Development 21: powerment." Washington, D.C. 277-88. ------. 1996. The World Bank Participation Yukio, Ikemoto. 2001. "Poverty Alleviation Poli- Sourcebook. Environmentally Sustainable De- cies and Ethnic Minority People in Vietnam." velopment Network. Washington, D.C. Institute of Oriental Culture, University of ------. 1995. "Designing Community Based De- Tokyo. accessed March ment Department Paper No. 007. Participation 2005. Series. Washington, D.C. World Bank and IMF 2004. "Global Monitoring Report 2004: Policies and Actions for Achiev- 1 9 9 OED PUBLICATIONS Study Series Addressing the Challenges of Globalization: An Independent Evaluation of the World Bank's Approach to Global Programs 2003 Annual Review of Development Effectiveness: The Effectiveness of Bank Support for Policy Reform 2004 Annual Review of Development Effectiveness: The Bank's Contributions to Poverty Reduction Agricultural Extension: The Kenya Experience Assisting Russia's Transition: An Unprecedented Challenge Bangladesh: Progress Through Partnership Brazil: Forging a Strategic Partnership for Results--An OED Evaluation of World Bank Assistance Bridging Troubled Waters: Assessing the World Bank Water Resources Strategy Capacity Building in Africa: An OED Evaluation of World Bank Support The CGIAR: An Independent Meta-Evaluation of the Consultative Group on International Agricultural Research The CIGAR at 31: An Independent Meta-Evaluation of the Consultative Group on International Agricultural Research Country Assistance Evaluation Retrospective: OED Self-Evaluation Debt Relief for the Poorest: An OED Review of the HIPC Initiative Developing Towns and Cities: Lessons from Brazil and the Philippines The Drive to Partnership: Aid Coordination and the World Bank Economies in Transition: An OED Evaluation of World Bank Assistance Evaluating a Decade of World Bank Gender Policy: 1990-99 The Effectiveness of World Bank Support for Community-Based and ­Driven Development: An OED Evaluation Evaluation of World Bank Assistance to Pacific Member Countries, 1992-2002 Financial Sector Reform: A Review of World Bank Assistance Financing the Global Benefits of Forests: The Bank's GEF Portfolio and the 1991 Forest Strategy and Its Implementation Fiscal Management in Adjustment Lending IDA's Partnership for Poverty Reduction Improving the Lives of the Poor Through Investment in Cities India: The Dairy Revolution Information Infrastructure: The World Bank Group's Experience Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector Jordan: Supporting Stable Development in a Challenging Region Lesotho: Development in a Challenging Environment Mainstreaming Gender in World Bank Lending: An Update Maintaining Momentum to 2015? An Impact Evaluation of Interventions to Improve Maternal and Child Health and Nutrition Outcomes in Bangladesh The Next Ascent: An Evaluation of the Aga Khan Rural Support Program, Pakistan Nongovernmental Organizations in World Bank­Supported Projects: A Review Poland Country Assistance Review: Partnership in a Transition Economy Poverty Reduction in the 1990s: An Evaluation of Strategy and Performance The Poverty Reduction Strategy Initiative: An Independent Evaluation of the World Bank's Support Through 2003 Power for Development: A Review of the World Bank Group's Experience with Private Participation in the Electricity Sector Promoting Environmental Sustainability in Development Putting Social Development to Work for the Poor: An OED Review of World Bank Activities Reforming Agriculture: The World Bank Goes to Market Sharing Knowledge: Innovations and Remaining Challenges Social Funds: Assessing Effectiveness Tunisia: Understanding Successful Socioeconomic Development Uganda: Policy, Participation, People The World Bank's Experience with Post-Conflict Reconstruction The World Bank's Forest Strategy: Striking the Right Balance Zambia Country Assistance Review: Turning an Economy Around Evaluation Country Case Series Bosnia and Herzegovina: Post-Conflict Reconstruction Brazil: Forests in the Balance: Challenges of Conservation with Development Cameroon: Forest Sector Development in a Difficult Political Economy China: From Afforestation to Poverty Alleviation and Natural Forest Management Costa Rica: Forest Strategy and the Evolution of Land Use El Salvador: Post-Conflict Reconstruction India: Alleviating Poverty through Forest Development Indonesia: The Challenges of World Bank Involvement in Forests The Poverty Reduction Strategy Initiative: Findings from 10 Country Case Studies of World Bank and IMF Support Uganda: Post-Conflict Reconstruction Proceedings Global Public Policies and Programs: Implications for Financing and Evaluation Lessons of Fiscal Adjustment Lesson from Urban Transport Evaluating the Gender Impact of World Bank Assistance Evaluation and Development: The Institutional Dimension (Transaction Publishers) Evaluation and Poverty Reduction Monitoring & Evaluation Capacity Development in Africa Public Sector Performance--The Critical Role of Evaluation Multilingual Editions Allègement de la dette pour les plus pauvres : Examen OED de l'initiative PPTE Appréciation de l'efficacité du développement : L'évaluation à la Banque mondiale et à la Société financière internationale Determinar la eficacia de las actividades de desarrollo : La evaluación en el Banco Mundial y la Corporación Financiera Internacional Côte d'Ivoire : Revue de l'aide de la Banque mondiale au pays Filipinas: Crisis y oportunidades Reconstruir a Economia de Moçambique : All OED evaluations are available, in whole or in part, in languages other than English. For our multilingual selection, please visit http://www.worldbank.org/oed THE WORLD BANK