63683 Evaluative Directions for the World Bank Group’s Safeguards and Sustainability Policies 15 Evaluative Directions for the World Bank Group’s Safeguards and Sustainability Policies Anis Dani, Ade Freeman, and Vinod Thomas Evaluation Brief 15 2011 The World Bank http://www.worldbank.org/ieg Washington, D.C. © 2011 Independent Evaluation Group Communications, Learning, and Strategy The World Bank 1818 H Street, NW Washington, DC 20433 http://ieg.worldbankgroup.org Email: ieg@worldbank.org Telephone: 202-458-4487 All rights reserved This Evaluation Brief is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank. The findings, interpretations, and conclusions expressed here do not necessarily reflect the views of the Executive Directors of the World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank or IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. IEG encourages the dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to ieg@ worldbank.org. ISBN-13: 978-1-60244-185-8 ISBN-10: 1-60244-185-5 Contents v Abbreviations vii Acknowledgments 1 Introduction 3 I. The Policy Framework 3 Thematic Coverage of the Policies 3 Implementation Procedures and Structures 5 II. Effectiveness of the Policy Framework 5 Supervision in the World Bank Portfolio 8 Supervision in the IFC Portfolio 9 Gaps in Social Safeguards 10 Divergence in Categorizing Risks across the World Bank Group 12 Objective Criteria for Consistency in Categorization 15 III. Emerging Challenges 15 Strengthening Country Ownership 16 Safeguards Coverage for Policy and Program Lending 17 Projects Where the Use of Proceeds Is Not Fully Identifiable at Appraisal 17 Costs and Benefits 20 Enhancing Organizational Effectiveness 21 From “Do-No-Harm” to “Do-Good” Approaches 23 IV. Conclusions 23 Follow-Up 25 Bibliography Abbreviations CESI Environment and Social Department Investment Support Group (IFC) DPL Development policy lending ESSD Environmentally and Socially Sustainable Development IEG Independent Evaluation Group IFC International Finance Corporation MIGA Multilateral Investment Guarantee Agency M&E Monitoring and Evaluation P4R Program for Results QACU Quality Assurance and Compliance Unit SDN Sustainable Development Network UCS Use of country systems XPSR Expanded Project Supervision Report v Acknowledgments This paper is derived from and builds on the 2010 Redwood for the analysis of use of country systems. report by the Independent Evaluation Group Hans-Martin Boehmer, Ken Chomitz, Cheryl Gray, Safeguards and Sustainability Policies in a Daniela Gressani, Monika Huppi, Martin Taylor- Changing World: An Independent Evaluation of Dormond, Stoyan Tenev, and Christine Wallich World Bank Group Experience. The authors thank provided critical oversight and valuable comments the management and staff of the Independent on the evaluation report. Gabriel J. Campbell, Kirk Evaluation Group for their generous support and Hamilton, Arthur Dennis Long, and H˚ kon Fonseca a contributions to the evaluation of the World Bank Nordang provided peer review comments on the Group’s 10-year experience (fiscal 1999–08), which report. This paper would not have been possible provides the empirical evidence for this paper. were it not for their respective roles. Nonetheless, In particular, the authors would like to acknowl- any gaps or inaccuracies in this paper are attribut- edge the contributions of the remaining team able to the authors only. members—Jouni Eeerikainen, Peter Freeman, Ramachandra Jammi, Andres Liebenthal, Stephan The authors also acknowledge the specific Wegner, and Richard Carlos Worden—for their vital support received from Jiro Tominaga in produc- role in the evaluation; Andaleeb Alam and Unur ing this paper, Heather Dittbrenner in editing Demberel for their specific contributions to the the paper for publication, and Aline Dukuze in cost-benefit analysis; and Kris Hallberg and John providing administrative support. vii Introduction T he financial and food crises are reminders of the growing challenges countries face in sustaining economic growth and lifting the living stan- dards of the poor. Equally, the emerging impacts of global warming on natural disasters and on agriculture are warning signs of the urgency to care for the environment and society. Recent global experience in the financial and environmental arenas demonstrates clearly the need to put in place and enforce regulatory frameworks that balance costs and benefits, both private and social. In this context, the crucial questions in the recent Lessons from the Public and evaluation by the Independent Evaluation Group Private Sectors (IEG) of the World Bank Group’s safeguards and sustainability policy framework (IEG 2010b) The World Bank Group is using two policy concern the effectiveness of the instrument in frameworks: the safeguards framework of the World mitigating adverse environmental and social Bank, largely for the public sector, and the perfor- impacts of development programs, and suggest- mance standards framework of IFC and MIGA for ing ways to improve the results. IEG’s evalua- the private sector. The two share similar objectives: tion covered projects approved from FY1999 to the Bank seeks “to avoid, mitigate, or minimize FY2008. During this period, social and environ- adverse environmental and social impacts of mental effects were significant in half of World projects…” and ensure that they are “environmen- Bank projects—1,402 with commitments of $109 tally sound and sustainable.” IFC seeks “to manage billion; 88 percent of projects financed by the social and environmental risks and impacts and to International Finance Corporation (IFC)—1,662 enhance development opportunities in its private with commitments of $35 billion; and 217 guaran- sector financing” (IEG 2008). tees by the Multilateral Investment Guarantee Agency (MIGA). The two policy frameworks have different strengths and weaknesses. The evaluation does The main thrust of the evaluation findings is that not call for a wholesale shift from the World the World Bank Group’s safeguards and sustain- Bank’s safeguards framework to IFC/MIGA perfor- ability policies have helped avoid or mitigate mance standards or from the latter to the former. large-scale social and environmental risks in But it recommends that the Bank Group adopt the projects it financed, but many projects strong features from each approach to improve with substantial environmental and social implementation, results, and benefits. impacts remain of concern primarily because of inadequate supervision and follow-up. Policy The Bank’s safeguards contain mandatory require- implementation must be improved to get better ments, with mitigation measures designed results going forward. before project approval. Accordingly, attention to 1 EVALUATION BRIEF 15 safeguards and performance standards was reason- of new lending across the World Bank Group), ably good during the appraisal of projects. The greater emphasis on developing client ownership weakness in practice has been the lack of adequate and systems are needed. Among private sector supervision and monitoring of outcomes, especially partners, ownership has improved with the in the case of medium-risk projects. Institutional introduction of the IFC and MIGA’s new perfor- incentives and organizational constraints need to mance standards approach. But these standards be addressed to ensure equal attention to project were introduced at IFC in 2006, and MIGA in appraisal and supervision. In contrast, the crucial 2007, and the impacts on social and environmen- weakness for IFC is that the oversight remains with tal outcomes of IFC’s and MIGA’s new policies its private sector partners without third-party verifi- are not yet known. More vitally, in the absence cation or adequate disclosure. of third-party monitoring to supplement the client partners reporting on the externalities for As the Bank has moved beyond traditional invest- they are the source, these impacts will not be ment projects (which constitute less than half independently verifiable. 2 I. The Policy Framework Thematic Coverage of the Policies B oth the private sector and the public sector under-invest, and some- times heavily, in mitigating environmental and social side effects of their actions. Since 1989, when the World Bank introduced Operational Directives for environmental assessment of Bank-financed projects, the World Bank has developed nine additional policies to guide countries and staff on separate environmental and social effects. In 1997 the Bank identified these 10 policies as its suite of safeguard policies and labeled them “do no harm” policies. These current safeguard policies consist of six environmental, two social, and two legal policies. When the safeguard policies were labeled “do no sustainability and eight performance standards harm” policies, the Bank’s senior management divided equally among social and environmen- made public commitments to enforce compli- tal standards (see table 1). MIGA followed suit ance with these mandatory requirements. This in 2007, but its experience is more limited than led to significant improvement in environmental that of IFC. For that reason, this paper focuses and social performance compared with the 1990s. primarily on the findings and lessons from the However, the Bank’s list of safeguard policies World Bank and IFC. was restricted to existing policies designed to mitigate adverse environmental and social Implementation Procedures impacts, effectively freezing policy development. and Structures The procedure for policy revisions, even small ones, has proved to be so cumbersome and The World Bank conducts a review of all invest- time consuming that there is great reluctance to ment loans to determine whether the project revise and improve the policies even when the triggers safeguards policies and to define the lessons of experience suggest that this would be scope of the Environmental Assessment or Social beneficial. Many other multilateral development Assessment to be undertaken, if needed. The banks initially based their own safeguard policies Bank classifies the proposed project into one of for public sector lending on those of the World four categories (A, B, C, and FI), depending on its Bank, although some have since customized and potential environmental impacts. expanded these policies. The government is responsible for the assess- The International Finance Corporation (IFC) ments required by the safeguard policies; the and Multilateral Investment Guarantee Agency Bank is responsible for reviewing the assess- (MIGA) also adopted these safeguard policies ments and consequent mitigation plans to ensure until 2006. IFC replaced the safeguards in 2006 compliance with its operational policies. The with a single policy on social and environmental Bank’s Legal Vice Presidency oversees the policies 3 EVALUATION BRIEF 15 addressing international waterways and disputed The World Bank’s Executive Board of Directors areas. The Sustainable Development Network established the Inspection Panel, a permanent Vice Presidency is responsible for oversight body reporting to the Board of Directors, to of all other safeguard policies and operational ensure accountability of the World Bank and support to task teams. The Quality Assurance investigate complaints against violations of its and Compliance Unit (QACU), housed within policies (World Bank 2009a). For IFC and MIGA the Operations Policy and Country Services Vice the Compliance Advisor and Ombudsman was Presidency, supports the networks and is respon- created in 1998, reporting to the World Bank sible for clearances and compliance with all the Group president as a mechanism for grievance safeguard policies. redress; grievances that cannot be resolved by the Ombudsman are investigated further through For projects financed by IFC, during the appraisal a compliance audit to determine if corrective process IFC identifies which performance standards actions are needed to ensure compliance with its are applicable to a project. During implementation, policies (World Bank 2009b). performance is monitored against those standards, as is compliance with applicable local, national, and Note international laws.1 1. Retrieved on March 18, 2011 from http:// go.worldbank.org/QL7ZYN48M0. Table 1: Comparison of World Bank Group Safeguards and Performance Standards IFC/MIGA Policy and Performance Standards on Social World Bank Safeguard Operational Policiesa and Environmental Sustainability (2006–07) Environmental Performance Standard 1: Social and Environmental and Social Assessment and Management System Environmental 4.01Environmental Assessment (1999) Performance Standard 6: Biodiversity Conversation and 4.04 Natural Habitats (2001) Sustainable Natural Resource Management 4.36 Forests (2002) Performance Standard 3: Pollution Prevention and Abatement 4.09 Pest Management (1998) Performance Standard 8: Cultural Heritage 4.11 Physical Cultural Resources (2006) 4.37 Safety of Dams (2001) Social 4.12 Involuntary Resettlement (2001) Performance Standard 5: Land Acquisition and Involuntary 4.10 Indigenous Peoples (2005) Resettlement Performance Standard 7: Indigenous Peoples Performance Standard 2: Labor and Working Conditions Performance Standard 4: Community Health, Safety and Security Legal 7.50 International Waterways (2001) 7.60 Disputed Areas (2001) Source: IEG. Note: IFC = International Finance Corporation; MIGA = Multilateral Investment Guarantee Agency. a. Except for Pest Management, all World Bank Operational Policies have accompanying Bank Procedures. Consultation and disclosure processes are integral to the World Bank Group safeguard and sustainability policies. 4 II. Effectiveness of the Policy Framework Supervision in the World Bank Portfolio A t the World Bank Group, the quality of preparation and attention to safeguards at appraisal was reasonably good—satisfactory for 85 percent of projects—despite some weaknesses and inconsistencies across the Bank Group in project categorization. Policies do emphasize up-front risk assessment; however, they lack adequate incentives and systems for su- pervision and monitoring and evaluation (M&E). Quality of supervision was assessed in terms of the The Independent Evaluation Group (IEG) was follow-up on the mitigation measures and action able to confirm satisfactory supervision in only plans prepared to address the safeguard policies two-thirds of the Bank’s portfolio; one-third had triggered by individual projects, the composi- unrealistic safeguards ratings and weak M&E. tion of the supervision team—especially with Implementation Completion Reports rarely reference to the deployment of staff or consul- provide information on environmental and tants with relevant skills—and the appropriate- social results. Supervision quality was better in ness and supporting evidence for the safeguard category-A projects, four-fifths of which were well ratings in the available Implementation Status supervised, but performance was low in category- and Results Reports and related aides memoirs. B projects and financial intermediary (category-FI) In the final instance, these results were compared projects (figure 1). Although this reflects better with the quality of M&E of safeguards relevant to attention to high-risk projects, it does not follow the project. that all category-A projects have to follow the Figure 1: Supervision of Safeguards in World Bank-Financed Projects by Safeguard Categories Total FI Category B A 0 20 40 60 80 100 Percent of projects satisfactory Effectiveness of safeguards M&E Appropriateness of ISR safeguards ratings Quality of Bank supervision Source: IEG portfolio review, FY99–08 approvals. 5 EVALUATION BRIEF 15 Box 1: Adaptive Management on Safeguards in Project Restructuring A World Bank-financed power project in Asia initially adopted remaining two components, which were working successfully. a sectorwide approach and triggered eight safeguard policies When the largest component was dropped, the World Bank at appraisal. The project was restructured midway when it and the client agreed that only three safeguard policies (envi- became apparent that the largest component—attracting pri- ronment, resettlement, and natural habitats) were applicable, vate investments for subprojects in the sector—was no longer effectively restructuring the safeguard design. This was a good viable because of conflict in the country. Project resources that example of adaptive management by Bank staff working with had been intended for that component were reallocated to the the client in a fluid political and security context. Source: IEG 2010b. safeguards design approved at appraisal. Some Second, Bank projects that rely on environ- projects have done an excellent job of adaptive mental and/or social policy frameworks during learning to modify the safeguards design when project appraisal are even less well supervised the project context changed (box 1). than projects that undertake an environmental or social assessment during appraisal (figure 2). Three key concerns emerge from supervision Third, the quality of supervision of safeguards deficiencies. First, projects with substantial impact across Regions is very uneven. (category-B) are not being adequately supervised and monitored. Most of these are delegated to Almost a third of World Bank projects now respective sectors in the interest of increasing rely on policy frameworks for projects with ownership and efficiency. This is having the multiple subprojects whose environmen- perverse effect of leaving the effects of safeguards tal and social impacts are not fully known or unsupervised in a large number of projects. assessed at appraisal. These include the projects Figure 2: Supervision Quality of Projects with Policy Frameworks versus Projects with Mitigation Plans 100 100 Projects with satisfactory supervision (%) 80 71 64 60 58 59 51 40 20 0 Environment Resettlement Indigenous people Framework Plan Source: IEG 2010. 6 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES Figure 3: Supervision of Safeguards in World Bank-Financed Projects Total SAR MNA LCR ECA EAP AFR 0 20 40 60 80 100 Percent of projects satisfactory (%) Effectiveness of safeguards M&E Appropriateness of ISR safeguards ratings Quality of Bank supervision Source: IEG portfolio review, FY99–08 approvals. Note: Regions: AFR = Africa; EAP = East Asia and Pacific; ECA = Europe and Central Asia; LCR = Latin America and the Caribbean; MNA = Middle East and North Africa; SAR = South Asia. ISR = Implementation Status and Results Report; M&E = monitoring and evaluation. implemented by financial intermediaries, many in other Regions, more than one-third of the of which rely on an Environmental and Social projects suffered from inadequate M&E. The Management Framework during appraisal on the weaknesses lie in lack of specificity of monitor- assumption that the financial intermediaries will ing indicators, underinvestment in a client’s be undertaking or commissioning environmen- monitoring capacity, and poor follow-up during tal and social assessments during implementa- supervision. Safeguard monitoring would be tion. IEG found projects with policy frameworks much more effective if safeguard indicators were in the portfolio review to be less well supervised integrated within the overall results framework of than those with full Environmental Assessments the project and if clients were to take the respon- or Resettlement Action Plans. If the Bank relies sibility for systematically collecting relevant data on policy frameworks during preparation, it to monitor safeguards indicators. This would needs to invest proportionately greater resources require much greater investment in strengthen- in supervising these projects to help the client ing client institutions and systems for M&E than implement them well. is currently the case. Environmental and social performance indicators ought to be integrated There are significant regional differences in and results reported in completion reports. safeguards performance (figure 3). East Asia and the Pacific, which is the best regional performer Too often, safeguards activities are considered on preparation and appraisal, was also the best an add-on and left to environmental and social performer on supervision quality, with most other specialists who are under-resourced and not well Regions lagging significantly behind. The Latin integrated into supervision teams. This is not America and the Caribbean and the Middle East simply a resource constraint. Matching skills to and North Africa Regions were found to have overly demand will require management attention and optimistic safeguard ratings from the evidence up-front commitment of staff and resources for presented in the supervision documentation. supervision and client capacity building as an integral part of work program planning, if this M&E was the weakest aspect of World Bank constraint is to be overcome. supervision. Except for resettlement monitoring, which was of high quality in the East Asia and The Bank needs to revise supervision arrange- Pacific and the South Asia Regions but weaker ments, aligning incentives, responsibility, and 7 EVALUATION BRIEF 15 accountability to ensure adequate supervision 2007, but environmental and social supervision and M&E. Staff incentives and predictability of quality is still below the real sector level (figure resources for supervision need to be improved 4), because of fewer staff resources devoted to for greater effectiveness. the financial intermediary sector.1 However, IFC has strengthened rules for project supervision Supervision in the IFC Portfolio and site visit efforts, and its overall knowledge gap has decreased from 12.5 percent in FY08 to IFC focuses on the quality and capacity of their 5.8 percent in FY10. client’s Environmental and Social Management System and implementation of the Environmen- IFC requires its clients to submit Annual Monitor- tal and Social Action Plan that is disclosed and ing Reports, which need further strengthening, agreed with the client upon appraisal as the means but clients’ annual reporting has been a challenge to achieve performance standards. IFC is putting for IFC’s supervision. The staff survey reveals greater emphasis than in the past on supervision that about 30 percent of investment officers and monitoring; nonetheless, supervision quality and environmental and social specialists felt the has been lower than appraisal quality. timeliness and quality of client monitoring was inadequate. Within IFC’s sustainability framework, Based on IEG’s validation of IFC’s Extended the clients’ Annual Monitoring Reports and IFC Project Supervision Reports (XPSR), the quality site visits are the main instruments for monitoring of environmental and social supervision after projects’ performance. Because clients’ first Annual FY07 for pre-performance standard projects has Monitoring Report is due six months after the first improved. IEG has evaluated IFC’s environmen- year of project approval, the post-performance tal and social quality since 2004 as a part of the standard portfolio review focused on projects XPSR validation program. IFC’s environmental that had been approved at least two years before. and social supervision quality for pre-perfor- Of the 28 random sample projects, including all mance standard projects has improved since pre-performance standard and post-performance Figure 4: IFC’s Environmental and Social Work Quality at Supervision for Pre-Performance Standard Projects 100 90 Satisfactory rate (%) 80 70 60 50 2004–06 2005–07 2006–08 2007–09 Fiscal year All projects Non-FI FI Sources: IEG’s Environmental and Social Reviews for FY04–09 XPSRs, 209 real sector (non-financial intermediary) and 139 financial intermediary projects. See IEG 2010b. Note: FI = financial intermediary. 8 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES standard real sector projects older than two years, but narrowed the relevance of social safeguards only 50 percent (14 of 28) provided IFC with to a much smaller segment of the portfolio. satisfactory Annual Monitoring Reports. In most such cases, IFC identified the deficient informa- Although the World Bank’s social safeguards help tion in the Annual Monitoring Report for correc- mitigate unintended consequences of projects, tion in the following year, but in many cases the their limited thematic coverage is problematic. deficiencies continued despite corrective actions Current Bank social safeguards do not provide by IFC, reflecting insufficient communication and adequate coverage of community impacts; labor frequency of IFC feedback, as well as poor client and working conditions; and health, safety, and intake of corrective requirements. security issues at the project level, provisions that are integral to IFC and MIGA performance The portfolio review also found that IFC had not standards. monitored the implementation of the Environ- mental and Social Action Plans in 21 percent of IFC’s performance standards have better thematic the projects older than two years. Because the coverage of social risks, including labor and plans are designed to remedy gaps in the client’s community impacts, which are also relevant to social and environmental management system the World Bank’s project portfolio. Social risks identified during appraisal, they represent a subsequently addressed by IFC and MIGA—includ- major part of the value IFC adds to the project. ing the performance standards on Labor and Without IFC monitoring of implementation, it Working Conditions, or the one on Community cannot be assumed that this value was added. Health, Safety, and Security—have not been integrated into the Bank’s safeguard policies. Nor IFC’s supervision quality overall is showing do other social risks, such as impacts on gender or signs of improvement. However, supervision other vulnerable groups2 or risks covered by the of financial intermediary projects, IFC’s listed World Bank Group’s Environmental Health and equities and trade finance, and agrobusiness Safety Guidelines,3 receive adequate attention by projects needs more attention. The weak link safeguards practitioners even in projects where in the chain is the self-reporting of environmen- these risks are relevant since they are not defined tal and social outcomes by IFC’s client partners, as safeguard risks. The narrow coverage of social who themselves are the sources of the potential safeguards in Bank projects, compared to IFC and negative externalities. Therefore, they must MIGA, leads to an underestimation of risks and, be subjected to independent verification— in some instances, to risk avoidance when the especially for higher-risk projects—to ensure safeguards are perceived as not addressing the accountability. Furthermore, IFC must enhance most relevant risks. disclosure as the Bank has done. The frequency with which the safeguards and Gaps in Social Safeguards performance standards are triggered by the lending portfolio gives some indication of the The safeguards suite has functioned as a prescrip- relevance of these policies to the portfolio. tive framework for existing social policies and a Safeguards data from the 10-year portfolio for restrictive framework excluding consideration the Bank and 3 years since the introduction of of other social risks that are routinely assessed performance standards at IFC are shown in the by other members of the World Bank Group. two charts in figure 5. The policies that are more The existence of an umbrella policy for Environ- frequently triggered at the World Bank and that mental Assessment provided an open-ended are common to both the World Bank and IFC are mandate for engaging with borrowers and clients triggered in roughly similar proportions. on the environmental agenda. By contrast, the restriction of social safeguards at the Bank to two Among IFC’s performance standards, the one policies focused attention on these two effects on the client’s Social and Environmental Assess- 9 EVALUATION BRIEF 15 Figure 5: Safeguards and Performance Standards in the World Bank Group Portfolio Safeguards triggered in World Bank projects, Performance standards triggered in IFC investments, FY1999–2008 FY2007–09 Policy Policy Disputed areas 1% Indigenous peoples (PS7) 2% Dam safety 6% Cultural heritage (PS8) 8% Forests 6% International Biodiversity (PS6) 14% waterways 6% Land acquisition/involuntary Pest resettlements (PA5) 23% management 8% Community health and Natural habitats 11% 44% safety (PS4) Physical cultural 12% Pollution prevention and resources abatement (PS3) 49% Indigenous 17% peoples Labor and working Involuntary conditions (PS2) 52% 30% Environmental and resettlement Environmental social assessment 53% 72% and management (PS1) assessment 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Percent of projects Percent of investment projects n = 506 Sources: World Bank internal database. Note: 1. World Bank data are based on results for all 2,056 investment projects in the portfolio approved in FY99–08. This excludes 439 Development Policy Loans and structural adjust- ment loans from the same portfolio; 6 percent of the structural adjustment loans approved prior to September 2004 triggered OP 4.01 (Environmental Assessment) and are reflected in this chart. Note 2. IFC figures are depicted as a proportion of the entire IFC portfolio, including FI and C category projects; however, the data are an underestimate, as they do not portray the performance standards triggered by financial intermediary subprojects during implementation. ment and Management System affects more than Moreover, in some regions, the application of half, whereas that on Involuntary Resettlement the Indigenous Peoples Policy has proved to is triggered by 23 percent and Biodiversity by 14 be problematic and contentious. A policy on percent of IFC’s portfolio, which is similar to that in community impacts may be more acceptable the World Bank’s portfolio. But IFC’s performance as a means of addressing impacts on vulnerable standards on Labor and Working Conditions, and communities, including impacts on indigenous Community Health, Safety, and Security apply peoples. The answer may not be an open-ended to about twice as many projects, and to more expansion of the scope of social safeguards, but than 90 percent of real sector projects. There are the World Bank will need to consider the lessons some differences between the priorities of public from IFC and other multilateral organizations for and private sector clients, but many investment determining the most relevant social safeguards projects in the World Bank resemble IFC’s real for World Bank-financed projects and programs. sector projects. There is no obvious reason to presume that community and labor impacts are Overall, the current social safeguard policies not relevant to the World Bank’s portfolio. appear to be more problematic than environ- mental policies because of the limited coverage The priority given to mitigation of existing of the social safeguards (Involuntary Resettle- safeguard policies effectively crowded out ment Policy, Indigenous Peoples Policy). The attention to other social impacts on local World Bank would need to address the gaps in communities, including gender impacts in social safeguards, consolidating additional social Bank-supported projects, as shown by a recent themes under one umbrella for efficiency gains. IEG evaluation of World Bank Group support for gender and development (2010a). The label Divergence in Categorizing Risks across safeguards has also created an artificial barrier the World Bank Group precluding attention to emerging themes such as climate change and occupational health and The evaluation estimates that World Bank Group safety under the safeguards framework. projects generally lead to substantial environ- 10 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES mental and social benefits beyond their costs. category-C dropped from 40 to 18 percent. The But when risks are underestimated, or when safeguards profile varies across regions because of communities are excluded from project benefits the divergent nature of client needs reflected in to avoid dealing with safeguard risks, environ- differences in the project portfolio. The East Asia mental and social costs significantly outweigh and the Pacific Region had the highest propor- the benefits. Project categorization affects the tion (23 percent) of category-A projects, driven quality and quantity of environmental studies, by infrastructure projects, whereas Latin America public consultations, reporting, and frequency of and the Caribbean had the lowest (4 percent). The supervision and signals the risks of irreversible Europe and Central Asia Region had the highest and unprecedented impacts to the public. proportion of category-FI projects (13 percent) and had relatively fewer category-A and -B projects. World Bank Group projects categorized as having high environmental or social risk (category-A) are In IFC’s portfolio, the share of category-A projects relatively better managed, but these projects are declined since introduction of the performance less than 10 percent of the portfolio. Financial standards, from 6 percent to about 3 percent of intermediary projects across the Bank Group projects, but remained unchanged in commit- and Bank projects categorized as medium risk ment amount. Category-B projects accounted for (category-B), which are more than 50 percent of 50 percent. The share of category-C declined from the portfolio, are less well supervised. 20 percent to 12 percent, and that of category- FI projects increased from 27 to 35 percent as Over the 10-year period covered by IEG’s evalua- a result of the shift of IFC’s business away from tion, an average of 9 percent of the World Bank’s project finance toward financial intermediary, project portfolio was classified as category-A (very corporate equity, and trade finance projects high impact), 44 percent as category-B (substantial (figure 7). MIGA’s portfolio composition has impact), 29 percent as category-C (low impact), also shifted over time, with a significant increase and 4 percent as category-FI,4 but the distribution in the share of guarantees for financial sector changed substantially over time (figure 6). During projects during the past decade. this period, the proportion of category-A projects increased from 5 to 11 percent, and category- Categorization of projects based on environ- B increased from 37 to 51 percent; in contrast, mental and social risks differs across the World Figure 6: World Bank Investment Lending by Safeguard Category 100 100 Percentage of investment commitments 80 80 Percentage of projects 60 60 40 40 20 20 0 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Fiscal year Category-A Category-B Category-C FI – No. of projects A – Commitments Source: World Bank internal database, as of April 12, 2010. 11 EVALUATION BRIEF 15 Figure 7: Trends in IFC’s Portfolio 60 50 40 Percent 30 20 10 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Committment year (FY) A (by volume) B (by volume) FI (by volume) C (by volume) A (by number) B (by number) FI (by number) C (by number) Source: IFC’s MIS database. Bank Group and is not based on objective criteria social outcomes. Although the categorization of to assess risks. Importantly, several high-risk these projects appears to have been in compli- category-B projects (substantial impact) financed ance with IFC’s procedures, the World Bank by IFC would have likely been categorized as would likely have classified them otherwise, category-A (very high impact) projects using pointing to a lack of consistency of safeguards the Bank’s screening system. In the evaluation’s implementation across the World Bank Group. judgment, this difference affects 27 percent (10 of 37) of the category-B projects in the sample. Objective Criteria for Consistency In five cases that involved the construction of in Categorization new infrastructure or greenfield facilities, the scale of the impacts would have led the World The lack of clear guidance and objective criteria to Bank to classify the projects as category-A. In screen projects also affects the quality and consis- six additional cases, the sensitive nature of the tency of categorization of World Bank projects. impacts, associated as they were with hazard- The portfolio review revealed inconsistencies ous waste, indigenous peoples, natural habitats, in categorization, with a tendency toward risk or cultural resources, would have likely led the avoidance by over-categorization when impacts World Bank to classify them as category-A. Also, were not known at appraisal (IEG 2010b, p. 19). To projects that IFC categorizes as C may have large examine the rigor and consistency of the Bank’s environmental impacts, as illustrated by some categorization system, IEG developed a risks and IFC projects that are currently under Compli- benefits model (box 2) to rate the environmen- ance Advisor and Ombudsman review. Some tal and social risks of each project on a four-point trade finance, agribusiness projects with supply scale along four parameters—magnitude, chain risks and large power and industrial plant intensity, duration, and sensitivity of expected projects pose similar issues of miscategorization impacts—using transparent criteria to assess (IEG 2010b, pp. 22–23). each project (table 2). Data for estimating risks were obtained from IEG’s review of appraisal and Categorization in principle would be a major supervision documentation for all 102 completed determinant of the eventual environmental and Bank projects in the portfolio review. 12 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES Box 2: Risks and Benefits Model IEG developed an analytical model to rank the environmental and risks. For this purpose we rely on the rating for mitigating negative social risks of each project along four parameters—magnitude, impacts (MNI), the best documented performance indicator. This intensity, duration, and sensitivity—using transparent criteria rating from the portfolio review reflects IEG’s assessment of the to rank each project on the basis of data and documentation extent to which the risks identified at appraisal have been mitigated. obtained for the portfolio review. The model postulates risks (R) Each project’s success in mitigating negative impacts was rated as to be a function of magnitude (M), intensity (I), duration (D), and excellent (E = 1.00), satisfactory (S = 0.75), partially unsatisfactory sensitivity (S), with separate indicators for rating social risks (RS) (PU = 0.5), or unsatisfactory (U = 0.25). On this basis, a measure of and environmental (RE) risks along these four criteria. Data for the actual benefit (B) from the implementation of the World Bank estimating risks were obtained from IEG’s review of appraisal Group’s safeguards for a specific project is estimated as: and supervision documentation. The aggregate risk (R) is the sum of RS and RE, where B = MNI (RS + RE). RS = log(MS + IS + DS + SS) and RE = log(ME + IE +DE +SE). Although B is only an ordinal indicator of the benefits of safe- guards implementation, it can be appropriately used to compare The risk model provides a modality to impute value to benefits benefits against costs to analyze allocative efficiency and cost by weighting the environmental and social outcomes of World Bank effectiveness of World Bank Group and client resources expended Group projects by the significance of environmental and social on meeting safeguards and sustainability objectives. Table 2: Indicators for Estimating Social and Environmental Risks Risks High Substantial Moderate Low SOCIAL Magnitude >10,000 project-affected 1,001 10,000 project- 101 1,000 project- 100 (No. of project- persons or >1,000 displaced affected persons or up to affected persons or affected persons— 1,000 displaced up to 100 displaced displaced persons get 10 times the weight of other project-affected persons) Intensity Physical displacement Economic displacement Workplace safety Community impacts Duration Permanent (beyond the (Late project life) Midterm (Early project life) < 1 year or by effectiveness project’s closing date) review—closing date >1 year midterm review Sensitivity— Substantial risks to be Potential risks identified Projects mainstreaming Targeted IP projects Indigenous mitigated as per IP Plan in IP Framework benefits to IP Peoples (IP) ENVIRONMENTAL Magnitude— Global, regional, National or multi- State or provincial Localized area affected or transnational provincial Intensity Irreversible Severe Moderate Mild Duration >100 years >10–100 years >1–10 years <1 year (seasonal or intermittent) Sensitivity— Significant impact on Significant degradation Degradation other NH, Conservation and Natural Habitats critical NH of NH parks or reserves rehabilitation of NH (NH) Source: IEG 2010b, p. 67. 13 EVALUATION BRIEF 15 The risk assessment provided IEG with the There is an urgent need to issue clearer guidance means to examine the consistency between to promote the use of transparent criteria for objective environmental and social risk criteria categorization. There is an equally urgent need and safeguard categorization in Bank projects. to ensure consistency in categorization across A probit regression was carried out on the the World Bank Group. completed projects in the Bank’s portfolio review to test the effect of environmental and Notes social risk ratings, regional effect, and network 1. Five environmental and social specialists have effect on project categorization. IEG findings now been dedicated to financial intermediar- from the risk analysis indicate that categoriza- ies but this is less than 10 percent of the 57 tion is not always determined by the riskiness of dedicated to other sectors, although financial a project; neither is it based on use of objective intermediaries are a third of the portfolio. criteria to assess environmental and social risks. 2. These social impacts are currently covered by the The application of transparent risk indicators to Bank’s policy on Project Appraisal (OMS 2.20) the subset of completed projects, whose adverse and the policy on Gender and Development (OP impacts were known, provided evidence of the 4.20), which were excluded from the suite of weaknesses in the current practice. IEG found safeguard policies. See IEG 2010, pp. 8, 87–89. both errors of exclusion caused by underclassi- 3. The Environmental Health and Safety Guide- fication of category-B projects that should have lines have been prepared and issued by the been category-A and errors of inclusion from World Bank Group. over-classification of category-A projects that 4. The remaining 14 percent included lending op- should have been category-B (IEG 2010b, pp. erations for which safeguards were not consid- 68–70). ered relevant and were therefore unclassified. 14 III. Emerging Challenges Strengthening Country Ownership S afeguard policies were developed as “do no harm” instruments, with covenants used to enforce them. The perception created by enforce- ment of Bank policies, rather than development of and compliance with national legislation, often leads to lack of ownership for these measures at the country level. This situation then translates into weak compliance, weak supervision, and weak M&E. In 2004, the Bank piloted the use of country however, even in Brazil the Indigenous Peoples systems (UCS) for safeguards. The rationale was Policy has not been triggered. to scale up development impact by encouraging the use of improved systems for government Before agreeing to use country systems, the Bank expenditures to increase country ownership, assesses the strengths and weaknesses of borrower build institutional capacity, promote donor safeguards systems and identifies targeted—or harmonization, and increase cost effectiveness gap-filling—measures to strengthen such systems. for both the Bank and the borrower. Although The Bank has developed a tool for this analysis: the these objectives are still relevant, IEG’s evaluation Safeguards Diagnostic Review. This tool evaluates found that the country systems approach adopted the equivalence of the borrower’s system (the for safeguard policies was too self-limiting and extent to which it is designed to achieve the same not sufficiently robust and flexible for scaling up, objectives and adhere to the same principles as the and had lost ownership among Bank staff and Bank’s safeguard policies) and the acceptability of clients (IEG 2010b, pp. 85–87). borrower implementation practices, track record, and capacity. Measures to achieve and sustain The UCS approach was piloted initially in individ- equivalence and acceptability are identified, ual sectors and projects in six countries. This included in the legal agreement for the project, and was extended in a second phase to seven more then actively supervised during implementation. countries, including three corporate systems (in Brazil, India, and South Africa) and the first two The primary weakness of the UCS has been the country-level pilots in Croatia and Mauritius. decision to adopt a piecemeal approach, with These pilots focused primarily on environmental “country systems” being applied in practice to safeguards. Because of more significant differences individual projects, rather than to a country’s, with Bank policies and procedures, the piloting of or at least a sector’s, environmental and social Involuntary Resettlement and Indigenous Peoples management system; and to individual Bank safeguard policies was avoided entirely in the first safeguard policies, rather than to the Bank’s phase, either through project design or by simply safeguard policy suite as a whole. applying normal Bank safeguards procedures. In the second phase, involuntary resettlement Initial borrower ownership of the UCS pilot is being piloted in the parastatal corporations; scheme was mostly positive, but the IEG evalua- 15 EVALUATION BRIEF 15 tion found interest had dissipated. Participating full range of environmental safeguards. However, governments wanted to get away from the use of such separation of environmental from social dual systems and hoped that the UCS approach safeguards would throw into doubt the feasibil- could be extended to additional sectors and ity of combining all the safeguard policies under a projects. But recent experience revealed that the single umbrella. Nonetheless, significant revisions anticipated time and cost savings in the process- need to be made to the policy framework and ing of subsequent operations have not material- approach if the application of country systems to ized, as new Safeguards Diagnostic Reviews have safeguard policies is to be scaled up. been required for subsequent projects in the same country. Client feedback regarding the UCS pilots Safeguards Coverage for Policy and indicates that there is an inconsistency between Program Lending their expectations of the purpose of UCS and that presumed by the Bank. For example, client The World Bank’s portfolio has seen a rapid expectations that Bank safeguard responsibilities increase in types of lending to which safeguards would be transferred to the borrower did not and performance standards are not well suited. happen. Anticipation that UCSs could automati- Development policy lending (DPL) for institu- cally be applied to subsequent Bank-financed tional and policy reforms, programmatic or projects has likewise been frustrated. sectorwide lending, and lending through financial intermediary projects now comprises more than The benefits of the UCS pilots to clients thus half the portfolio. Figure 8 depicts the propor- remained unclear. Even China, which IEG’s tion of DPL lending in the Bank’s portfolio during evaluation confirms as having one of the best the review period. Even before the financial crisis records on safeguard policies, found the design led to a scaling up of DPL lending, it constituted too onerous and opted against participating in 20 percent of the portfolio. the UCS experiment. Financial intermediary lending constitutes another Within the Bank, enthusiasm for the UCS pilot four percent of the portfolio (6 percent of invest- had also waned. Country directors and sector ment lending). In addition, a significant propor- managers were concerned that UCSs increased tion of projects (likely to be another 20–30 percent the cost and time of project preparation and of the portfolio) supports the performance of a supervision, as well as increased reputational government program and institutions at country, risk. Environmental and social practitioners too sectorwide, or subnational levels. Recognizing the felt that the current costs of UCSs for safeguards inability to apply either the DPL policy (OP 8.60) or outweighed the potential benefits, despite some the current Investment Lending Policies effectively positive aspects of the initial pilots. to program lending, the Bank is developing a new Program-for-Results (P4R) lending instrument Efforts need to continue to develop greater to respond to the changing development needs country ownership, responsibility, and capacity and demand from borrowing countries. The to follow up on safeguards without diluting the safeguards policies too will need to be adapted to objectives of safeguard policies, but improving render them applicable to P4R lending. the results. There is a general consensus that the concepts underlying the use of country systems The application of safeguard policies to program- are sound, but the piecemeal application of the matic, sectorwide lending and financial interme- UCS approach to individual projects and safeguard diary projects is much more challenging, as they policies appears unworkable and needs a major do not involve discrete geographic areas where redesign to be successfully scaled up. National environmental and social effects can be readily systems can and should be used, where possible. assessed and mitigated. Development policy If they cannot be applied to all the safeguard operations have their own environmental and policies, they should at least be applied to the social requirements and are therefore excluded 16 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES Figure 8: World Bank Lending by Commitment Volume and Number of Projects 100 30,000 Commitment amount ($ millions) 25,000 80 Number of projects 20,000 60 15,000 40 10,000 20 5,000 0 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Fiscal year Investment – No. of projects DPL – No. of projects Investment – Commitments DPL – Commitments Source: World Bank internal database, as of April 12, 2010. Note: DPL = development policy lending. from the safeguard policies. Adaptation of the corporate or equity investments in companies with safeguard policies to the new P4R instrument is several production facilities and various activities yet another challenge. It is vital to seek consistency pose a substantial challenge for environmental among the approaches followed in these growing and social appraisal, supervision, and evaluation. segments of the portfolio to ensure coherence in MIGA’s portfolio composition has also shifted environmental and social sustainability outcomes. over time, with a significant increase in the share of guarantees for financial sector projects whose Projects Where the Use of Proceeds Is environmental and social effects are difficult to Not Fully Identifiable at Appraisal assess from 30 percent in fiscal years 2000–04 to 53 percent in fiscal years 2005–09. These shifts In financial intermediary and decentralized present a challenge for the World Bank Group to projects and in sectorwide and community-driven ensure continued relevance and effectiveness of development programs, the use of proceeds the safeguards and sustainability policies. is not fully identifiable at appraisal. The World Bank utilizes environmental and social policy Costs and Benefits frameworks to apply safeguards to these kinds of projects. About 30 percent of projects in the review One of the main constraints in assessing the sample used policy frameworks that, as discussed value added of the safeguards and performance above, were found to be less well supervised than standards is that costs and benefits are not system- other projects in the sample. Projects relying on atically tracked by the World Bank Group. Costs policy frameworks also had significantly weaker incurred by the World Bank are not distinctly implementation results both on environmental recorded, and client costs are available only for and social performance, compared with projects the projects with large impacts. IFC’s own costs that had an environmental or social assessment are more readily available, but costs incurred by and management plan (figure 9). IFC’s clients are not because they are considered proprietary. IEG’s evaluation estimated benefits IFC’s business has evolved in recent years from by extrapolating from the assessment of environ- project finance toward a growing portfolio of mental and social risks and comparing the results trade finance and equity investments which against available costs to analyze the efficiency exceed a third of the portfolio (figure 7). IFC’s of resource use. The assessment shows that 17 EVALUATION BRIEF 15 Figure 9: Mitigation of Adverse Impacts in Bank Projects, FY1999–2008 Supervision of Bank projects, FY1999–2008 Mitigation of adverse impacts in Bank projects FY1999–2008 Projects with satisfactory supervision (%) Projects with satisfactory mitigation (%) 100 100 100 91 80 80 75 71 64 58 59 59 61 60 60 56 56 51 40 40 20 20 0 0 Environment Resettlement Indigenous people Environment Resettlement Indigenous people Framework Plan Framework Plan Source: IEG portfolio review. the World Bank Group’s safeguards framework financing and 3 percent of total costs for the generates significant benefits in the mitigation of projects in the table 2 sample. World Bank clients environmental and social risks of projects, even tend to allocate resources efficiently in meeting as they need to be measured better. safeguards requirements, but results cannot be established for IFC partners because IFC does IEG assessed the cost effectiveness of the World not collect client cost data. Bank expenditure Bank Group’s approaches by plotting the data on on category-A projects was 8 percent, compared costs incurred by the World Bank Group distilled with 4 percent for category-B projects. painstakingly for separate samples of projects financed by the World Bank and IFC against a From a resource management perspective, a ranked scale of benefits estimated from a Risks simple test of the efficiency of the World Bank and Benefits Model constructed for the evaluation Group’s sustainability framework is whether the (box 2). Despite data limitations, these analyses costs incurred are allocated in proportion to the yielded valuable insights into the effectiveness of environmental and social risks of projects, and the World Bank Group’s approaches to managing whether they achieve the desired outcomes. environmental and social risks. Efficiency was assessed along a quadrant of the risk-adjusted benefit and costs, with the separation Costs incurred by World Bank clients on safeguards between high and low based on median values of are estimated at about 5 percent of World Bank benefits and costs for safeguards and performance Table 3: Average and Median Costs for Safeguards World Bank costs World Bank client costs IFC costs Environmental (n = 60) (n = 53) (n = 37) category Average Median Average Median Average Median A 72,412 51,061 19,230,200 8,357,000 254,450 60,264 B 45,675 22,876 5,168,489 4,031,200 24,654 12,195 Sample Total 59,766 38,700 13,544,300 5,920,000 62,953 19,062 Source: World Bank and IFC data. Note: World Bank data are based on 60 projects: 28 completed projects (22 category-A and 16 category-B), and 32 active projects (15 category-A, and 17 category-B). IFC data are based on 37 projects: 6 completed projects (all category-B); and 31 active projects (6 category-A and 25 category-B). While they are instructive in providing the relative proportion of safeguard costs, and in comparing costs of individual projects with risk-adjusted benefits, they should not be used to draw inferences for resource allocation over the entire life of projects. These cost tables include data from completed and active projects in the portfolio sample and provide an incomplete picture of full costs on safeguards/performance standards at closure. 18 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES Table 4: Distribution of Projects on Benefit-Cost standards. Table 4 shows the distribution of World Quadrant Bank Group performance along the benefit-cost quadrant. IFC’s spending on its sustainability World Bank IFC framework is being allocated somewhat more (percentage) (percentage) efficiently toward projects with higher risks and Benefit-cost quadrant (n = 35) (n = 36) benefits, but allocative efficiency is less evident in High benefit—low cost 26 19 World Bank spending on safeguards. High benefit—high cost 29 33 Low benefit—low cost 23 28 Analysis of the risk-adjusted benefits and World Low benefit—high cost 23 19 Bank costs on safeguards in the sampled projects Source: IEG risk analysis. for which cost data are available do not provide Note: These figures are based on costs incurred by the World Bank Group and exclude client costs. clear evidence of allocative efficiency, particularly for category-B projects, several of which incurred A stylized benefit-cost model of World Bank high costs. Although category-A projects indicate and IFC projects was used to illustrate the kind relatively better resource use, with higher costs of assessment and insights that could be drawn incurred on riskier projects (and correspond- with adequate quantitative data on benefits and ingly greater benefits), some high-cost category- costs of sustainability policies. On their own, A projects were also found to have much lower the stylized models showed that the benefits of benefits (figure 10A). safeguards outweighed their costs, compared with the without project situations.1 IFC’s allocation of supervision resources for environment, social, health and safety has been The benefit-cost ratio for social safeguards broadly aligned with risks, and the alignment derived from examples of World Bank projects has improved since introduction of the perfor- was found to be in the range of 0.8–1.3 in the case mance standards. The greatest costs have been of a transport project, and in the range of 1–3.5 in incurred on projects facing relatively higher the case of a wastewater treatment and sanitation risks and higher benefits (figure 10B). Projects project. However, these benefits are more muted implemented with performance standards also because of the narrow scope of the current social show more efficient allocation of resources than safeguards. In the IFC model, safeguards had a projects under the Safeguards Policy. positive payoff in every case. Estimated benefit-cost Figure 10: World Bank Group Costs and Benefits of Safeguards A. Benefits and World Bank costs B. Benefits and IFC costs 20 20 18 18 Risk-adjusted benefit Risk-adjusted benefit 16 16 14 14 12 12 10 10 8 8 6 6 4 4 2 2 0 0 0 05 100 150 200 250 25 50 75 100 145 310 990 World Bank safeguard costs ($ thousands) IFC budgetary cost for ESHS ($ thousands) Category-A Category-B Safeguards PS Source: IEG. Source: IEG; cost data from IFC. Note: ESHS = environmental, social, health, and safety; PS = performance standards. 19 EVALUATION BRIEF 15 ratios for social safeguards in a gold mine project for safeguard oversight while it ensures separation ranged from 1.5 for community health and safety of responsibilities to avoid conflicts of interest. to 6.5 for land acquisition and resettlement, and in a manufacturing and services project ranged from Over the past decade, several institutional reforms 7.7 for labor and working conditions to 8.2 for have been introduced to manage the World community health and safety. The model shows Bank Group’s safeguards agenda. These reforms that the potential of IFC projects is enhanced by have affected the incentives and accountabilities additional benefits derived from attention to labor of staff. The establishment of QACU ensured conditions and community impacts. greater attention to safeguards screening during the appraisal process. Since then, QACU and The stylized models illustrate that environmen- the Environmental and International Law Unit tal benefits also outweigh the costs of safeguards of the Legal Department have provided central implementation. In the IFC model, the benefit- guidance on all matters relating to safeguards. cost ratios for environmental effects from pollution prevention and abatement alone were In 2004, a system of delegation was introduced estimated to be 3.6 for the gold mine project and whereby the safeguard screening during project 9.9 for the manufacturing and services project. appraisal identified the high-risk projects that The absence of relevant project data, particularly would be monitored by QACU, which retains information on the nature and cost of externali- oversight responsibility for all category-A ties, posed significant challenges in the estima- projects and category-B and -FI projects with tion of the benefit-cost ratio for environmental potentially high reputational and social safeguard effects in the World Bank stylized model. risks. Responsibility for project processing and supervision of other projects, including some Although environmental and social performance with substantial or lower risk, is delegated to the has improved since the 1990s, supervision and appropriate sector units managing the invest- monitoring deficiencies constrain the World ment projects. Bank’s ability to evaluate safeguards results. Without a clear framework to assess the perfor- In 2006, the World Bank consolidated the Environ- mance and impacts of its safeguard policies, mentally and Socially Sustainable Development important gaps remain in managing the environ- Network (ESSD) and the Infrastructure Network— mental and social risks induced by Bank-financed into the Sustainable Development Network (SDN) projects. The stylized models for the World Bank under one vice president, bringing the environmen- and IFC demonstrate that with adequate data on tal and social staffs and their internal clients from costs, benefits, risk, and externalities incurred the infrastructure and agricultural sectors under during implementation, it is feasible to quanti- one umbrella. At the time of that merger, QACU tatively estimate the impacts of safeguards. and its counterparts—the Regional Safeguards Better monitoring, documentation, and report- Advisors—in the Regions, were transferred from ing of environmental and social effects would be ESSD to the Operations Services group to ensure invaluable to improve the insights obtained from that project clearances were not unduly influenced benefit-cost analysis of safeguards. by being housed within the same Network to offset the perception of conflict of interest. As under Enhancing Organizational Effectiveness the former ESSD network, QACU continues to rely largely on the technical staff in the regional Current safeguard and sustainability policies were environmental and social units to undertake the designed to address environmental and social technical work needed for appraisal. impacts at the project level. Policy implementa- tion is influenced by organizational arrangements Notionally, the regional environmental and social and incentives—the most relevant being report- units are also responsible for effective implemen- ing arrangements and budgetary arrangements— tation and supervision of safeguards. However, 20 EVALUATIVE DIRECTIONS FOR SAFEGUARDS AND SUSTAINABILITY POLICIES Figure 11: Organizational Arrangements at the World Bank and IFC Quality assurance and clearance units CESPQ Policy QACU & Regional and Quality Organizational boundaries WB-VPU Safeguard Advisors Assurance Unit boundary IFC World Bank Task team Environmental Project team CESI support leaders and socal sector leaders units units IFC Directorate boundary Budget authority for safeguard supervision Source: IEG. the budget for safeguards supervision in the and supervision lies with the CESI units, which World Bank is controlled by the task team leaders allows them to deploy resources across projects for each project, who determine the intensity where they are most needed. of supervision and choose the team members or consultants for safeguards supervision. The Structurally, IFC’s organizational arrangement concern for technical capacity of the task team resembles that of the World Bank prior to leader to make this judgment, or the conflict 2006, with the two functions—clearances and of interest, is not deemed relevant during the operational support—kept distinct from the supervision phase. As illustrated by the previous project proponents to avoid any pressure from analysis of costs and benefits, the reliance on an the project proponents. The budget authority internal market for safeguards supervision has and responsibility for supervision assigned to resulted in considerable inefficiencies in resource the CESI units has eliminated both the conflict of allocation for safeguards oversight at the World interest and the moral hazard of inflated demands Bank, compared to IFC. for environmental and social work, because CESI units have to manage this work from their A schematic comparison of the Bank’s organi- resource pool. Furthermore, responsibility now zational arrangements with IFC reveals some lies with units that are technically competent to functional similarities (figure 11), but there are address these issues. IFC’s challenges now lie in two crucial differences: the location of the organi- its interface with external clients. zational boundaries as firewalls to avoid conflicts of interest and budget authority for appraisal and The World Bank, however, faces both internal and supervision. The Policy and Quality Assurance Unit external challenges. To increase the effectiveness is responsible for clearances; two Central Environ- of safeguards supervision, the incentives arising mental and Social Investment Support Units out of current organizational and budgetary (CESI) provide technical support for appraisal arrangements will need to be addressed. and supervision; the project team leaders manage the projects: (1) the firewall keeps the project From “Do-No-Harm” to “Do-Good” proponents (project team leaders) separate from Approaches the unit providing clearance and the CESI units providing operational support; and (2) the budget An unintended consequence of the 2006 reorga- authority for environmental and social appraisal nization has been a growing separation between 21 EVALUATION BRIEF 15 the work on safeguards (overseen by QACU) environmental and social sector boards appears and the work on environmental sustainabil- to have exacerbated this divide. The effect of ity (overseen by the sector managers of the these tendencies is that across the Bank, most environmental and social units) in the SDN of the Bank staff who work on safeguards do not Anchor and the Regions. There is evidence of work on environmental and social sustainability, more careful screening of projects at entry and whereas those who work on sustainability no greater attention to category-A projects but longer work on safeguards. This is not an optimal more risk aversion reflected in an inflation of use of Bank resources and is in contrast to IFC and projects being classified as category-B. Staff have MIGA, where the mitigation agenda is an integral pointed to more centralized control and, in some part of social and environmental sustainability. instances, divergence in interpretation of policies and standards between the regions and centrally The Bank Group needs to expand further its based staffs. focus on issues such as biodiversity, climate change, and benefit-sharing to enhance social The artificial separation of environmental and impacts on the poor. To do this, it could social staff between those who work on safe- complement its compliance based policies— guards and those who work on social or environ- moving beyond the “do-no-harm” approach—to mental sustainability is a cause for concern. The encourage attention to enhancing environmen- merger of infrastructure sectors with the environ- tal and social results. mental and social development sectors under one vice presidency has given rise to a surge in demand Note for safeguards services, but the demand-driven 1. See IEG 2010, pages 78–80. For a fuller de- nature of the relationship between infrastructure scription of the stylized models see the on- task team leaders and environmental and social line Annex 7, Valuing Benefits and Costs of staff is forcing an unnecessary division of labor Safeguards (http://siteresources.worldbank. among the social and environmental staff. The org/EXTSAFANDSUS/Resources/Safeguards_ separation of QACU from the oversight of the annexes.pdf). 22 IV. Conclusions T his evaluation upholds the role the World Bank Group must continue to play in being at the cutting edge of promoting sustainable develop- ment. But it goes further in noting that the actual effectiveness of the regulatory regime depends not only on up-front risk assessments that the World Bank Group and countries carry out, but crucially also on effective implementation and supervision and on the checks and balances provided by M&E, disclosure of findings, and verification of results. The comparative analysis among the World Bank supervision reports. The evaluation has also Group has identified relative strengths of differ- made recommendations for the Bank Group to ent approaches. The World Bank’s categorization render support for client capacity; strengthen system and disclosure policy is more rigorous, supervision arrangements, implementation, and whereas the systems introduced recently by IFC accountability; and help ensure better environ- for monitoring and supervision are more system- mental and social results. atic, with clearer responsibilities for its client partners. The World Bank and IFC serve differ- The Bank Group needs to expand further its ent clients, yet they can strengthen the quality of focus on issues such as biodiversity and climate environmental and social results by adopting the change, and benefit-sharing to enhance social merits of each other’s systems. impacts on the poor. To do this, it could comple- ment its compliance based policies—moving The evaluation concludes that the Bank Group’s beyond the “do no harm” approach—to encour- environmental and social policies have been age attention to enhancing environmental and beneficial but need revision to strengthen the social results. focus on benefits, client institutions, and results, to keep pace with the Bank Group’s rapidly Follow-Up evolving portfolio. The Bank’s recent commit- ment to update and consolidate its environmen- The evaluation aimed to inform the updates tal and social safeguard policies over the next two of IFC’s sustainability framework, the World years is one step in the right direction. Bank’s environmental strategy, and the ongoing reform of World Bank investment lending. The As a crucial player in promoting better environ- Board Committee on Development Effective- mental and social outcomes worldwide, IEG ness endorsed a comprehensive update of the recommends that the Bank improve the thematic World Bank’s safeguard policies and harmoniza- coverage of its safeguard policies and address tion of project categorization across the World institutional impediments to effective manage- Bank Group. The Committee also reiterated ment of the environmental and social agenda. the importance of effective implementation of IFC needs to significantly enhance disclosure safeguard policies and strengthened supervi- of supervision information and introduce sion; stressed the value of checks and balances independent verification of its monitoring and provided by M&E, disclosure of findings, and 23 EVALUATION BRIEF 15 verification of results; and concurred with the social results will now be crucial in providing the need to strengthen client capacity and enhance promised gains from the reforms. responsibility and ownership. The Bank, IFC, and MIGA management agreed The update of IFC’s Policy and Performance to work together in fiscal year 2011 to ensure Standards on Social and Environmental Sustain- the adequacy, rigor, and consistency in project ability (2011) and the Access to Information categorization across the World Bank Group. This Policy draws on evaluation findings on project intended harmonization is yet to be achieved. categorization, financial intermediaries, and The update of the World Bank’s environmental supply chains. The introduction of subcategories strategy is in progress. for financial intermediary projects based on the risks, investment type, and use of proceeds is an In response to the recommendation for an important first step. This now requires articula- update of the safeguard policies, World Bank tion of explicit guidance on the environmental management committed to a review of global and social requirements for each subcategory in good practice—including a consultative process the update of Environmental and Social Review with diverse shareholders and stakeholders Procedure. IEG also supports the emphasis on over a 24-month period—that will integrate capacity building in the Implementation Action the update of safeguard policies. At the conclu- Plan to strengthen IFC’s and clients’ capacity and sion of this process, Bank management would strategic partnerships, and IEG endorses IFC’s report to the Board on how the Bank intends to plan to use more advisory services to support strengthen environmental and social sustainabil- clients’ implementation. Independent third- ity in projects, including the possibility of a more party monitoring of the environmental and social consolidated policy framework and strengthened results and disclosure of environmental and institutional arrangements. 24 Bibliography ADB (Asian Development Bank). 2009. “Safeguard IFC (International Finance Corporation). Forthcoming. Policy Statement.” Board Policy Paper. Manila. Update of Policy and Performance Standards on Social Bosshard, P. 2004. “The World Bank’s Safeguard and Environmental Sustainability. Washington, DC. Policies under Pressure: A Critique of the World ———. 2006. Performance Standards on Social and Bank’s New Middle Income Country Strategy.” Environmental Sustainability. Washington DC: International Rivers Network, http://www.inter World Bank. nationalrivers.org. IDB (Inter-American Development Bank). 2009. Cernea, M., and Davis, S. (eds.). 2000. Risks and Sustainability Report. Washington, DC: IDB. Reconstruction; Experiences of Resettlers and MIGA (Multilateral Investment Guarantee Agency). Refugees. Washington, DC: World Bank. 2007. Policy on Social and Environmental Clark, D., J. Fox, and K. Treakle. 2003. Demanding Sustainability. Washington, DC: World Bank. Accountability: Civil Society Claims and the World Multilateral Financial Institutions Working Group. Bank Inspection Panel. Lanham, MD: Rowman 2005. A Common Framework for Environmental and Littlefield. Assessment Good Practice Note, and Rome Declara- EBRD (European Bank for Reconstruction and tion on Harmonization; Multilateral Financial Development). 2008. Environmental and Social Institutions Working Group on the Environment. Policy. London: EBRD. http://www.aidharmonization.org. EIB (European Investment Bank). 2008. The EIB Picciotto, R., W. van Wicklin, and E. Rice (eds.). Statement of Environmental and Social Princi- 2001. Involuntary Resettlement: Comparative ples and Standards. Luxembourg: EIB. Perspectives. World Bank Series on Evaluation Hicks, R., B. Parks, J.T. Roberts, and M. Tierney. 2008. and Development, Volume 2. New Brunswick, NJ: Greening Aid? Understanding the Environmen- Transaction Publishers. tal Impact of Development Assistance. Oxford: World Bank. 2011. “A New Instrument to Advance Oxford University Press. Development Effectiveness: Program For Results IEG (Independent Evaluation Group). 2010a. Gender Lending—Revised Concept Note.” Investment and Development: An Evaluation of World Bank Lending Reform, Operations Policy and Country Support, 2002–08. Washington, DC: World Bank. Services Document. February 1. ———. 2010b. Safeguards and Sustainability Policies ———. 2009a. Accountability at the World Bank: The in a Changing World: An Independent Evaluation Inspection Panel at 15 Years. Washington, DC: of the World Bank Group Experience. Washing- World Bank. ton, DC: World Bank. http://www.worldbank ———. 2009b. Annual Report of the Compliance Advisor .org/ieg/safeguards. Ombudsman. Washington, DC: World Bank. ———. 2009. Achieving Sustainable Development: ———. 2009c. Investment Lending Reform Concept Annual Review of Development Effectiveness. Note. Board Report, World Bank, Washington, DC. Washington, DC: World Bank. ———. 2008. “Use of Country Systems in Bank- ———. 2008. Environmental Sustainability: An Supported Operations.” Board Paper R2008-0036/5. Evaluation of World Bank Group Support. Washington, DC. Washington, DC: World Bank. 25