76383 v3 IEG INPENDENT EVALUATION GROUP WORLD BANK · IFC · MIGA Volume III: Management Action Record Results and Performance of the World Bank Group 2012 Results and Performance of the World Bank Group 2012 Volume III: Management Action Record © 2013 Independent Evaluation Group 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: http://ieg.worldbankgroup.org Some rights reserved 1 2 3 4 5 15 14 13 This work is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. Note that IEG and the World Bank Group do not necessarily own each component of the content included in the work. IEG and the World Bank therefore do not warrant that the use of the content contained in the work will not infringe on the rights of third parties. The risk of claims resulting from such infringement rests solely with you. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of IEG, the World Bank Group, its Boards of Executive Directors, or the governments they represent. IEG and the World Bank Group do not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG or the World Bank Group concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation on or waiver of the privileges and immunities of IEG and the World Bank Group, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 3.0 Unported license (CC BY 3.0) http://creativecommons.org/licenses/by/3.0. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: IEG (Independent Evaluation Group). 2013. Results and Performance of the World Bank Group 2012—Volume II: Appendixes. Washington, DC: World Bank. License: doi: 10.1596/978-0-8213-9853-1. Creative Commons Attribution CC BY 3.0 Translations—If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by the Independent Evaluation Group or the World Bank Group and should not be considered an official IEG/World Bank Group translation. IEG and the World Bank Group shall not be liable for any content or error in this translation. All queries on rights and licenses should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Contents WORLD BANK MANAGEMENT ACTION RECORD 2012 ........................................................................... 1 1. Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population - An Evaluation off World Bank Group Support Since 1997 ...................................................................................................................................... 2 2. How Effective Have Poverty and Social Impact Analyses Been? An IEG Study of World Bank Support to PSIAs 43 3. The World Bank's Country Policy and Institutional Assessment (CPIA) ................................................................. 49 4. Gender and Development: An Evaluation of World Bank Support, 2002–08 .......................................................... 54 5. Water and Development: World Bank Support, 1997–2007 ................................................................................... 59 6. Poverty Reduction Support Credits: An Evaluation of World Bank Support MAR 2012 ......................................... 66 7. Safeguards and Sustainability Policies in a Changing World – An Independent Evaluation of World Bank Group Experience ...................................................................................................................................................................... 73 8. Phase II: The Challenge of Low-Carbon Development - Climate change and the World Bank .............................. 84 9. An Independent Assessment – The World Bank’s Involvement in Global and Regional Partnership Programs .... 94 10. Growth and Productivity in Agriculture and Agribusiness: Evaluative Lessons from the World Bank Group Experience .................................................................................................................................................................... 103 11. Social Safety Nets - An Evaluation of World Bank Support, 2000–2010 ...................................................... 115 12. Trust Fund Support for Development - An Evaluation of the World Bank’s Trust Fund Portfolio .................. 123 WORLD BANK IMPLEMENTATION REPORT 2012 ................................................................................ 129 13. Public Sector Reform: What Works and Why? – An Evaluation of World Bank Support .............................. 130 14. Doing Business: An Independent Evaluation – Taking the Measure of the World Bank-IFC Doing Business Indicators 138 15. The World Bank Group Guarantee Instruments 1990–2007 ......................................................................... 154 16. Decentralization in Client Countries: An Evaluation of World Bank Support 1990-2007............................... 161 17. Environmental Sustainability – An Evaluation of World Bank Group Support ............................................... 167 18. Climate Change and the World Bank Group Phase I—An Evaluation of World Bank Win-Win Energy Policy Reforms 178 19. Using Knowledge to Improve Development Effectiveness – An Evaluation of World Bank Economic and Sector Work and Technical Assistance, 2000-2006 ..................................................................................................... 188 MANAGEMENT ACTION RECORD IFC 2012 .......................................................................................... 195 1. PEP-ECA 2007 .................................................................................................................................................................. 196 2. Indonesia 2007 .................................................................................................................................................................. 201 3. IEDR 2008 ......................................................................................................................................................................... 203 4. Nigeria 2008 ...................................................................................................................................................................... 207 5. BROE 2008 ........................................................................................................................................................................ 210 6. IEDR 2009 ......................................................................................................................................................................... 213 7. HNP 2009 .......................................................................................................................................................................... 220 8. Peru CPE ........................................................................................................................................................................... 225 9. Safeguards ........................................................................................................................................................................ 230 10. Agribusiness .................................................................................................................................................................... 236 11. Climate Change............................................................................................................................................................... 244 2012 MIGA MANAGEMENT ACTION RECORD (MAR) ........................................................................... 247 Finalized April 14, 2012. IEG Contact: Bidjan Nashat (x.30779) ............................................................................................... 248 World Bank Management Action Record 2012 1 1. Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population - An Evaluation off World Bank Group Support Since 1997 LIST OF IEG RECOMMENDATIONS 1.1. Intensify efforts to improve the performance of the World Bank’s health, nutrition, and population support. a. Match project design to country context and capacity and reduce the complexity of projects in low-capacity settings through greater selectivity, prioritization, and sequencing of activities, particularly in Sub-Saharan Africa. b. Thoroughly and carefully assess the risks of proposed HNP support and strategies to mitigate them, particularly the political risks and the interests of different stakeholders, and how they will be addressed. c. Phase reforms to maximize the probability of success. d. Undertake thorough institutional analysis, including an assessment of alternatives, as an input into more realistic project design. e. Support intensified supervision in the field by the Bank and the borrower to ensure that civil works, equipment, and other outputs have been delivered as specified, are functioning, and are being maintained. 1.2. Renew the commitment to health, nutrition, and population outcomes among the poor. f. Boost population and family planning support in the form of analytic work, policy dialogue, and financing to high-fertility countries and countries with pockets of high fertility. g. Incorporate the poverty dimension into project objectives to increase accountability for health, nutrition, and population outcomes among the poor. h. Increase support to reduce malnutrition among the poor, whether originating in the HNP sector or other sectors. i. Monitor health, nutrition, and population outcomes among the poor, however defined. j. Bring the health and nutrition of the poor and the links between high fertility, poor health, and poverty back into poverty assessments in countries where this has been neglected. 1.3. Strengthen the World Bank Group’s ability to help countries to improve the efficiency of their health systems. k. Better define the efficiency objectives of its support and how efficiency improvements will be improved and monitored. l. Carefully assess decisions to finance additional earmarked communicable disease activities in countries where other donors are contributing large amounts of earmarked disease funding and additional funds could result in distortion in allocations and inefficiencies in the rest of the health system. m. Support improved health information systems and more frequent and vigorous evaluation of specific reforms or program innovations to provide timely information for improving efficiency and efficacy. 2 1.4. Enhance the contribution of support from other sectors to health, nutrition, and population outcomes. n. When the benefits are potentially great in relation to the marginal costs, incorporate health objectives into non-health projects, for which they are accountable. o. Improve the complementarity of investment operations in health and other sectors to achieve health, nutrition, and population outcomes, particularly between health and water supply and sanitation. p. Prioritize sectoral participation in multisectoral HNP projects according to the comparative advantages and institutional mandates, to reduce complexity. q. Identify new incentives for Bank staff to work cross-sectorally for improving HNP outcomes. r. Develop mechanisms to ensure that the implementation and results for small health components retrofitted into projects are properly documented and evaluated. 1.5. Implement the results agenda and improve governance by boosting investment in and incentives for evaluation. s. Create new incentives for monitoring and evaluation for both the Bank and the borrower linked to the project approval process and the midterm review. This would include requirements for baseline data, explicit evaluation designs for pilot activities in project appraisal documents, and periodic evaluation of main project activities as a management tool. LEVEL OF ADOPTION Level of Adoption Recommendations Mgmt IEG Intensify efforts to improve the performance of the World Substantial Substantial Bank‘s health, nutrition, and population support. Renew the commitment to health, nutrition, and population Substantial Substantial outcomes among the poor. Strengthen the World Bank Group‘s ability to help countries Substantial Substantial to improve the efficiency of their health systems. Enhance the contribution of support from other sectors to Medium Medium health, nutrition, and population outcomes. Implement the results agenda and improve governance by Substantial Substantial boosting investment in and incentives for evaluation. MANAGEMENT RESPONSE 2012 1.1.Intensify efforts to improve the performance of the World Bank’s health, nutrition, and population support. 3 Original Response: Management agrees that complexity can be at least partially addressed by adopting IEG recommendations, such as thorough technical preparation, including solid analytical underpinning, political mapping, high quality at entry, including prioritizing interventions relative to the institutional context, and establishing a good results framework, followed by in-depth supervision and parallel policy dialogue with client and partners. However, HNP operations are rarely institutionally or technically simple, since the desired outcome depends on a complex and interacting set of social, cultural, and institutional factors. This is recognized by donors and policy makers, whether in low-income, middle-income, or high- income countries. The inherent complexity of the sector may be attributable to the political economy in a multi-stakeholder environment, the need for extensive coordination and partnership with national and international agencies and civil society organizations, and the often difficult technical and social nature of the subject. The recent international recognition of the need to invest in (complex) health systems in order to ensure the success of vertical disease control programs in low-income countries is testimony to the fact that there are few easy ways to avoid systemic complexity. Investing in simple programs would not necessarily provide for lasting impact. Management agrees in principle to carry out political mapping exercises prior to investments in the sector where appropriate. As there are currently mandatory risk assessment and mitigation steps built into the project cycle and approval process, we anticipate improved risk mitigation strategies in newer HNP operations. These enable staff to identify major political and technical risks and devise with the client suitable risk-mitigation strategies. However, it is also evident that despite good assessments and risk-mitigation strategies, neither technical nor political risks can be completely offset. Management agrees to the recommendation to phase reforms to maximize the probability of success. Undertake thorough institutional analysis, including an assessment of alternatives, as an input into more realistic project design. Management agrees in principle with this recommendation. Supervision requirements, both in terms of staffing mix and budgeting, are being reassessed Bank-wide within the context of the ongoing review of investment lending. Given the inherent dispersed nature of many HNP investments, care must be taken during project design to ensure that the client assumes responsibility for ensuring that civil works, equipment, and other outputs have been delivered as specified, are functioning and being maintained, while the Bank audits/appraises/confirms that such monitoring is taking place so that detailed supervision of projects can be properly conducted within likely budget norms. Management action: In response to the portfolio quality challenges, the HNP Sector Board has introduced a quarterly portfolio monitoring and benchmarking system, which is being used by the Sector Board and regional management to improve portfolio performance. MAR 2012 Building on the substantial progress made in 2010 and 2011, HNP efforts to review and improve portfolio quality continue to be a key priority for the HNP Sector Board (SB) and an ongoing commitment of staff/operational teams at all levels, and are yielding further positive results. 4  The Portfolio Improvement Action Plan (PIAP) continues to be reviewed by the HNP SB periodically. It summarizes actions taken/to be taken, constraints and when projects at risk are expected to come out of the at-risk status. It also flags projects that are not at risk but are closely being monitored. Of the 14 projects approved between February 2011 and January 2012, HNP Hub organized Quality Enhancement Reviews (QER) for 11 of them, while the respective Regions organized QER for 2 projects. One (an emergency project) did not require a QER. All operational teams continue to make strides towards improved portfolio, as illustrated for the most recent period by the Regions in the paragraphs below.  In the Africa Region (AFR), at the end of January 2012, only 9.7% of HNP projects and 10.9% of AFR HNP projects were actual problem projects, coming down significantly from February 2011 (18.8% and 25%, respectively) and now better than or close to the Bank and AFR regional averages (13.6% and 10.4%, respectively). The percentage of exited projects rated satisfactory increased from 17 percent in 2008 to 57 percent in 2009 and to 68 percent in 2010. AFR has continued its program of intensive support for problem projects, resulting in a reduced number of problem projects, increased proactivity, and reduced net disconnect.  In South Asia Region (SAR) a significant share of the India portfolio has been restructured, resulting in improved portfolio performance, and teams are organized such that each includes staff with a specialized focus on operations, an economist, and a sector specialist. Most Task Management is now led from Country Offices, and only two TTLs reside in headquarters. This has assisted in coping with the budget constraints. SAR HNP has also leveraged substantial trust funds (TFs) to support analytical and advisory activities which complement supervision, as well as TFs which support actual supervision costs. However, most of the strengthening in supervision was not introduced over the past year, but in the two years prior, and focused on having Lead Specialists oversee all Task Teams (2 of 3 are based in the field). A Senior Operations Officer and the designated Lead Specialist review all ICRs (required every six months) before they are reviewed and cleared by the SMU and CMU.  In Europe and Central Asia Region (ECA), the portfolio is closely monitored to promptly address bottlenecks. For instance, an experienced task team leader (TTL) moved to Kazakhstan to cover ECA operations in Kazakhstan and the Kyrgyz Republic as well as to provide mentorship to field staff. Also, field-based staff are assuming task team leadership with coaching from more senior TTLs. Project Development Objectives (PDOs) and results framework are fine-tuned and streamlined to better match activities supporting the PDOs and improve their achievability. ECA teams include IT specialists to ensure quality assurance of IT investments and functionality. There is only one problem HNP project out of 15 in ECA which is being proactively managed.  In the Middle East and North Africa Region (MNA), the portfolio quality team reviews each ISR for adherence to guidelines and candor in ratings. In addition, the quality team is undertaking a complete HNP portfolio review (Jan-Feb 2012), with particular emphasis on problem projects and how to improve implementation. This review also includes training of task teams in pertinent operational guidelines and their application.  In East Asia and Pacific Region (EAP), as of January 2012, 23 of 27 staff are based in field offices, significantly strengthening the ability to engage in policy dialogue, coordinate project preparation, and provide implementation support to the clients. For example, in 2011 a senior staff member was deployed to Timor Leste to help to restructure and provide intensive support to a problem project. 5 HNP continues to incorporate risk assessment into reviews and has made progress in developing strategies to mitigate risks on a case by case/country by country basis.  The Operational Risk Assessment Framework (ORAF), which was not in existence at the time of drafting the HNP Management Action Plan,1 is an ―integrated framework for assessing the risks of investment lending projects.‖ HNP has taken steps to adopt the ORAF. Promoting staff knowledge and understanding of the ORAF is critical and all HNP staff have been encouraged to attend Operations Policy and Country Services (OPCS) training sessions on the ORAF. In addition, HNP, in cooperation with OPCS, conducted sessions on the ORAF during HD Week (FY11Q3).  All of HNP QERs examine the ORAF in detail, provide advice to teams on risk mitigation measures, and confirm/adjust ratings based on risk management/mitigation as appropriate.  The Regions continue to prioritize risk management and mitigation throughout the project cycle, including utilization of the ORAF. For example, in EAP, non-EAP peer reviewers, peer reviewers with expertise on risk management/mitigation, and external experts are used in the regional quality assurance processes. SAR employs both ORAF and Governance and Accountability Action Plans (GAAP) to identify, mitigate, and assess risks. Finally, ORAF is also an integral part of all new LAC project packages and some projects have developed a Governance and Anticorruption Council (GAC). HNP has made progress and continues to undertake significant efforts to improve the quality and realism of project design.  Fourteen new IBRD/IDA HNP projects were approved between February 2011 and January 2012, not including additional financing projects.2 Of the 14 new HNP projects, 11 have a significant focus on health systems strengthening.3 Institutional analysis remains a critical component of HNP‘s project development. All 14 new HNP Projects are based on institutional analysis (100 percent). At the regional and country levels, use of institutional analysis remains a foundation for the project development cycle. For example, a study in Uttar Pradesh revealed that weak institutions, accountability mechanisms, and poor organization had adverse effect on service delivery. This finding informed the design of the Uttar Pradesh Health System Performance Project (P100304).  Stakeholder analyses informed the design of all 14 projects approved between February 2011 and January 2012. For instance, the Mali Strengthening Reproductive Health (P124054) project preparation was highly participatory with extensive consultations among line ministries, NGOs, and Development Partners. This ensured that the project avoided overlapping with the activities of other donors and eventually identified substantial gaps in reproductive health service coverage.  The Flagship Course on Health Sector Reform and Sustainable Financing include stakeholder analysis (in the context of the politics of health systems reform) as a core component and institutional analysis in the sections on organization.‖ The course has reached over 80 HNP Staff in FY11, and additionally 35 HNP staff participated in the shortened version of the course during the 2011 HD Learning Week. 1 Published July 2010, OPCS. 2 P094755, P110156, P119090, P120391, P113349, P116167, P106735, P118708, P117157, P127949, P119917, P124054, P125740, P100304 3 P094755, P110156, P119090, P120391, P113349, P106735, P118708, P117157, P119917, P125740, P100304 6  In SAR, institutional analysis has been an explicit part of the preparation of the Nepal HNP&AIDS SWAp and the India ICDS Systems Strengthening and Nutrition Improvement Project.  In AFR, there continues to be a mandatory requirement for an internal review of each PCN and PAD, with input from both other AFTHE staff and the Operations Advisor in AFTHD. Further, in AFR almost all new operations are based on analytical work such as country status reports or public expenditure reviews.  In order to provide support for strengthened institutional analysis, AFR launched the Health Systems for Outcomes (HSO) program in FY10. HSO supports governments, Bank TTLs and development partners in over 24 countries in Africa by providing technical, analytical and knowledge management assistance directly related to health systems strengthening. For example, HSO‘s work has included contributions to the policy dialogue and analytical contributions in the form of country programs such as support for public expenditure reviews (PERs), public expenditure tracking surveys (PETs), country status reports (CSRs), to name a few. This work has resulted in: the development of an Africa Investment Case (to achieve the health related MDGs and other health objectives); numerous workshops on health insurance, improving financial access to care, governance and regulation of the pharmaceutical industry, and public/private partnerships; development of a Health Financing Flagship course; establishment of health systems communities of practice in Africa and a common platform for exchange of health systems information.  HNP has also carried out a review of the Bank's work on health systems analysis. An HNP Discussion Paper, Health Systems Analysis for Better Health System Strengthening, was published in May 2011. It provided examples from Mexico, Ethiopia, and Turkey on the positive contributions that health systems analysis has made to development of successful health system strengthening policies.  Additionally, where applicable, reforms in the health sector are phased to improve the chances of success. In the Bangladesh over the past decade Health, Nutrition and Population Sector Programs (HNPSP) built on the success of previous ones. The Health Sector Development Program (P118708), which was approved last year, built on the HNPSP 2005–2011 that made progress in decentralization and contracting out services to private service providers. HNP is continuing to ramp up its efforts to strengthen overall portfolio and match project design to country context and capacity.  All HNP QERs focus on simplifying design and providing guidance to teams on how better to match the complexity of the operation to the capacity of the country.  HNP is working to strengthen supervision in the field, and this remains a critical component of operational work despite the challenges due to declining supervision budgets over time. All regions continue to regularly review new and on-going projects to ensure appropriate fit to country context and reduced complexity. The Cluster Leader (CL) system in the Africa Region has substantially increased the management oversight of projects. CLs participated in a number of missions during the design stage to help ensure simplicity of project design, and also targeted problem projects and/or those identified as requiring restructuring. About 15 supervision QERs were conducted in 2010 on AFTHE projects (mostly problem projects), and the reduction in the number of problem projects over time partly reflects these efforts. An external review (by an HNP anchor staff and a consultant) was done of all projects that were upgraded from MU in FY11 to ensure that the ratings improvement was justified. 7  Of the 6 new HNP projects that had civil works components, 4 all 6 projects outline borrower responsibility and five include use of standards for civil works.5 Specific efforts have been undertaken to strengthen monitoring and evaluation (M&E) of HNP projects and improve the focus on results.  QERs pay significant attention on the results framework to ensure that the PDO and the intermediate results indicators adequately capture both the interventions supported under the project and the expected results of those interventions.  AFR continues to build upon the work done in recent years to improve M&E, including analytical work on target setting, finalization of a results chain (and guidance to staff on indicator selection), and an M&E checklist for task teams during design and implementation of operations.  In AFR, there continues to be a great deal of emphasis on Results-Based Financing (RBF) for which robust M&E arrangements are crucial – the region is currently at various stages of policy dialogue with 27 countries. In FY12, projects with RBF components have been or are expected to be approved in Benin, Burkina Faso, Mali, Burundi, and Sierra Leone. A recent review by AFTDE indicated that 100 percent of the HNP projects reviewed had M&E arrangements that were rated Satisfactory or Highly Satisfactory. 1.2.Renew the commitment to health, nutrition, and population outcomes among the poor. Original Response: Management agrees with this recommendation for high-fertility countries and regions—in particular as those areas have received less attention from other development partners as well over the past decade. Demand for stand-alone population (family planning) programs has declined over time. The Bank should increase support for reproductive health programs, which are usually better implemented when they are fully embedded into public health/clinical services. We would generally not support a return to stand-alone vertical family planning projects. Moreover, AIDS control projects (or components) and the work through UNAIDS have substantially contributed to reproductive health—and greatly expanded coverage of family planning for otherwise hard to reach population groups. In countries that are significantly advanced through the demographic transition, clients increasingly request advice and financing on financial protection, labor markets, and long-term care needs to address the ongoing demographic and epidemiological shifts resulting in an aging society. Finally, Development Grant Facility –financed programs such as the Special Program of Research, Development and Research Training in Human Reproduction (HRP), into which the Bank has substantial technical, financial, and managerial input, are contributing to the global reproductive health agenda. Management generally agrees with the need to ensure that project design responds to the priorities and needs of the poor, and to measure the full impact of improved health services for the poor. Management will therefore seek to ensure that adequate attention is given to poverty dimensions in project design and supervision. However, direct assessments, where feasible, may be technically complex and expensive. The Bank publication Attacking Inequality in the Health Sector—A Synthesis of Evidence and Tools (Yazbeck 2009) lays out a policy menu (pro-poor policy reforms 4 P100304, P110156, P119090, P120391, P113349, P118708 5 P100304, P119090, P120391, P113349, P118708 8 along six dimensions) and a list of the analytical tools for understanding the constraints to pro-poor targeting of public health investments in poor countries. As opposed to specific income groups, disease control programs must focus on the prevailing epidemiology (for example an AIDS program must focus on high-risk groups, irrespective of income). A malaria program focused solely on the poor would fail to eliminate malaria. Polio could only be eradicated from the Western Hemisphere by focusing on large, inclusive campaigns targeting all income groups. Such investments in public health and control of communicable diseases are global public goods, generating positive externalities for society, irrespective of income status. Investments in health systems should result in increased access and better quality of services— also benefiting the poor. Investment in social security and social safety net systems prevents the middle class from falling into poverty in case of a catastrophic health event. Management notes the substantive improvements over the past years in quantity and quality of HNP involvement in Poverty Reduction and Economic Management Network (PREM)–led analytical work, and agrees that HNP must be fully included in all Poverty Assessments and fully examined in the preparation of CASs. Management agrees with this recommendation. Particularly in the context of the global food crisis, the Bank needs to increase investments in nutrition, with a particular focus on maternal and infant nutrition. Management is therefore investing significant resources in the next few years to ramp up the Bank‘s analytical and investment work and leverage resources from other donors. The agenda for scaling-up nutrition is being catalyzed with additional budget resources, starting in 2009 and continuing for three years. The increased allocations are being utilized principally in Africa and South Asia, two Regions where the malnutrition burden is highest. These funds will be complemented by additional trust fund resources from Japan, and possibly from other donors that are currently engaged in discussions on this issue. Management agrees in principle with this recommendation, and will seek to ensure adequate provision for data collection, where technically feasible, during quality-at-entry and supervision reviews, in particular as far as poverty targeting is concerned. In order to accomplish this in a sustainable manner, management believes that the first priority in many poor countries is to establish routine health monitoring systems (surveillance, facility reporting, vital registration, census data, resource tracking, household surveys, and the like). These data systems need to be strengthened in parallel to investing in project-specific management information systems, in order to provide data and indicators that are needed for program targeting and monitoring for (but not limited to) the poor. Management agrees. Substantial progress regarding this recommendation has been made in some regions over the past years, both in terms of the analytical underpinnings, the need for capacity building, and investment needs. Management will seek to improve cross-sectoral collaboration with the PREM Network at country level as a precondition to further improvements. Management action: Recent major analytical work for staff and policy makers prepared by the Human Development Network to improve effectiveness in reaching the poor includes: Reaching the Poor with Health, Nutrition and Population Services—What Works, What Doesn‘t and Why (Gwatkin, Wagstaff, and Yazbeck 2005) and Attacking Inequality in the Health Sector—A Synthesis of Evidence and Tools (Yazbeck 2009). It is expected that this work will help clients and staff achieve better results in reaching the poor with health services. Concerning nutrition, in addition to disseminating the new Nutrition Strategy (Repositioning Nutrition as Central to Development—A Large Scale Action [World Bank 2006c]) the Regional Reprioritization Fund will allocate US$4 9 million over fiscal years 2009–11 to strengthen Bank capacity to scale up nutrition support and leverage resources from other donors. MAR 2012 HNP continues to be a key global player in sexual and reproductive health rights, particularly with the implementation of the World Bank Reproductive Health Action Plan.  In May 2010, the World Bank presented its Reproductive Health Action Plan (RHAP) for 2010- 2015 to the Bank‘s Executive Board. It reinvigorates the World Bank's commitment to reproductive health as outlined in its Health, Nutrition and Population Strategy of 2007, and is a central part of the Bank's focus on strengthening country health systems and efforts to accelerate progress toward MDGs 4 and 5. The RHAP lays out the key issues, challenges, priorities, and opportunities to mobilize greater attention and support and deepen the Bank‘s work on reproductive health. The RHAP targets: interventions for 57 countries with high maternal mortality and high fertility, focusing on meeting the reproductive health needs of the youth; improving RH outcomes through health systems strengthening; and leveraging partnerships with governments, CSOs, academia, bilaterals and multilaterals. The Bank presented a Year 1 implementation update to the Board, at an Executive Directors' Seminar on July 19, 2011. Considerable progress was made in Year 1 as illustrated below.  HNP has undertaken significant Analytical and Advisory Work on RH which have facilitated policy dialogue. For instance, in FY11 RH Profiles for 47 priority countries were completed and disseminated. The RH Profiles provide readily available information on the country‘s RH status, constraints, and key actions to accelerate progress towards MDG 5. The RH Profiles are being used to incorporate key information on MDG 5 into the project documents, policy dialogue with governments, and provide evidence and data for incorporation into the Country Assistance Strategy (CAS). For instance, the analysis and policy implications in the RH profile for Mali was used in policy dialogue with the Government and Development Partners and was incorporated in the Mali Strengthening Reproductive Health Project (P124054) which was approved in December 2011. Additionally, all 6 countries with high MMR and/or TFR (Honduras, Bolivia, Haiti, Nepal, Tanzania and Guinea) that had a new CAS or an Interim Strategy Note approved between February 2011 and January 2012 addressed RH issues.  Additionally, the Bank is finalizing case studies to better understand the impact of political leadership on the progress on MDGs 4 and 5 (Bolivia, Nepal, Egypt and China). AFR is undertaking a flagship multisectoral analytical work on Demographic Dividend in African countries and the Africa RHAP is being finalized. In EAP, the analytical work Delivery strategies for sexual and reproductive health and second chance education and training for youth supported policy changes through identifying effective strategies in Papua New Guinea, Solomon Islands, and Vanuatu. In ECA, the recommendations of the Review of reproductive health content in medical and nursing education curricula are being incorporated into curricula in Tajikistan, Uzbekistan, Moldova, Armenia and Azerbaijan. In SAR, the recommendations of Review of access barriers to RH services on RH and HIV convergence, stigma and harm reduction, resulted in the National Plan for Convergence on RH and HIV and influenced design of the new National AIDS program in India. SAR has just launched a region-wide effort on RH; it has mobilized regional and trust fund resources, appointed focal points, briefed Country Directors and will be convening clients and staff in Sri Lanka in April 2012 to consider strategies. The effort includes identifying ways to expend support for reproductive health under existing and planned projects, as well as analytical work to raise attention to maternal and reproductive health as 10 well as to inform the national response. In LAC, the framework on the Review of legal framework of legal rights on teen pregnancy was undertaken in Central America.  A number of health projects approved in FY11 had RH components in countries such as DR Congo, Mozambique, Nigeria, Swaziland, Bangladesh, Pakistan, Iraq, Yemen, Argentina, and Brazil. So far in FY12, an El Salvador project addressing RH and two RH projects in Mali and Burkina Faso were approved. The two RH projects focus on the provision of family planning to the poor and have contraceptive prevalence rate as a PDO indicator.  The Bank has enhanced its capacity and expertise in RH. Eighteen focal points from across all regions have been designated for Population and Reproductive Health. They actively coordinate and facilitate RH activities in the respective regions with support from HNP Hub. A session on the RHAP was organized during the 2011 HD Learning Week. In November 2011, HNP Hub held consultative meetings with regional Sector Managers and RH focal points to update and plan RH activities across regions.  The Bank‘s course on MDG 5 ―Population and Reproductive Health - Achieving the Millennium Development Goals: Poverty Reduction, Reproductive Health and Health Sector Reform ‖, was revised to better reflect the recommendations of the RHAP, with the first training held in May 2011 in Budapest and another scheduled to be held in February 2012.  The Bank is engaged in an ongoing and productive dialogue with partners (UNFPA, UNICEF, UNAIDS, WHO and UNWOMEN as part of the H4+ partnership) and has been working together at the national and regional levels in an effort to enhance synergies and achieve maximum results. The Bank continues to collaborate with UNFPA, UNICEF and WHO in developing global and country maternal and child mortality estimates.  The Bank provided $1.05 million in FY12, through the Development Grant Facility to support civil society organizations for capacity development for the Population & Reproductive Health Capacity Building Program.  A CSO Consultative Group has been formed to ensure continued dialogue and priority for sexual and reproductive health and rights at the World Bank and to ensure accountability of RHAP at the national level. Improving health results for the poorest and most vulnerable continues to be a cornerstone of the work of HNP.  In September 2011, HNP launched a video series ―Maya and Health Systems‖ which provides a simple but vivid illustration of the core messages on how strengthening health systems is crucial to deliver health results.  HNP‘s ongoing seminar series Making Health Systems Work for the Poor continues to be a success in presenting evidence from health reforms with a measurable impact on the poor.  HNP has made progress in incorporating poverty dimensions into new projects. Of the 14 new HNP projects, the poor are the main beneficiaries of 12 projects.6 Nine out of the 14 projects (64%) explicitly target the poor in the PDO and/or indicators of the results framework. Although 3 projects do not mention the poor explicitly, these projects target the poor by focusing on geographic areas where the population is predominantly poor (rural areas – e.g. Uzbekistan) or areas where progress is lagging for key health indicators (e.g. Mali, Burkina 6P094755, P110156, P113349, P116167, P106735, P118708, P117157, P127949, P119917, P124054, P125740, P100304. Two projects do not specifically target the poor due to the nature of the projects (university hospital modernization and hospital waste management). If these are excluded, 75% of projects approved in the last 12 months explicitly target the poor. 11 Faso).  Increased emphasis on poverty and equity continues to be a key focus of the Bank HNP regional work. For example, AFR has established a baseline on the proportion of projects that explicitly set out to measure equity in access or outcomes and has incorporated equity concerns in its results chain and other job aids (14% in 2010). AFR intends to conduct a follow up survey in the near future. Support to the Africa Programme on Onchocerciasis Control (APOC) is also ongoing and has made great strides in reducing onchocerciasis incidence. Moving forward, there are plans to use this program to launch an effort on neglected tropical diseases which disproportionately affect the poor.  In SAR, all new operations include indicators which track impact upon the poor or marginalized groups, and the Region has started work using the Equality of Opportunity Index, initially in Pakistan.  Additionally, ECA operations aim at improving access to health care for the poor with a special emphasis on out-of-pocket payments and informal payments that represent a financial barrier for the poor. A household survey was carried out to investigate NCD risk factors and coverage of selected NCD interventions among adults by socioeconomic status in seven ECA countries: Azerbaijan, Georgia, Moldova, Russia, Tajikistan, Uzbekistan and Ukraine. Various analytical work on the pricing of pharmaceuticals (which typically account for a large share of health expenditures by the poor) and out-of-pockets payments have been carried out to provide inputs for the regional health report. Support to improving nutrition outcomes continues to be a significant component of HNP‘s work to deliver improved health results.  The World Bank continues to support the evolution of the global Scaling Up Nutrition (SUN) movement. Launched at the Bank/IMF Spring Meetings in April 2010 with the publication of the SUN Framework for Action, there has been substantial progress in 2011. To date, 27 countries have joined the SUN movement and stewardship arrangements for the SUN were prepared and discussed in late 2011. The Bank has agreed to be the donor convener or co- convener in seven of the 27 countries (Bangladesh, Benin, Ethiopia, Guatemala, Nepal, Peru, and Senegal), taking on the responsibility for liaising between the national nutrition focal point and the community of donor partners in each of the "SUN early riser" countries for which it has taken on the coordinating role. Ultimately, more harmonized financing and support for scaling up nutrition at national level will result. The Bank has been an active participant on both the Transition Team (the interim SUN global leadership entity) and on Task Force D (the Donor Partners' task force) during the first full year of the SUN movement and has taken on an advising role in the new leadership structure, represented by Vice President Tamar Manuelyan Atinc who will form part of the Lead Group of the SUN movement. The Bank will also continue to be a member of the Donors' Network, working to align donor agency support for SUN countries as they develop and implement plans for nutrition programming at scale.  The Bank team is completing programmatic guidance briefs on agriculture/food-security- nutrition, health-nutrition and social protection-nutrition linkages. These guidelines, which are directed at Bank TTLs, will advance the ability of the Bank to support positive nutrition outcomes through nutrition sensitive activities across multiple sectors (e.g., Agriculture, Social Protection, and Health, with possible more to come (Education, Climate, etc.).  A new initiative started in 2011 is the SecureNutrition: Linking Agriculture, Food Security and Nutrition Knowledge Platform. This cross-sectoral initiative is led by HDNHE in collaboration 12 with PREM and ARD, with funding until FY14 from the Reform Secretariat. Realizing the urgent need to bring the agriculture, food security and nutrition agendas closer together in order to achieve sustainable improvements in the nutrition status of vulnerable populations including pregnant women, and children up to the age of 24 months, the design team is working to engage with stakeholders both within the Bank and with external partners to address operational knowledge gaps, enable easy access to new information as it becomes available; encourage the development of networks between institutions, governments, program implementers, and individuals and to support innovation for improved nutrition outcomes through projects in agriculture and food security. The launch of the knowledge platform website is scheduled for April 2012, and the team has been actively collaborating with both internal and external Partners to discuss the Platform's key audiences and activities.  With recruitment of field-based nutrition staff, progress on nutrition work in the regions has been significant. Since FY11, 18 analytical and advisory products and 20 new operations addressing nutrition and food security have been delivered or approved.  SAR produced a Regional Assistance Strategy (RAS) for Nutrition with explicit indicators of success, which has been adopted by the Regional Management Team. Over the past year, SAR has further expanded its large nutrition initiative, and now has multisector work underway in Nepal (under the 1000 Days Project which will incentivize communities to address the proximate multisectoral factors contributing to malnutrition) and Afghanistan where a multisectoral nutrition action framework has been completed and is now under review. Country Directors in SAR have been particularly helpful in engaging counterparts around the nutrition agenda and supporting HNP staff to pursue both AAA and Lending. In Pakistan, where the Ministry of Health was recently eliminated and provinces bear the debt burden of accepting Bank financing, the team has been working with AusAID to use their grant funds as co-financing to encourage poorer provinces to work with the Bank on addressing malnutrition. The Region has also initiated a large work program across sectors aimed at reducing food and nutrition security through analysis and advocacy- under a multi-donor trust fund. SAR currently has 4 active nutrition projects, two of which are in Nepal, one in Bangladesh, and one in Sri Lanka. A total of 8 projects with nutrition components are in the pipeline for SAR, two of which will be implemented through the Agriculture sector, aiming to improve grain storage facilities in Bangladesh and food security in Nepal.  An increasing number of projects in AFR contain nutrition related interventions (7 approvals in FY11 compared to 4 in FY10), and the Bank has been instrumental in advocating for food fortification as a cost-effective approach to improve nutrition. In Tanzania, for example, a technical assistance product done by the Bank (―Action Plan For the Provision of Vitamins and Minerals to the Tanzanian Population through the Enrichment of Staple Foods‖) showed that food fortification would have a benefit-cost ratio of 8.22:1, leading to a government decision to make Tanzania the first country in East Africa to institute mandatory fortification of wheat flour, maize flour and edible oil. In Mali, the integration of early child development activities and nutrition are being piloted, as well as a social safety nets program in Niger including nutrition objectives. Eight projects for nutrition in the Africa region are currently in the pipeline, with five of them (in Cote d'Ivoire, Madagascar, Malawi, Mozambique, and Senegal) to be approved by the end of FY12.  In LAC, a number of countries have implemented activities directed at children in the first 1000 days. Nicaragua, with the support of the Japan Trust Fund for Nutrition, and building on the success of the Bank/Japan TF-supported work in Peru with the development of a highly 13 effective video on stunting and young child undernutrition, has made a similar video for use with health staff training in the communities. In Panama, the Japan TFs supported a regional workshop on community-based nutrition programs in Central America in October 2011 where nine countries from the region participated, sharing experiences in a south-south learning forum that was highly successful. Follow-on activities to build capacity in community-based nutrition programs in Central America are now planned for the remainder of FY12. In Haiti, the Bank is working with the government to (i) finalize the revision of the nutrition training curriculum for the community agents, (ii) supervise the implementation of the nutrition training of the agents, (ii) supervise the nutrition interventions in the field of the household development agents and (iv) continue to provide technical assistance to the President‘s food security and improved nutrition initiative ABA GRANGOU. Moreover, non-lending technical assistance work supported by the Bank and DFID helped to revise the country's National Nutrition Policy, which included, among others, an update of the nutrition situation and the addition of a module on reducing vulnerabilities in emergency situations.  Pilots of innovative approaches (for example, a locally-produced Ready-to-Use Therapeutic Food commodity and Corn Soya Blend in Ethiopia and a pilot to test the effectiveness of social transfers, food or cash, to improve the birth weight of children in Nepal) are being implemented in Ethiopia, Haiti, Guatemala, Peru, Ecuador, and Nepal. After the pilots are evaluated, lessons learned will be disseminated through South-to-South exchanges, regional consultations, and other fora. Analytical work (that provides information on trends in HNP outcomes among the poor) and capacity-building activities (specifically the delivery of training courses and development of analytical tools) is equipping Bank staff and client countries to better, and more easily, monitor HNP outcomes among the poor (as noted above on the PDO indicators focusing on the poor).  Overall, HNP is continuing to work on systematically increasing the emphasis on poverty and equity through a work program on monitoring equity and financial protection in the health sector. This work cuts across units and is produced by staff from HDNHE, DECRG, WBI, and the Regions. A major ESW on ―Health Equity and Financial Protection‖ led by HDNHE, but with large contributions from staff in DECRG, WBI and the regions, was completed in Q3 FY12 and the concept notes for new knowledge products, training activities (external and internal) and technical assistance are currently being prepared. Management of the knowledge produced through these activities is currently available on the HNP website www.worldbank.org/povertyandhealth (internal and external), and will soon also be available through the a new pilot knowledge exchange site on Health Equity and Financial Protection that is being developed in collaboration with the Chief Economist‘s office and will equip all staff to monitor health outcomes among the poor.  Three poverty assessments were delivered between February 2011 and January 2012. All assessments addressed priority health issues in the specific country context – high maternal mortality and TB rates and ongoing sector-wide reform (Kyrgyz Republic), nutrition-related problems and high fertility (Niger), and chronic malnutrition and need for prenatal care for the poor (Panama). During the same period, 23 Country Assistance Strategies/Country 14 Partnership Strategies/Interim Strategy Notes were approved by the Board, of which 19 (82.6%) included specific health outcomes.7 As noted earlier, the CASs for high fertility/high maternal mortality countries all addressed RH issues.  In terms of knowledge generation, the past year saw the completion of a number of analytical products that will enable Bank staff and clients to better monitor the degree to which health systems improve health outcomes among the poor, provide financial protection from the costs of out-of-pocket spending, and the pro-poorness of government spending and revenue-raising. First, the database on HNP outcomes and coverage rates by wealth quintile (with data for 98 countries from 1985-2007) which was completed in calendar year 2011, was published on-line. Second, seven new reports in the Health Equity and Financial Protection country report series were also completed. Third, 54 country-specific Health Equity and Financial Protection datasheets were completed. These datasheets draw together data from WHS, DHS, MICS and other household surveys to provide country-specific pictures of levels and inequality of health outcomes, health care utilization and financial protection. Finally, Bank staff across the HNP Family produced a number of HNP discussion papers, working papers and health sector review chapters which report on changes in HNP outcomes on the poor; these are available on the HNP website.  In terms of capacity-building activities, there were substantial gains. WBI delivered its well- established e-learning course on Health Outcomes and the Poor in Q3 FY12 and supported the delivery of a regional course on equity (in collaboration with the Asia Network on Health System Strengthening) in Q2 FY12. DECRG continued to update and improve the health modules of the ADePT-Health software that automates and simplifies the analysis of financial protection and equity in the health sector and, in Q4 FY11, published an ADePT Health user manual called ―Health Equity and Financial Protection‖. Together, HDNHE and DECRG developed a modular one-day hands-on training course on the Measurement of Equity and Financial Protection in the Health Sector using ADePT, adding additional modules over the course of the year. This course was delivered nine times to a total of approximately 300 participants, including two sessions for Bank staff (during HD Week and Q2 FY12), at conferences in Canada, Senegal, Colombia, South Africa, the Philippines and Thailand, and on- demand to Bank clients and staff in India and Morocco, with about 95 percent of training participants rating overall training quality as 4 or 5 on a five point scale. 1.3.Strengthen the World Bank Group’s ability to help countries to improve the efficiency of their health systems. Original Response: Management generally agrees with this recommendation. The efficiency argument is a key rationale, in particular for working with health systems in middle-income countries. Since national health expenditures rise with national income, improving sector efficiency makes an important contribution to fiscal sustainability. However, in the health sector there are important efficiency equity trade-offs. This points to the fact that efficiency gains should not be the sole objective of Bank-financed health programs. 7 Of the 4 CASs that did not explicitly include health, the reasons were: 1) other agencies provide more significant funding (Honduras); 2) social inclusion of minority groups is a CAS outcome, but the results framework does not specifically mention the health aspect (Bulgaria); 3) improved water and sanitation are expected to impact health outcomes (Kiribati); and 4) there were significant gains in health outcomes in the recent years (Algeria). 15 Management agrees that it is necessary to carefully assess the need for additional finance where other donors are contributing substantial amounts. While fiscal space and potential budget substitution by ministries of finance should be closely monitored, the empirical evidence of distortionary effects of large vertical disease programs is scanty. Proposals for Bank support for new disease-specific programs are closely coordinated with other donors and often fund complementary financing and institutional needs, for which financing was unavailable from other donors. Management agrees partially with this recommendation: Technical support and financing for management information systems as well as routine surveillance and vital statistics systems should be ramped up. However, the outcomes of management information system investments may be hard to evaluate fully within the timeframe of a project, and multiple determinants influence health outcomes. Management also notes that the Paris and Accra Declarations and the new OP 13.60 emphasize the use of pooled funding and country-level M&E systems instead of ring-fenced funding and stand-alone M&E systems. Hence, the standard should be that sufficient evidence on outputs, intermediate outcomes, and outcomes should be collected to establish a credible story line to assess the link between Bank-financed investments and overall sector progress, including efficiency and efficacy gains. Management action: The Human Development Network and the Regions have carried out major analytical work that will help policy makers and Bank staff to better understand challenges and trade-offs in health financing, risk pooling and insurance, the issue of fiscal space and external assistance (Health Financing Revisited—A Practitioner‘s Guide [Gottret and Schieber 2006]). Furthermore, the Bank is a lead sponsor of the International Health Partnership (IHP+). This is a country-led and country-driven partnership that calls for all signatories to accelerate action to scale up coverage and use of health services and deliver improved outcomes against the health related MDGs, while honoring commitments to improve universal access to health. MAR 2012 Efficiency and quality are both important for improving health results. HNP continues to work with internal and external partners to maximize efficiency while maintaining or improving quality of HNP results.  Of the 14 new HNP projects approved between February 2011 and January 2012, 12 address efficiency, including cost benefit analysis, overall potential efficiency gains, reduction in wastage and leakages, and increased cost-effectiveness, for example.8 Additionally, indicators on improving efficiency were incorporated in the results framework of all projects. For instance, the Tanzania project (P125740) has the following indicator for improving efficiency in delivery of health services: average outpatient attendances per clinical health worker by Local Government Authority.  RBF programs not only improve health service utilization and quality, increasing but also improves efficiency, and enhances equity. Some RBF programs work on the ―supply-side‖, paying incentives to health facilities for performance tied to services. Health facility incentives can be used for staff bonuses and for improving the performance of the health facility by, for example, financing more outreach visits, or purchasing essential equipment. In addition, more 8 P119917, P124054, P125740, P100304, P094755, P110156, P119090, P120391, P106735, P113349, P118708, P117157 16 investments in health management information system improvements are made since payments are contingent upon providing credible data. Others work on the ―demand-side‖, providing financial or ―in-kind‖ incentives (such as transport vouchers or supplies for the newborn) to certain target groups (such as pregnant women and the poor) to overcome barriers to health service utilization. In some countries a combination of demand- and supply-side incentives are used.  Implementing RBF strategies have to be carefully modulated in low-resource settings to avoid overly complex operations that overwhelm country capacities. Hence, country context is the most important deciding factor when deciding which RBF mechanism is most suitable. For example, Liberia‘s post-conflict context necessitated that the RBF intervention initially focus on addressing supply-side constraints. Since Liberia has had a long history with PBC through the support of NGOs, the capacity of its local institutions to deliver health services is weak. The Ministry of Heath is keen to promote the use of result-based supply-side financing at the health facility level as a means to empower its own institutional capacities.  In other countries, focusing on the supply-side allows them to better align the intervention with their overall health sector reform agenda. In Lao PDR, where a health equity fund to increase access to health for the poor is being implemented, the government has decided to scale-up the provision of free MCH services using an RBF supply-side approach. While free services may increase health utilization rates it may not necessarily translate into better health outcomes. Consequently, the RBF intervention will focus on both quantity and quality of services.  Recognizing that some barriers to accessing MCH services can best be addressed through a demand-side intervention, some countries have also turned to using a combination of supply- side and demand-side incentives. For example, following the successful implementation of the national PBF scheme, the Rwanda pilot is focusing more attention on improving MCH outcomes using demand-side incentives. The current pilot pays for demand-side incentives, as well as Community Health Worker (CHW) incentives. The role of the CHWs is to encourage more referral of clients to the health centers.  Increased efficiency at the regional and country levels is also a high priority. RBF is a systemic intervention that aims to enhance transparency and accountability and improve health information systems since reported results are verified when linked to incentives. RBF also supports the use of quality assurance mechanisms which are incentivized in some countries through the use of balanced score cards. Local authorities (e.g., Rwanda) and/or external organizations (e.g., Afghanistan) verify results, which are publicized for each organization or health facility. Independent verification encourages countries to invest in improving health Management Information Systems since payments are linked to verified results. Monitoring is not only the central and/or public sector responsibility, but also includes civil society in many countries. Community involvement at the health facility level helps ensure accountability for performance incentives.  In EAP, RBF approaches have been utilized to improve efficiency and create stronger linkages to results, especially in Lao, Vietnam, Philippines, and the Pacific Islands. EAP has also supported China in M&E activities at a decentralized level for the implementation of the ambitious Chinese National Health Reform. In SAR, both Bhutan and Nepal have conducted assessments of public expenditure for health, which aim to identify opportunities for efficiency gains to create fiscal space. Innovative work is ongoing in AFR, where a randomized controlled trial (RCT) of different approaches to deliver drugs to peripheral health centers in Zambia is being carried out. This is the first published RCT on logistics management in health. The best 17 approach identified by this study proved to be extremely efficient in reducing stock outs and has now become national policy in Zambia. Finally, work in LAC focuses on strengthening regional health systems. As such, operations aim at improving essential public health functions (Argentina), public sector management (Brazil), health insurance (Argentina, Mexico), health networks (Brazil), hospital management (Brazil), pharmaceutical policies, and other issues aiming at improving coverage, quality, and efficiency of regional health systems. The Bank has developed guidelines of strategies to improve organizational performance in service delivery and a discussion paper ―Health Systems Analysis for Better Health System Strengthening‖ was published in May 2011. The plan is to develop opportunities to work with TTLs and clients to carry out the analysis needed to apply this approach and use it to improve efficiency of health systems.  The Bank has developed guidelines and tools for country teams to integrate costing, cost- effectiveness analysis and sustainability analysis into the impact evaluations of its RBF for health programs. The new tools and guidelines, which have been piloted in several countries including Afghanistan, Benin and Rwanda, are being finalized.  The IHP+ continues to be a key partnership in catalyzing harmonized support for the national health strategy with a focus on delivering on its results framework as well as reducing transaction costs and increasing efficiency, based on the principles of Aid Effectiveness (described in more details below).  The Health Financing and Health Systems Global Expert Team has worked with the Sector Board to develop a strategic list of priority countries for FY12 and has responded to a number of requests from SMs and TTLs and different Bank regions to advise on project design and reform plans to improve health system efficiency and equity. Some highlights of this work include: high level consultations with South Africa, Russia (Putin Commission on the Future of Health Care), India (High Level Expert Plan), China and Indonesia (Health Insurance Integration). In the next few months, it is expected to confer with Mexico on strategies to strengthen Seguro Popular and move to a "phase 2" of reforms, with the new Egyptian government on universal coverage, and ongoing development of practitioner to practitioner capacity building in East Asia, South Asia, and Africa on health insurance and provider payment.  The Harmonization for Health in Africa (HHA) partnership (AfDB, JICA, UNAIDs, UNFPA, UNICEF, USAID, WHO, World Bank) is also important for increasing efficiency. The World Bank HSO team closely coordinates and collaborates within the context of this partnership to ensure harmonized support for African countries, increasingly working in the context of joint missions and shared responsibilities. HHA, in turn, works closely and collaboratively with other regional and global partnerships such as the Global Health Workforce Alliance, Joint Learning Network, and IHP+. Increasingly, the HHA has carried out joint efforts with the IHP+ in support of a number of countries.  Similarly, in SAR, SWAps and IHP+ principles defined the standard for HNP operations, outside of India. The Region has increasingly consolidated multiple projects so as to reduce transaction costs, but also so as to support clients in making decisions across their entire sector expenditure/development program. This has been complemented by HNP Public Expenditure Reviews in Nepal and Bhutan, and the initiation of similar reviews in Pakistan. The regions has sought expertise in Public Financial Management (PFM), and collaborated more closely with broader PFM and governance initiatives with a view to focusing on sector performance. The new Uttar Pradesh State Health Project in India exemplifies the kinds of operations which 18 target sector governance. While Bank financing is a fraction of the State‘s health budget, it will improve the use of Government of India and other donor resources.  Additionally, ECA operations support the health sector as a whole, including line ministries‘ stewardship capacity, health financing, health technology assessments, public health, and cost benefit of a basic package of services. Examples of such operations include the Armenia Health Sector Modernization Project, Kazakhstan Health Sector Technology Transfer and Institutional Reform Project, and the Turkey Health Transformation and Social Security Reform Project, which are supporting country efforts to improve efficiency of their health systems in a comprehensive manner. HNP continues to be a leader in the global dialogue on delivering results in line with the principles of aid as well as more general development effectiveness outlined in the Paris Declaration on Aid Effectiveness, the Accra Agenda for Action and the Busan outcome document.  HNP continues to work towards improvement in selectivity/complementarity of support in order to maximize the impact of support at the country level and to continue to work towards implementing the principles of aid effectiveness. Increased aid effectiveness remains a key aspiration of ongoing projects as well. In EAP, for example, the approach has been to consider support provided by others and to serve as a donor of last resort in the areas receiving significant support from other sources. In SAR, financing for HIV/AIDS components has been rolled into existing SWAps in Nepal and Bangladesh and are part of a provincial level health sector operation in Pakistan. Collaboration with the Global Fund is routine in SAR, with periodic meetings between their staff and the SAR UNP team as a whole and country-level collaboration around each project – e.g., joint missions are the norm. Bilateral engagement is continuous, with staff in each Country Office closely engaged with other DPs with explicit partnership arrangements. In Swaziland, development partners including the US President‘s Emergency Plan for AIDS Relief (PEPFAR) and the Global Fund are supporting prevention and treatment of HIV/AIDS but impact mitigation and response management were not adequately addressed. To fill the gap, the Swaziland Health, HIV/AIDS & TB Project (P110156), co- financed jointly with the EU, is supporting the implementation of the Orphans and Vulnerable Children (OVC) cash transfer scheme to increase social safety net access for OVC. It is also establishing a Management Information System and designing an impact evaluation.  A number of HNP-managed partnerships continue to support the aid effectiveness agenda, highlighting the need for collaboration amongst partners and coordinated financial and technical support to countries based on a single national health plan, thus resolving a number of existing issues regarding financing for earmarked diseases. These include the IHP+ and the related Health Systems Funding Platform.  IHP+ continues to support the implementation of the Joint Assessment of National Strategies (JANS). The JANS tool, which lays out the parameters for assessing a country‘s national health strategy, is now widely considered a global public good in the health sector. JANS have been carried out in a number of countries – Ethiopia, Nepal, Uganda, Ghana, Vietnam, Malawi, Mali and Rwanda – in addition to having been used more informally in others (e.g. Zambia, Bangladesh and Togo). Based on the lessons learned so far the JANS Tool and Guidelines have been revised and an options paper on the JANS process has been developed. The issue of how to take forward the JANS approach will be the subject of a multi-stakeholder meeting in February 2012. Work on a common M&E framework within IHP+ is being led by WHO in close collaboration with the Bank and many other partners. An IHP+ guidance document for a 19 country led platform on "Monitoring, Evaluation and Review of National Health Strategies" was finalized in 2011. The M&E work is closely coordinated with the implementation of the recommendations of the Commission on Information & Accountability for Women’s and Children’s Health, where IHP+ is the lead on the work on monitoring of results in countries. Similarly, IHP+ will be the lead for work on health sector reviews as part of implementing the recommendations of the Commission. Within the Bank, TTLs are actively involved in all aspects of IHP+ activities.  The Health Systems Funding Platform has also made progress. The World Bank is working with the GAVI Alliance, the Global Fund, WHO, and other partners to accelerate progress towards the MDGs to coordinate, mobilize and streamline the flow of existing and new international resources to support national health strategies. For example, Nepal is using the Platform to further implement the aid effectiveness agenda and bring more donors into a joint fiduciary framework. A JANS was conducted in 2010 and the World Bank, AusAID and DFID were able to use this process in place of the stand alone appraisals. The leading aid donors (DFID, World Bank, the GAVI Alliance, USAID, UNFPA and UNICEF9) agreed to sign a joint financing agreement for pooling and non-pooling partners committing additional resources through a single framework for financial management, with one report and one audit. Nepal is currently urging donors to carry out their commitment to using one M&E framework. Similarly, development partners and the MoH in Ethiopia are working towards agreement on a joint fiduciary framework, following a joint financial management assessment in November 2011. This work links directly to the preparation of a new RBF health project using the new Program for Results (P4R) lending instrument. Other platform work is ongoing in Cambodia and Benin. Financed by IHP+, the Bank has led work with GAVI, the Global Fund and WHO on how to conduct a Joint Financial Management (FM) Assessment; the report was endorsed by all agencies in November 2011. Joint FM Assessments are now planned for several countries including Sierra Leone and Senegal in order to consolidate the approach. These joint FM assessments are an important step towards establishing Joint Financing Arrangements. The FM work is being carried out in close collaboration with the FM Sector Board and country level Bank staff. Health Information Systems (HIS) are a critical component to a functioning health system and effective health governance and M&E. HNP continues its strong support of this work with a diverse array of mechanisms.  Building capacity at the country level is critical for the sustainability of strong monitoring and evaluation systems. Of the 14 new HNP projects, 13 projects include components to strengthen country M&E systems.10  As reported in the 2010 HNP MAR, ―Under the IHP+, global health partners have developed a common monitoring and evaluation framework.‖ Additionally, also reported in 2010, ―In partnership with the Health Metrics Network, the Bank has supported country assessments of health information systems in over 80 countries.‖  HNP supports country impact evaluation (IE) programs of RBF for health projects, through 9 It is expected that AusAID and KfW will sign the JFA in due course. 10P119917, P124054, P125740, P100304, P094755, P110156, P119090, P120391, P116167, P106735, P113349, P118708, P117157. The project, P127949, is a 1.5 year emergency project in the Horn of Africa and strengthens UNHCR‘s HIS, but not a country M&E system. 20 four priority areas: (i) financing IE activities through the HRITF, (ii) development of technical resources such as an IE Toolkit, questionnaires, and training manuals; (iii) direct technical support to country teams on design and implementation of prospective, rigorous IE; and (iv) training in IE in annual IE training workshops for both Bank staff and counterparts. To date, the Bank has directly supported over fifteen RBF for health project teams on IE activities.  The Tamil Nadu Health Project won a national IT award for its HIS efforts and has been sharing the examples with other States and now with Sri Lanka. Building Management Information Systems, and M&E capacity is a core part of the projects in Nepal, Bangladesh, Afghanistan, Uttar Pradesh and India AIDS.  In Chile, the Bank provided assistance to the Government, under a fee-for-service agreement, to assess and make recommendations for improving the health information systems as well as purchasing mechanisms and health system incentives. Increased support of health systems, strengthened partnership and coordination, and improved knowledge management and learning remains critical to HNP‘s efforts to make progress on the health-related MDGs.  The HNP Knowledge Resource Center (KRC) was successfully launched by the Hub in FY10, with strong positive feedback from the regions. The KRC delivers just-in-time advisory services that respond to TTL requests within 24 hours. The strongest positive feedback has been for the consultant/TORs database, which is available on the internal website and updated continuously, and the HNPWeekly internal e-newsletter. Other products include HNPFLASH, the external monthly subscription-only e-newsletter, which has been active since June 2010. In spite of this short time and no marketing, HNPFLASH has 1,000 new subscribers. The KRC also coordinated HNP's 2011 Learning Week, which consisted of 2 parallel course series including the nascent HSS Staff Flagship Program. The full HNP Learning Program, which consists of BBLs and seminars jointly-sponsored with other units, remains as active as in previous years. This year the KRC finalized and delivered the first On-Boarding Program in the Bank, which received excellent reviews and is being offered on a monthly basis. FY12 will see the development of the first cohesive HNP Learning Strategy based on the newly developed core competency framework and the Bank‘s own new Learning Strategy. The KRC also collects and synthesizes knowledge content for the central HNP Website, edits the HNP Discussion Paper Series and HNPNotes series, and conducts vigorous outreach to the regions to meet internal demand for knowledge and learning.  As part of the Bank‘s overall Knowledge Management Agenda, the HNP Sector Board asked the HNP Hub to establish a process by which the relevant knowledge generated inside and collected from outside the Bank is accumulated and stored for future access. In parallel, the HD Knowledge Management function is working on a prototype for a ―Knowledge Exchange‖, through which the accumulated knowledge is made easily accessible in ways that meet operational demands. Knowledge Repositories, an interim step towards Knowledge Exchange, is a process of collecting knowledge and a place to store/organize it. They are being set up for each thematic area. The Knowledge Exchanges to be delivered in FY12 are topic-specific, shared sites, which facilitate easy access to key documents and best practices as well as synthesize and distill current thinking. The first 3 Knowledge Exchange themes are Pharmaceuticals, Health Equity and Financial Protection and Nutrition. 21 1.4.Enhance the contribution of support from other sectors to health, nutrition, and population outcomes. Original Response: Management agrees in principle, but implementation can be challenging. For example, health is a secondary objective for many clients in water and sanitation operations and borrowers are reluctant to add potentially costly components. In addition, the current mandate, staffing, and budget of the water sector does not allow the sector to be ―accountable for health benefits‖ as IEG suggests. Under IDA 15 guidelines, it is mandatory to track ―safe and sustainable access to water and or sanitation services‖ as an outcome, with health benefits classified as impacts, rather than outcomes. Because it is difficult and expensive to track health impacts, the number of water projects with a health objective has declined, and this trend is likely to continue absent changes in policy and/or additional resources. Management generally agrees that we need to support client countries to seek improved complementarity of water and sanitation programs with health projects, in particular as clients are reluctant to burden all water and sanitation operations with potentially small (albeit cost effective) health impact objectives, and to demonstrate and document empirically such impact and outcomes in each case. Management agrees and notes that significant additional resources have been mobilized from fiscal year 2009 on to better respond to client and partner requests for embedding Bank support in multisector programs, or to provide just-in-time advice to countries and partners in development. Specifically, over US$3 million additional Bank budget will be allocated per year to ramp up Bank support for the 14 IHP+ countries, mostly in Africa, including the establishment and staffing of two regional technical support hubs in Africa. Additional resources are being provided by other donors. Management agrees, but notes that this is a concern across the institution that needs institutional analysis and solutions, including revisiting the incentives needed for improved cross-sectoral and cross-departmental collaboration. Management agrees, subject to the limitations mentioned previously related to M&E and attribution. Management action: the HNP sector continues to expand and deepen cross-sector engagement. Recent products include work on Social Safety Nets, Ageing and Demographic Change, Early Childhood Development and Poverty, Environment and Health. Ongoing initiatives include work with Water and Sanitation, Transport, Poverty Reduction and Economic Management, Agriculture and Rural Development, and Operations Policy and Country Services on global challenges ranging from global infectious threats, climate change (part of the 2010 World Development Report), road safety, and an Advanced Market Commitment to develop a pneumococcal vaccine suitable for use in Africa. MAR 2012 Achieving improved health results is a product of health and non-health sectors alike. Multisectoriality, while a challenge, remains critical to achieving results in the health sector.  A working group including education, social protection, environment, transport, and water and sanitation colleagues was convened in February 2011 to address non-health sector inputs to 22 health results. Later in 2011, the group formed a partnership with IHME, which resulted in a number of meetings, consultations and research projects. A paper on cross-sectoral metrics was submitted for publication to the Lancet. A joint seminar was conducted with the EC and European partners and bilaterals. In April 2012, a seminar and workshop will be conducted at the Global Health Forum in Cape Town, South Africa.  HNP is actively engaged with other sectors in the context of climate change, working with the Climate Change Management team to assure HDN input and cross-fertilization within the Bank. HDN, also continues to work with internal and external partners (such as the Institute of Health Metrics and Evaluation in Seattle) to find more appropriate ways of measuring health impact through health and non-health interventions in the Banks portfolio, and benchmarking the achievements of the Bank vis-à-vis other development partners such as the Global Fund, GAVI and bilateral partners.  Since FY12 HDNHE hosts the Global Program for Avian Influenza Control and Pandemic Preparedness. Bank-financed operations under this program integrate actions in animal health (to prevent avian influenza as well as other zoonotic diseases from spreading into the human population), human public health (to improve surveillance and address any human cases), and disaster preparedness and response (to deal with pandemic impact on the health and non- health sectors). The Bank has collaborated with WHO, FAO and OIE to prepare a strategy for "One Health." One Health approaches are being used in operations that follow on the avian flu projects to address critical issues at the interfaces between animal, human, and ecosystem health (e.g., operations in Central Asia and in South Asia). Gaps at these junctures can lead to lapses in disease control and thus warrant more coordination, capacity building, and resources. Though much remains to be done to improve collaboration between veterinary and public health systems, initial advances were made in some 60 countries that have received support from the Bank for their avian and human influenza programs. If these efforts continue, the expected human public health and economic benefits of reducing pandemic risks through control of the pathogens at their animal source are very high. Moreover, as a global public good, control of diseases at their animal source benefits all countries.  HNP continues to solicit the participation of other sectors in QERs, based on project need. Between February 2011 and January 2012, of the 10 HNP Hub-organized 10 QERs, 4 included inputs from non-HNP specialists.  As part of capturing contributions from other sectors, HNP is about to complete, by the end of FY12, a large scale review of non-HNP projects addressing HNP issues (paper titled: "Health in all the Right Places: Supporting Health and the Health Sector through non-Health Sector Lending). The review covers projects approved since FY90 systematically and how the health sector enters DPLs/PRSCs/ILs in over 900 projects in a 20 year period. The findings will be useful in understanding how other sectors‘ work in HNP has evolved over time and how other sectors can be mobilized for HNP outcomes through ongoing and future collaborations. In addition, HNP tracks 100 percent of all non-HNP sector Board operations for core health sector indicators.  As noted above in Section 3.3, the nutrition team continues to actively engage with several sectors (e.g., Agriculture and Rural Development, Social Protection, Education, and PREM) to improve nutrition outcomes. Additionally, HNP has been working closely with the PRMGE in the implementation of the Reproductive Health Action Plan.  Multi- and cross-sectoral work has also been very successful in HNP regions. ECA supports several DPLs (e.g., Armenia, Romania, Croatia, and Tajikistan) in which health sector is playing 23 a significant part in the dialogue between the Bank/IMF and Governments. Another example of multisectoral collaboration for health includes nutrition activities in Tajikistan (with funding from the Global Food Crisis Response Program). SAR has also begun to ramp up cross sectoral collaboration, having recently incorporated a road safety component into a Pakistan roads project, including community level nutrition in the India livelihoods project, and implementing a regional Food and Nutrition Security Initiative with the Agriculture sector. Other examples include the HD PER in Bhutan and the SP-HNP collaboration on health insurance in India and Maldives. Strengthening collaboration with teams working on public sector reform and private sector development has also been a focus in SAR. This includes a special joint initiative with IFC on Health in India.  EAP has provided inputs to other sectors supporting health activities, with a particular view to increase the impact at the community level. For example, it provided inputs to Social Protection on conditional cash transfers in Indonesia and the Philippines.  In LAC, cross-sectoral work is very positively rated in OPEs, which provides incentives for staff to work with other sectors. HNP operations in Brazil, Guatemala, and Panama, to name a few, have strong support to other sector projects with health goals. In Brazil, the multisector SWAps aim to improve HNP outcomes through improvement in public sector management and, in some cases, improvements in other sectors like Water and Sanitation have positive spillovers to the health sector. All HIV/AIDS projects are also multisectoral, engaging sectors such as Education, Transport, and Tourism to improve HNP outcomes. In some HIV/AIDS projects (Barbados), increased selectivity of engagement of other sectors has been important and a key objective with the intent to improve performance and shift the focus to groups at higher risk of infection. Given that several development partners were assisting the Government of Swaziland with HIV/AIDS prevention and treatment in the health sector, the Health, HIV/AIDS & TB Project (P110156) is strengthening of the OVC Safety Net through the Department of Social Welfare under the Deputy Prime Minister‘s Office. Regional Influenza and Nutrition projects are also multisectoral, engaging with the Agriculture sector and, in the case of nutrition, working closely together to improve food security, for example. In Argentina, LAC HNP is assisting the implementation of a Road Safety Project widely known as best practice for its multisectoriality. Finally, LAC HNP also works closely together with the Environment to improve hospital waste management throughout the region (Barbados, Brazil, Costa Rica) and management of the use of pesticides for communicable diseases (Argentina, Brazil). 1.5.Implement the results agenda and improve governance by boosting investment in and incentives for evaluation. MAR 2012 In line with the 2007 HNP Strategy, which called for renewed Bank‘s focus on HNP results, the Bank continues to implement results-based financing (RBF) projects and document lessons learned, leveraging partner resources to support work at the country level.  HNP continues to support RBF activities through the Health Results Innovation Trust Fund (HRITF). This multi-donor TF supports RBF approaches in the health sector for achieving the health-related MDGs, particularly those linked to MDGs 1c, 4 and 5. The governments of Norway and the United Kingdom have committed US$555 million equivalent to the HRITF through 2022. Country Pilot Grants (CPGs) are the centerpiece of the HRITF activities, providing financial support to country programs funded by IDA for RBF projects. They are 24 prepared and supervised within the World Bank‘s operational framework, ensuring regional and country management oversight and rigorous design, implementation, and evaluation support. Twenty CPGs have been approved, of which five are being or have been implemented. The other 15 HRITF supported operations are in varying stages of preparation, and are expected to move into implementation during CY12-13.  The HRITF also provides knowledge and learning grants to 24 countries to gather and disseminate lessons learned about successful RBF programs.  Additionally, the HRITF is contributing to the evidence base on RBF for health through its support of rigorous, prospective impact evaluations on the causal effects, costs, and operational feasibility of RBF. Currently, 22 impact evaluations are in the works that address various types of RBF interventions in different country contexts. Most of the Country Pilot Grants have a rigorous impact evaluation component embedded in the design and implementation of the program to ensure learning. Among the planned impact evaluations, four have a completed baseline, three are currently collecting baseline data, four are at the technical review stage, and a further 10 are under preparation. In 2011, the Bank held its third annual RBF for Health Impact Evaluation workshop. The workshop was attended by representatives of 18 countries implementing impact evaluations, and resulted in the submission of three new impact evaluation designs.  By June 2012, the Bank will finalize a toolkit for Impact Evaluation for RBF projects which build on the publication ―Impact Evaluation in Practice‖ and will provide task teams with the tools they need to successfully implement impact evaluation, including TORs, data collection instruments, and best practice recommendations.  In April 2011, the Bank brought together global stakeholders to a symposium ―RBF for Health: Can it help achieve the MDGs and more?‖ The focus was to encourage global collaboration towards results and develop a shared vision for how RBF for health can help strengthen health systems and meet the MDGs. Over 120 officials from ministries of health and finance, non- governmental organizations (NGOs), donor agencies, foundations, and academic and research institutions from more than 25 countries participated in this dialogue, exchanging views and discussing key lessons learned and challenges met in implementing RBF programs  RBF and RBF-like approaches have shown tremendous success at the regional and country levels. AFR has provided a large amount of technical assistance to countries on RBF, including organization of regional workshops and establishment of a large and very successful ―community of practice‖ on RBF. LAC operations have been successful in utilizing a number of approaches, including RBF (also SWAps and APLS – SWAPLs) in order to increase the focus on results. These projects disburse against achievement of specific, measurable outcomes/outputs, and results are verified by independent technical auditors. Finally, in MENA, a results-based approach to project payments in Yemen has been introduced. In Egypt, use of an independent verification and validation agency has been incorporated into all investment project designs. ECA supports several results-based financing-type operations as stand-alone or as component (e.g., Tajikistan, Kyrgyz Republic, Bosnia & Herzegovina). It also supports, through its products, surveys (e.g., health provider surveys), Health Sector Performance Assessments, and National Health Accounts Report (e.g., Armenia HSPA and NHA are considered best practices by WHO). In Serbia, ―undercover‖ evaluators are recruited to pose as patients in order to assess the quality and performance of health service providers. ECA RBF operations under preparation in Tajikistan and Kyrgyz Republic include a dedicated impact evaluation component. Programmatic assessment of the performance-based incentive scheme in Turkey is 25 being conducted with funding from the HRITF. Incentives for governance and evaluation continue to be critical for improving HNP results.  A regular practice in HNP, the Results Framework and M&E arrangements of all Preparation and Supervision QERs supported by the Hub continue to be reviewed by the M&E specialist in the Hub or by an M&E specialist Panel member. All the 14 projects approved between February 2011 and January 2012 have results frameworks that target HNP outcomes, output, and system performance. Additionally, all HNP projects with first Implementation Status and Results Reports (ISR) between February 2011 and January 2012 had baseline data for at least one PDO indicator.  In EAP, for example, improvements in creating incentives for monitoring and evaluation have been seen in ensuring baseline data availability during the design stage. Utilizing government indicators has helped to ensure ownership of indicators and utilization of indicators as a management tool. Restructuring of various indicators has been supported by management in Lao, Timor Leste, and Vietnam. Finally, a regional evaluation of SWAps in health and education has been undertaken in the Pacific Islands in conjunction with other global partners (Australia, New Zealand, Asian Development Bank).  In SAR, incentives have been strengthened with the use of DLIs (in Uttar Pradesh) and an APL (in the India nutrition project) tying additional financing to results. In Nepal, and more recently in Bangladesh, performance allows for acceleration of disbursements if results are attained. In Afghanistan, a rigorous evaluation is ongoing, to determine whether tying financing to results improves performance of service providers.  IEG RESPONSE 2012 IEG‘s response is based on the evidence provided by management (above) in light of the specific actions and commitments in the World Bank Management Action Plan for the HNP evaluation (pp. 161-164 of the evaluation), in addition to statements of intent from the text of management‘s original response – in cases in which the Action Plan itself does not seem to include actions reflecting those commitments. 1.1.Intensify efforts to improve the performance of the World Bank’s health, nutrition, and population support: substantial. a) ―Match project design to country context and capacity and reduce the complexity of projects in low-capacity settings through greater selectivity and sequencing of activities, particularly in Sub-Saharan Africa.‖ In the Action Plan, management committed to undertake intensive Quality Enhancement Reviews (QERs) in 75% of all new HNP projects, focusing on technical preparation, M&E, and institutional and risk assessments and mitigation measures, beginning in FY10, and to undertake quarterly reviews of the HNP portfolio by the Sector Board. The HNP Sector Board continues to review the Portfolio Improvement Action Plan periodically. Quality Enhancement Reviews were conducted for 13 of the 14 projects approved from February 2011-January 2012. It is reported that all HNP QERs focus on simplifying design and providing guidance to teams on how to match complexity of the operation to the capacity of the country, and in Africa the participation of Cluster Leaders on a number of missions has helped ensure simplicity of project design for both new and 26 restructured projects. Management reports that as a result of intensive support for problem projects, the share of problem projects in the Africa Region HNP portfolio is said to have declined from 25 percent to 11 percent of the projects approved during the above period. That share is now reported to be below the average of all sectors in Africa and among the lowest in the HNP sector. No specific evidence has been provided to show a reduction in complexity or greater selectivity of activities in projects in Africa or other regions. Adoption of this sub- recommendation continues to be high. b) ―Thoroughly and carefully assess the risks of proposed HNP support and strategies to mitigate them, particularly the political risks and the interests of different stakeholders, and how they will be addressed.‖ In the Action Plan, management committed to ensuring that 100% of the QERs focus on risk, beginning in 2010, and that a course would be developed and rolled out on project risk analysis for HNP teams, disseminating best practices and lessons learned, starting in FY10. According to management‘s response, all HNP staff have been encouraged to attend training sessions on the Operational Risk Assessment Framework (ORAF), which is a framework for assessing risk in investment operations that was introduced in 2010, and all QERs are using this tool. In South Asia, this tool is used along with Governance and Accountability Action Plans to identify, mitigate, and assess risks, and some Latin America & Caribbean projects have developed a Governance Anticorruption Council. However, it would be useful to know what share of task leaders have received this training, and the ORAF framework does not explicitly address ―the political risks and interests of different stakeholders and how they will be addressed.‖ Management reports that ―stakeholder analysis (in the context of the politics of health systems reform)‖ is a core component of the new Flagship Course on Health Sector Reform and Sustainable Financing, taken by more than 80 HNP staff. Management reports that stakeholder analysis informed the design of all 14 recently approved projects, but it is unclear whether they included political risk analysis based on the interests of different stakeholders or mitigation strategies . Overall, adoption of this sub-recommendation continues to be substantial. c) ―Phase reforms to maximize the probability of success.‖ In the Action Plan, management committed to increase the analytical work focusing on reform for HNP reform projects, with 100% of all new projects focusing on health system strengthening (HSS) or broadly on health reform to be based in analytical work, including political risks and the interests of different stakeholders, starting in FY10. According to Management‘s response, all 11 new projects approved from February 2011-January 2012 with a significant health system strengthening component are based on institutional analysis, which speaks to the Action Plan commitment. The sector has also carried out a review of the contribution that health systems analysis has made to development of successful health strengthening policies, highlighting evidence of a positive contribution in Mexico, Ethiopia, and Turkey. However, the thrust of this sub- recommendation is about the extent to which health reforms are being phased. Adoption of this sub-objective is rated substantial. d) ―Undertake thorough institutional analysis, including an assessment of alternatives, as an input into more realistic project design.‖ In the Action Plan, management committed to conduct institutional analysis in 80% of all new projects 27 beginning in FY10 and to design and deliver a training/learning program focusing on HNP sector institutional and stakeholder analysis. Management indicates that all 14 new HNP projects are based on an institutional analysis as part of preparation. Institutional analysis is part of the organization section of the Health Reform Flagship course, which reached 80 HNP staff in FY11. It remains unclear the extent to which these analyses are discrete products that take into account institutional incentives and that propose alternatives at the design stage, per the recommendation. Adoption of this sub-objective remains substantial. e) ―Support intensified supervision in the field by the Bank and the borrower to ensure that civil works, equipment, and other outputs have been delivered as specified, are functioning, and are being maintained.‖ In the Action Plan, management committed to specify in project design the borrower’s responsibility for civil works and equipment maintenance in all new HNP projects, beginning in FY10, and that all new HNP projects would be designed to ensure adequate recurrent cost budgeting for civil works and equipment maintenance. The Action Plan‘s commitments are about project preparation rather than the intent of the sub-recommendation, which is intensified supervision of existing as well as new projects. In its 2012 response, Management notes that all 6 new projects with civil works outlined the borrower‘s responsibility and five included standards for civil works. It reports increased supervision in the Africa Region that has reduced the number of problem projects, but it is not clear how this has affected the delivery, functioning, and maintenance of civil works, equipment, and other outputs., Scant information is provided about the extent to which supervision has been intensified in other Regions and Management notes that supervision budgets have been declining over time; no information has been provided on the extent to which civil works and other outputs have been maintained. Adoption of this sub-objective is rated medium. 1.2.Renew the commitment to health, nutrition, and population outcomes among the poor: Substantial a) ―Boost population and family planning support in the form of analytic work, policy dialogue, and financing to high-fertility countries and countries with pockets of high fertility.‖ In the Action Plan, management committed to writing an analytic and advisory activity policy note on reproductive health, including family planning (FP), by FY10; to incorporate FP into health system strengthening projects, delivering 2 HSS projects in high fertility countries that include strengthening of FP in FY10; and to incorporate population and family planning into half of CASs for high-fertility countries, beginning in FY10. Implementation of the 2010 five-year Reproductive Health Action Plan began in FY11; it targets 57 countries, including those with high fertility or high material mortality. Reproductive Health profiles were completed for 47 priority countries and disseminated; it is reported that this material was incorporated into project documents, policy dialogue, and CASs. Management reports that all six countries with high maternal mortality and/or total fertility that had new CASs or Interim Strategy Notes over the past year ―incorporated RH issues.‖ The response indicates many initiatives on reproductive health and on MDGs 4 and 5, sometimes including HIV/AIDS and maternal and child health in general, but provides little information on incorporation of policies concerning the demand or service delivery for family planning into CASs, health system strengthening projects, and analytic work for high-fertility countries or for countries with pockets of high fertility, which is specifically the subject of the sub-recommendation and the Action Plan. An exception is that reproductive 28 health projects in Mali and Burkina Faso (both high fertility) are reported to focus on access to family planning by the poor, and include the contraceptive prevalence rate as an outcome indicator. No information has been provided on the results of the Africa Region‘s Population Policy Note mentioned in last year‘s MAR – whether it was concluded, what it recommended, and what has been achieved. While there appeared to have been substantial progress on this sub-recommendation in the previous MAR, in light of the lack of evidence of continued work on family planning in high-fertility countries or evidence that any of the policy changes from past years have found their way into country strategies and the portfolio of lending and analytic work in high-fertility countries, adoption of this sub-recommendation is rated medium. b) ―Incorporate the poverty dimension into project objectives to increase accountability for health, nutrition, and population outcomes among the poor.‖ The Action Plan committed to ensure that ―adequate attention is given to the poverty dimensions in project design and supervision, particularly project development objectives and key performance indicators;‖ the target was for 80 percent of all new HNP projects to incorporate the poverty dimensions ―where appropriate‖ beginning in FY10. Management reports that 12 of the 14 newly approved projects have the poor as the main beneficiaries (~85%) and 9 of them (64%) explicitly include the poor in the project development objectives or key performance indicators. Three other projects target the poor by focusing on poor areas. The Africa Region has reportedly established a baseline on the share of projects that explicitly set out to measure equity in access or outcomes and has incorporated equity concerns in its results chain, but the baseline statistics and available evidence on progress have not been reported. Management notes that all new operations in the South Asia Region include indicators that track the impact on poor or marginalized groups. In the absence of more specific information to document evidence of progress, adoption of this sub-objective continues to be rated substantial. c) ―Increase support to reduce malnutrition among the poor, whether originating in the HNP sector or other sectors.‖ The Action Plan committed to scale-up the Bank’s analytic and investment work and to leverage resources from other donors, by hiring 6 additional Bank nutrition staff, mobilizing additional trust fund money for nutrition from Japan and others; developing a Global Action Plan with key partners by FY10; and conducting 6-8 analytical and advisory activities or new investments in nutrition by FY11. Management cites a number of global, regional, and country-level initiatives aimed at reducing malnutrition, following on the hiring of additional nutritional staff (mostly in the field) and the Nutrition Global Action Plan from last year. Several of these highlight the contributions of multiple sectors to nutrition outcomes – the nearly launched knowledge platform SecureNutrition: Linking Agriculture, Food Security, and Nutrition and program guidance briefs on agriculture-food security-nutrition, health-nutrition, and social protection-nutrition linkages. Trust fund resources from Japan have been procured, as previewed in the Action Plan. It is reported that an increasing number of projects in the Africa Region include nutrition-related interventions, although no specific evidence is provided. An un-named piece of economic and sector work in Tanzania on food fortification is reported to have strongly influenced a decision by the Tanzanian government to institute mandatory fortification of wheat flour, maize four, and edible oil – the first country to do so in East Africa. Initiatives in Central America have conducted a regional workshop and follow-on activities on community-based nutrition programs. Since FY11, 18 additional pieces of analytical and advisory work and 20 new operations addressing nutrition and food security are reported to have been delivered or 29 approved. A continued concern (not addressed in the action plan, but part of the recommendation) is the extent to which these activities to reduce malnutrition among the poor. Overall adoption of this sub-recommendation remains high. d) ―Monitor health, nutrition, and population outcomes among the poor, however defined.‖ The Action Plan commits to tracking health outcomes and coverage of interventions among the poor by publishing an annual review of health indicators among the poor, starting in FY10, with the responsibility in the HNP Hub. Management reports on the development of a comprehensive database on HNP outcomes and coverage by wealth quintile, additional tools, discussion papers, and training. These are important achievements. However, the evaluation‘s sub-recommendation was based on the fact that very few projects or countries were monitoring health outcomes among the poor – an issue for country teams and operational staff, not a call for more analytic work at the center. Management‘s original response to the HNP evaluation also acknowledged that point (p. 157): ―Management agrees with the need to ensure that project design responds to the priorities and needs of the poor, and to measure the full impact of improved health services for the poor.‖ In last year‘s MAR, IEG noted that it ―would be useful if management could provide some statistics in this regard for next year‘s MAR.‖ As it did in the previous MAR, Management reports that all new operations in South Asia Region include indicators that track impact on poor or marginalized groups. However, no additional systematic information has been provided about the extent to which this sub-recommendation has been adopted and implemented in ongoing projects across the board in HNP operations. Until then, there remains an important agenda in monitoring HNP outcomes among the poor in the lending portfolio. Adoption of this sub-recommendation remains medium. e) ―Bring the health and nutrition of the poor and the links between high fertility, poor health, and poverty back into poverty assessments in countries where this has been neglected.‖ The Action Plan committed to increase inclusion of HNP in poverty assessments, with the target of 90% of all poverty assessments and 40% of all CASs assessing the health status of the poor, beginning in FY10. This year‘s response reports that all three poverty assessments issued from February 2011-January 2012 addressed health, and in the case of Niger, the assessment addressed high fertility and malnutrition. Of the 23 CASs approved in this period, 19 included specific health outcomes, but the extent to which the link between high fertility, poor health, and poverty has been addressed in these CASs is unclear. IEG rates adoption of this sub-recommendation as medium.. 1.3.Strengthen the World Bank Group’s ability to help countries to improve the efficiency of their health systems: Substantial a) ―Better define the efficiency objectives of its support and how efficiency improvements will be improved and monitored.‖ The Action Plan committed to expanding and making more explicit the PAD definition and discussion of efficiency objectives, measures, and monitoring framework, with 70% of HNP projects to include definition and analysis of improving sector efficiency, including discussion of the efficiency-equity trade- off beginning in FY10. It also committed to launch AAA tasks to review experience in improving health system efficiency. The 2012 MAR reports that 12 of the 14 recently approved projects ―addressed‖ 30 efficiency in different ways and that all of them have indicators of efficiency in their results frameworks, but it does not provide an update on the share of recently approved projects with explicit objectives to improve efficiency (last year it was 4 of 15 approved since March 2010, or 27%). There is also no mention of the status of the proposed AAA to review experience on health system efficiency. A discussion paper on ―Health Systems Analysis for Better Health System Strengthening‖ was published in May 2011, but no evidence is provided of its use to improve the efficiency of health systems in the lending program. Management reports that results-based financing has been incorporated in a number of operations, though it is not clear how many operations or the extent to which this has been used to improve health system efficiency. The efforts to reduce transaction costs among donors or other international initiatives cited in the response do not address the sub-recommendation, which is about efficiency objectives in the lending portfolio. Adoption is rated medium. b) ―Carefully assess decisions to finance additional earmarked communicable disease activities in countries where other donors are contributing large amounts of earmarked disease funding and additional funds could result in distortion in allocations and inefficiencies in the rest of the health system.‖ The Action Plan committed to coordinate proposals for new disease-specific programs with other partners, mapping all HNP projects with significant priority-disease components with contributions from other donors, and to ensure strengthening of health systems, starting in FY10. Management provided in the previous MAR substantial evidence of efforts to address the balance between earmarked disease funding and health systems development, taking into account support from other donors. This year‘s MAR notes that in Nepal, Bangladesh, and Pakistan, HIV/AIDS components have been financed as part of broader health initiatives. In Swaziland, the Health, HIV/AIDS, and TB Project supports both communicable disease control and the health system; the AIDS component supports social protection of orphans and vulnerable children, in light of the contribution of other donors. Adoption of this sub-objective is rated substantial. c) ―Support improved health information systems and more frequent and vigorous evaluation of specific reforms or program innovations to provide timely information for improving efficiency and efficacy.‖ The Action Plan committed to building statistical capacity in countries on priority HNP outcome indicators directly through Bank operations and/or supporting global partners’ country support, with half of new HNP projects to include strengthening of country M&E systems by FY10. It also advocated continuing to support the IHP+ efforts to strengthen M&E and health information systems, specifically developing a with global partners a strategy for global monitoring by FY09, and conducting assessments of health information systems in 10 countries by FY10. Management reports that 13 out of 14 new projects include an M&E component. As reported in the 2010 MAR, assessments of health information systems have been conducted in 80 countries and under the IHP+ the global health partners have developed a common M&E framework. Management points out that results- based financing is heavily reliant on verifiable performance information and independent verification; it would be important to understand how many recent operations have adopted this approach and whether the necessary health information system investments have been supported and are adequate in practice to make this approach work. The Action Plan itself does not respond to the intent of the sub-recommendation to ensure vigorous and ongoing evaluation of specific reforms or program innovations. Management notes that impact evaluations have been supported in a number of projects supporting results-based financing, 31 but there is no specific information on the incorporation of more frequent and vigorous evaluation of specific reforms or innovations to improve management decisions around efficiency and efficacy. Adoption of this sub-recommendation remains substantial. 1.4.Enhance the contribution of support from other sectors to health, nutrition, and population outcomes: Medium. Management‘s response has given many examples of collaboration by HNP staff with staff from other sectors, but the HNP evaluation and the intent of this recommendation was to advocate greater accountability and results from the health investments of other sectors (like water supply and sanitation) and greater synergy between projects in different sectors in the field. . a) ―When the benefits are potentially great in relation to the marginal costs, incorporate health objectives into non-health projects, for which they are accountable.‖ The Action Plan committed to provide incentives to non-HNP task team leaders to incorporate health objectives into non-health projects and to set up an intersectoral coordination thematic group for HNP results to identify constraints and incentives by FY10. The Action Plan had no targets with respect to health objectives in non-health projects, and did not suggest focusing on any particular sectors. The 2012 MAR makes no mention of the extent to which health objectives have been incorporated into relevant projects managed by other sectors, such as water supply and sanitation projects. Examples are offered of projects in other sectors that have added components on road safety (Pakistan and Argentina) and community-level nutrition in a livelihoods project (India), and of cross-sectoral collaboration on major initiatives like the Global Program for Avian Influenza Control and Pandemic Preparedness and the Global Food Crisis Response Program (both with the agriculture sector). Only for the Latin America and Caribbean region is there any mention of incentives for task team leaders to work cross-sectorally, through annual staff performance assessments. As reported in the previous MAR, a working group was set up in February 2011 to address non-health sector inputs into health results. The working group has reportedly sponsored several meetings, consultations, and workshops; a paper on cross-sectoral metrics was submitted to The Lancet. The adoption of this sub-recommendation remains medium, primarily because of a lack of systematic information on the extent to which projects in other sectors like water supply and sanitation have embraced and been made accountable for relevant health outcomes. Management reports that a paper will be completed before the end of FY12 to review projects in other sectors approved since FY90 that have addressed HNP issues. This study should reveal the extent to which health objectives have been included has increased. b) ―Improve the complementarity of investment operations in health and other sectors to achieve health, nutrition, and population outcomes, particularly between health and water supply and sanitation.‖ The Action Plan committed to develop, implement, and manage an intersectoral coordination thematic group for HNP results (same as (4a) above) by FY10. There is no mention of the results of the working group mentioned in the previous MAR that was set up in February 2011 to address non-health sector inputs into health results No evidence is provided of systematic efforts to improve complementarity of operations to improve HNP outcomes at the country level, particularly with water supply and sanitation. Initiatives on Avian Influenza and the Food Crisis Response have resulted in greater collaboration between HNP, agriculture, and social protection, although the effectiveness of this collaboration has not yet been assessed. Adoption of this sub-recommendation is rated medium. 32 c) ―Prioritize sectoral participation in multisectoral HNP projects according to the comparative advantages and institutional mandates, to reduce complexity.‖ The Action Plan committed to invite other sectors’ participation to HNP project design reviews (for example, Quality Enhancement Reviews), where appropriate, for all HNP projects starting in FY10. Management reports that 4 of 10 QERs conducted by the Hub included inputs from non-HNP specialists. However, this recommendation is not about engaging more sectors in the design of HNP projects. Rather, it is about reducing the complexity of multisectoral HNP projects by prioritizing a smaller number of sectors that are strategically the most important. The 2012 MAR mentions only one example of a multisectoral AIDS project (Barbados) that incorporated increased selectivity of the engagement of other sectors, with the intent to improve performance and shift the focus to groups at higher risk of infection. It would be sufficient to show, for example, that HNP projects in the countries with the least capacity have embraced fewer priority sectors. In the absence of additional information, adoption of this recommendation is rated negligible. d) ―Identify new incentives for Bank staff to work cross-sectorally for improving HNP outcomes.‖ The Action Plan referred back to the incentives proposed to sub-recommendation 4a on the intersectoral committee. As noted above, no additional information has been provided on the results of the intersectoral committee. As was the case in last year‘s MAR, management notes that in the Latin America and Caribbean region, cross-sectoral work is positively rated in annual performance assessments of HNP staff and there is evidence of strong support from HNP staff in LAC to projects in other sectors. As mentioned in last year‘s MAR, it would be important to demonstrate in the next MAR the incentives for staff in other sectors to address health outcomes in their projects. Adoption is rated medium. e) ―Develop mechanisms to ensure that the implementation and results for small health components retrofitted into projects are properly documented and evaluated.‖ The Action Plan commits to monitor health results achieved by the portfolio managed by non-health sectors, with the target to track half of all HNP and non-HNP Sector Board operations, starting in FY10. Management notes that as part of an effort to capture contributions from other sectors, the HNP anchor tracks all non-HNP sector board operations for core health sector indicators. However, this sub-recommendation arises from a finding of the HNP evaluation that small retrofitted health components in projects in other sectors are not being monitored or evaluated, and no evidence is presented in the 2012 MAR that addresses this point. Adoption of this recommendation is rated negligible. 1.5.Implement the results agenda and improve governance by boosting investment in and incentives for evaluation. Specifically, the recommendation was to ―create new incentives for monitoring and evaluation for both the Bank and the borrower linked to the project approval process and the midterm review. This would include requirements for baseline data, explicit evaluation designs for pilot activities in project appraisal documents, and periodic evaluation of main project activities as a management tool.‖ Substantial The Action Plan commits to: (i) support impact evaluations of at least 15 HNP projects in FY 11 through the Spanish Impact Evaluation Trust Fund (SIEF); (ii) pilot and evaluate the impact of output- and 33 performance-based financing for HNP-related projects/programs in 16 projects, with most loan proceeds for input-based finance; and (iii) introduce Results Frameworks targeting HNP outcomes, output, and system performance, reaching at least 70% of new projects approved by the Board beginning in FY09. The bulk of the 2012 MAR on this recommendation presents progress on implementing results- or output-based financing in the HNP lending portfolio, an approach that requires robust monitoring systems. There are 22 impact evaluations in progress or about to be launched that address types of results-based financing in different contexts, four of which have completed a baseline and 10 of which are under preparation. As noted last year, all QERs supported by the HNP hub are reviewed by an M&E specialist with respect to the results framework and M&E arrangements. In East Asia, management notes that the requirement to ensure baseline data during the design stage has improved incentives for monitoring and evaluation. It isn‘t clear whether the target on Results frameworks (70% of new projects) has been achieved. Management reports that all projects that had their first ISRs between February 2011 and January 2012 had baseline data for at least one outcome indicator. While progress continues to be substantial, it is not known whether these important initiatives flagged in the Action Plan have changed the incentive structure for evaluation, which is the thrust of the recommendation. Results-based financing can generate incentives for monitoring; it remains to be seen whether it creates incentives for greater evaluation. Adoption of this recommendation is substantial. Original HNP Action Plan: How Much & By IEG Recommendation Actions to be Taken When By Whom I. Intensify efforts to improve the performance of the World Bank‘s health, nutrition, and population support. (a) Match project design to -Carry out QERs - 75% of all new HNP HNP Sector country context and capacity focusing on technical projects have an Board, HNP and reduce the complexity of preparation, M&E, intensive QER focusing Hub, and projects in low-capacity and institutional and on technical regional settings, through greater risk assessments and preparation, M&E, and quality teams. selectivity, prioritization, and mitigation measures. institutional and risk sequencing of activities, assessments and particularly in Sub-Saharan mitigation measures, Africa. starting FY-10. -Reviews of HNP -Quarterly reviews of portfolio. HNP portfolio by HNP Sector Board ongoing. 34 How Much & By IEG Recommendation Actions to be Taken When By Whom (b) Thoroughly and carefully -Concentrate on risk -Of the QERs HNP Sector assess the risks of proposed management and conducted, 100% Board, HNP HNP support and strategies mitigation during include focus on risk, Hub, WBI to mitigate them, particularly QERs. starting FY-10. the political risks and the -Expand learning on -Develop and roll out interests of different HNP sector risk course on project risk stakeholders, and how they assessments and analysis for HNP will be addressed. mitigation strategies. teams, and disseminate best practices and lessons learned, starting FY-10. (c) Phase reforms to -Increase analytical -100% of new HNP HNP Sector maximize the probability of work focusing on projects focusing on Board, with success. reform for those health system technical HNP projects strengthening or support from focusing on health broadly on health HNP Hub, as system reform. reform to be based on needed. analytical work, including political risks and the interests of different stakeholders, starting FY-10. (d) Undertake thorough -Increase AAA for -80% of new HNP HNP Sector institutional analysis, institutional projects to be based on Board, HNP including an assessment of analysis, with institutional analysis, Hub. alternatives, as an input into increased attention starting FY-10. more realistic project design. through better policy dialogue and analytical work, adapted to country context. -Training program -Learning program designed and focusing on HNP delivered, starting FY- sector institutional 10. and stakeholder analysis. 35 How Much & By IEG Recommendation Actions to be Taken When By Whom (e) Support intensified -Project design to -All new HNP projects -HNP Sector supervision in the field by specify borrower starting preparation in Board, HNP the Bank and the Borrower to responsibilities for FY-10. Hub. ensure that civil works, civil works and equipment, and other equipment outputs have been delivered maintenance. as specified, are functioning, - All new HNP -Project design to and being maintained. projects, starting ensure adequate recurrent cost preparation in FY-10. budgeting for civil works and equipment maintenance. -Develop guidelines -By FY-10. and standard specifications for civil works and equipment and other health inputs, and their maintenance. II. Renew the commitment to health, nutrition, and population outcomes among the poor. (a) Boost population and -AAA policy note on -By FY-10. HNP Sector family planning support in the reproductive health, Board, HNP form of analytic work, policy including family Hub. dialogue, and financing to planning. high-fertility countries and -2 HSS projects in high -Incorporate family countries with pockets of fertility countries planning into HSS high fertility. include strengthening projects. of family planning delivery, by FY-10. -50% of CASs for high -In high fertility fertility countries, countries, starting FY-10. incorporate population and family planning issue into CAS. 36 How Much & By IEG Recommendation Actions to be Taken When By Whom (b) Incorporate the poverty -Ensure adequate -80% of all new HNP HNP Sector dimension into project attention is given to projects incorporate the Board, with objectives to increase poverty dimensions poverty dimension, technical accountability for health, in project design and where appropriate, support from nutrition, and population supervision, starting FY-10. HNP Hub, as outcomes among the poor. particularly PDOs needed. and KPIs. (c) Increase support to -Scale-up the Bank‘s -President's Regional HNP Sector reduce malnutrition among analytical and Reprioritization Fund Board, HNP the poor, whether originating investment work to hire 6 additional Hub. in the HNP sector or other and leverage Bank staff (US$4 sectors. resources from other million committed for donors. FY-09-11); Japan TF (US$2 million with potential for additional US$20 million); possibly funds from other donors that are currently engaged in discussions on this issue. -Global Action Plan designed and agreed with key partners, by FY-10. -Six to eight AAAs or new investment in nutrition by FY-11. (d) Monitoring health, -Track health -Annual review of HNP Hub nutrition, and population outcomes and health indicators outcomes among the poor, intervention among the poor, however defined. coverage among the starting FY-10. poor. -Annual report, -Publish report on starting FY-10. health indicators of poor people. 37 How Much & By IEG Recommendation Actions to be Taken When By Whom (e) Bring the health and -Increase inclusion -90% of all poverty HNP Sector nutrition of the poor and the of HNP in poverty assessments and at Board, with links between high fertility, assessments. least 40% of all CASs technical poor health, and poverty should assess the support from back into poverty health status of the HNP Hub and assessments in countries poor, starting FY-10. PREM, as where this has been needed. neglected. III. Strengthen the World Bank Group’s ability to help countries to improve the efficiency of their health systems. (a) Better define the -Expanded PAD -70% of HNP projects HNP Sector efficiency objectives of its definition and to include definition Board, HNP support and how efficiency discussion of and analysis of Hub. improvements will be efficiency objectives, improving HNP sector improved and monitored. measures and efficiency, including monitoring discussion of framework to be efficiency-equity trade- expanded and more off, starting FY-10. explicit. -Start FY-10. -AAA to analyze and review experience in improving health system efficiency. (b) Carefully assess decisions -Closely coordinate -100% of HNP projects HNP Sector to finance additional proposals for Bank with significant Board, with earmarked communicable support for new priority disease technical disease activities in countries disease specific components to map support from where other donors are programs with other contributions from HNP Hub, as contributing large amounts partners. other donors and needed. of earmarked disease ensure strengthening of funding and additional funds health systems, starting could result in distortion in FY-10. allocations and inefficiencies in the rest of the health system 38 How Much & By IEG Recommendation Actions to be Taken When By Whom (c) Support improved health -Build statistical -50% of new HNP HNP Sector information systems and capacity for client projects include Board, HNP more frequent and vigorous countries on priority strengthening of Hub. evaluation of specific reforms HNP outcome country M&E systems, or program innovations to indicators directly by FY-10. provide timely information through Bank for improving efficiency and operations and / or efficacy. supporting global partner‘s country support (e.g., MDGs). -Continue support of -Strategy for global the International monitoring Health Partnership‘s arrangement designed (IHP+) efforts to (in collaboration with strengthen global partners), by FY- monitoring and 09. evaluation and health information systems in countries. -Conduct country assessments of -Country assessments health information in 10 countries in FY- systems. 10. IV. Enhance the contribution of support from other sectors to health, nutrition, and population outcomes. 39 How Much & By IEG Recommendation Actions to be Taken When By Whom (a) When the benefits are -Provide incentives -Intersectoral HNP Sector potentially great in relation to non-HNP TTLs to coordination thematic Board, HNP to the marginal costs, incorporate health group for HNP results Hub, Country incorporate health objectives objectives into non- established to identify Directors. into non-health projects, for health projects. constraints and which they are accountable. incentives, by FY-10. (b) Improve the -Develop, -Group Functioning by HNP Sector complementarity of implement, and FY-10. Board, HNP investment operations in manage an Hub. health and other sectors to intersectoral achieve health, nutrition, and coordination population outcomes, thematic group for particularly between health HNP results. and water supply and sanitation. (c) Prioritize sectoral -Invite other sectors‘ -100% of all HNP HNP Sector participation in multisectoral participation to HNP projects, starting FY-10. Board. HNP projects according to project design the comparative advantages reviews (e.g., QERs) and institutional mandates, where appropriate. to reduce complexity. (d) Identify new incentives -See response to IV -See response to IV (a) -See response for Bank staff to work cross- (a) above. above. to IV (a) sectorally for improving above. HNP outcomes. 40 How Much & By IEG Recommendation Actions to be Taken When By Whom (e) Develop mechanisms to -Strengthen HNP -50% of all HNP and HNP Sector ensure that the portfolio non-HNP Sector Board Board, HNP implementation and results monitoring, operations tracked, Hub. for small health components including non-HNP starting FY-10. retrofitted into projects are projects, to properly documented and document health evaluated. results achieved through non-HNP sectors. V. Implement the results agenda and improve governance by boosting investment in and incentives for evaluation 41 How Much & By IEG Recommendation Actions to be Taken When By Whom (a) Create new incentives for -Implement US$2.8 -15 HNP projects, FY- HNP Sector monitoring and evaluation million Spanish 11. Board, HNP for both the Bank and the Trust Fund (SIEF), Hub. -16 active projects with borrower linked to the supporting impact most loan proceeds project approval process and evaluations. allocated to output- the mid-term review. This -Pilot and evaluate based financing, FY-10. would include requirements impact of output- for baseline data, explicit -At least 70% of new and performance- evaluation designs for pilot projects / programs based financing for activities in Project Appraisal approved by the Board, HNP-related Documents, and periodic starting FY-09. projects/p evaluation of main project activities as a management -Introduce Results tool. Frameworks targeting HNP outcomes, output, and system performance, including baseline data and output targets and programs. http://siteresources.worldbank.org/EXTWBASSHEANUTPOP/Resources/appJ.pdf 42 2. How Effective Have Poverty and Social Impact Analyses Been? An IEG Study of World Bank Support to PSIAs LIST OF RECOMMENDATIONS 2.1. Ensure that staff understand what the PSIA approach is and when to use it by providing clear guidance (perhaps through updating of the 2008 PSIA Good Practice Note) and actively disseminating this guidance, particularly on—  Whether and how the PSIA approach differs from other distributional analyses, including whether the inclusion of the word ―social‖ in Poverty and Social Impact Analysis suggests the need to include a different type of analysis  Whether or not PSIAs should be linked to specific reforms and identify beneficiaries and those adversely affected by the reform  What criteria should be used to determine when the PSIA approach is appropriate for a particular operation in a country program. 2.2. Clarify the operational objectives of each PSIA with regard to its intended effect and tailor the approach to those objectives, ensuring that the concept note  Contains a clear statement of the operational objectives of the PSIA with respect to the intended effect (not just the topics/issues to be analyzed)  Indicates how its approach, in particular stakeholder engagement, team composition, partner institutions, budget, and time frame, have been tailored to meet the operational objectives, and provided the rationale for the choices made  Shows how any tensions and trade-offs among the operational objectives will be reconciled  Discusses if the intended dissemination audience and strategy are consistent with the stated operational objectives. 2.3. Improve integration of the PSIA into the Bank’s country assistance program by—  Shifting decision-making and funding authority to the Regions to ensure that the PSIA topics, scope, and approach are consistent with the country assistance program and that PSIAs ask policy-relevant questions  Requiring that all earmarked funding for PSIAs be matched by a substantial contribution from the country unit budget. 2.4. Strengthen PSIA effectiveness through enhanced quality assurance, including—  Subjecting PSIAs to systematic review by Regional management at the concept and completion stages to ensure relevance and fit of the PSIA to the country assistance program and consistency of the proposed approach with operational objectives, in addition to ensuring technical quality  Ensuring that the Bank establishes a monitoring and self-evaluation system designed to assess if PSIAs are being undertaken where appropriate and are achieving their stated operational objectives. 43 STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG Ensure that staff understand what the PSIA approach is and when to use it. Bank management can do this by providing clear guidance (perhaps through updating of the 2008 PSIA High Substantial Good Practice Note) and actively disseminating this guidance. Clarify the operational objectives of each PSIA with regard to High Medium its intended effect and tailor the approach to those objectives. Improve integration of the PSIA into the Bank‘s country High Substantial assistance program. Strengthen PSIA effectiveness through enhanced quality Substantial Medium assurance. MANAGEMENT RESPONSE 2012 2.1. Original Response: Agreed. Management agrees that it should disseminate more widely what is meant by the PSIA approach as outlined in the 2008 Good Practice Note. Management, however, notes an important caveat in addressing this recommendation. Efforts to expand awareness about the PSIA approach must be careful to not create perceptions of minimum or universal standards for PSIA, and instead stress the flexibility of the approach to adapt to specific country and policy contexts. The revised Good Practice Note (GPN) discusses key elements of the PSIA approach: first, an analysis to determine the distributional impacts and, second, a process that engages appropriate stakeholders in policy making.3 The GPN also makes it clear that an ideal PSIA will vary based on country and reform-specific conditions. In particular, the range and extent of stakeholder engagement in the PSIA process will vary according to the political context of the reform and the related opportunities for promoting the public debate on policy options. The GPN clarifies when it could be a priority for a country team to carry out detailed PSIA: when there are potential negative poverty and social impacts on different groups, particularly poor and vulnerable groups; when there is potential to significantly improve the benefits of a reform for poor and vulnerable groups; the prominence and urgency of the report in the government‘s policy agenda; and the level of debate surrounding the reform. With regard to the methods and tools adopted for PSIA, the GPN stresses the need for flexibility as well. While multidisciplinary or mixed methods of analysis enhance the understanding of the poverty and social impacts of a reform, the design of the methodology and the selection of tools will depend on the nature of the reform problem being addressed, the availability and quality of data, the time available for analysis, and the available in-country capacity. PREM and SDV are already incorporating the revised GPN into their PSIA-related learning programs (Fundamentals of Bank Operations, PSIA e-learning, PSIA course). Management will further enhance outreach using these materials through an improved Web site that more clearly frames the main objective and potential benefits, as well as the need for a flexible approach, and more 44 frequent and accessible learning events, drawing on additional resources provided by the aforementioned Multi-Donor Trust Fund. Management will consider its agreed action complete with the full roll-out of these activities in fiscal 2010. MAR 2012: Management agreed with this recommendation and considers that it has been completed with the full roll-out of the following activities that it agreed to undertake in fiscal 2010, as indicated in its Management Response. Guidance on the PSIA approach has been revised, updated, and disseminated to staff throughout the Bank‘s relevant learning programs and PSIA website. The Good Practice Note on PSIA has been revised by PREM and SDV and incorporated into their PSIA-related learning programs (such as Fundamentals of Bank Operations, PSIA e-learning, and PSIA Course). OPCS‘s DPL Academy includes a dedicated session on how to use poverty and social impact analysis in the context of Development Policy Lending. Furthermore, the PSIA website hosted by PREM has also been significantly improved to enhance the outreach of these materials and more often and accessible learning activities have been offered to staff. Management rates the adoption of this recommendation as High. 2.2. Original Response: Agreed. Management agrees that poverty, social, and distributional impact analysis should have a clear operational objective and a methodology and strategy for stakeholder engagement and dissemination that is consistent with the operational objective. Almost all PSIA type analysis is done in the context of economic and sector work (ESW), where Bank guidelines require that the operational objective, scope, and participatory approaches be clearly specified in the concept note. The revised GPN Indicates that poverty, social, and distributional analysis should be an integral part of the ESW cycle. All dissemination activities need to be fully consistent with the Bank‘s Disclosure Policy. PSIA can also be done in the context of technical assistance, where the objective is to strengthen client institutions and capacity to influence reforms or as a factual technical document analyzing a specific policy reform supported by a Development Policy Operation. Once operational, the new PSIA trust fund will require that the Regions detail in their concept notes how they will ensure that poverty, social, and distributional analysis (supported by the Trust Fund) is treated as an ESW, technical assistance, or a project related factual technical document. Management will consider its agreed action complete with the introduction of these requirements. MAR 2012: Management agreed with this recommendation and considers that it has been completed with the full roll-out of the following activities that it agreed to undertake in fiscal 2010, as indicated in its Management Response. In FY10, the Bank set up a new Multi-Donor Trust Fund (MDTF) to support PSIA work and assigned responsibility to the Regions, in terms of allocating funds, monitoring and quality review of the analytical work funded by the Trust Fund. The MDTF requires that the Regions detail in their concept notes how they will ensure that poverty, social and distributional impact analysis (supported by the TF) is treated as ESW, TA or a FTD. Management rates the adoption of this recommendation as High. 2.3. Original Response Agreed. Poverty, social, and distributional impact analysis that is part of ESW, technical assistance, or a project-related factual technical document is currently managed and funded by the Regions. The revised GPN explicitly advises that poverty, 45 social, and distributional impact analysis be anchored in the Country Assistance Strategy (CAS). Under the revised policy for poverty reduction (OP 1.00), the CAS summarizes existing knowledge on poverty, identifies analytical gaps, and presents the work program by the Bank and others to fill these gaps. The CAS can usefully lay out key reform areas that the Bank will support and indicate whether there are any plans for poverty, social, and distributional impact analysis. The new PSIA trust fund will decentralize the management of resources for poverty, social, and distributional impact analysis carried out as ESW, technical assistance, and factual technical documents to the Regions. The Regions will be required to provide matching resources from the country unit budget. The annual Trust Fund Regional monitoring reports will indicate the share of matching resources provided to complement Trust Fund support. Management will consider its agreed actions complete with the roll-out of the trust fund. MAR 2012 Management agreed with this recommendation and considers that it has been completed with the full roll-out of the following activities that it agreed to undertake in fiscal 2010, as indicated in its Management Response. The Good Practice Note on PSIA has been revised and explicitly urges poverty, social and distributional impact analysis to be anchored in the CAS. The CAS Retrospective (Country Assistance Strategy: Retrospective and Future Directions) concluded by Management in FY10 confirmed that CASs, particularly in IDA countries, have included a comprehensive discussion and diagnosis of poverty, covering poverty incidence, trends, causes and correlates with links to the MDGs. In addition, the new PSIA MDTF has decentralized the management of resources for poverty, social and distributional impact analysis carried out as ESW, TA, and FTD to the Regions, which are required to provide matching resources from the country unit budget. Management rates the adoption of this recommendation as High. 2.4. Original Response Agreed. PSIA work that is part of ESW is subject to Regional quality assurance procedures and the preparation of an activity completion report. As noted in the Management Response to IEG‘s evaluation of ESW and technical assistance, management will undertake a review of analytic and advisory activities in fiscal 2010 that will address institutional arrangements, notably quality assurance. The quality and effectiveness of PSIA informing the Bank‘s Development Policy Operations has been monitored through the retrospective assessment of the Bank‘s Development Policy Lending portfolio that has been carried out every two years by Operations Policy and Country Services. A Development Policy Operation retrospective is under preparation and will be available early in fiscal 2010. The proposed PSIA trust fund annual Regional monitoring reports as well as the final independent Trust Fund evaluation will assess the extent to which poverty, social, and distributional analysis (supported by the Trust Fund) has met its operational objectives and has had an impact on the ground. Management will consider its agreed actions complete with the completion of the fiscal 2010 analytic and advisory activities review and the implementation of changes coming out of that process, the issuance of the Development Policy Operation retrospective, and the implementation of the trust fund reporting system. MAR 2011 As indicated in the Management Response, Management had agreed to implement 46 this recommendation through the following actions: (a) the completion of the FY10 AAA review broadly covering analytic and advisory activities and the implementation of changes coming out of that process; (b) the issuance of its DPL Retrospective; and (c) the implementation of the PSIA MDTF reporting system. Actions (a) and (b) have been completed, while action (a) is expected to be completed in FY11. The AAA Review will be completed in FY11 and will propose revised systems and procedures for analytic and advisory activities. The findings and recommendations of the AAA Review will be incorporated in Management‘s Knowledge Strategy and shared with CODE in the context of the discussion on the Knowledge Strategy in FY11. Hence, Management no longer intends to present the AAA Review as a separate product to CODE, as indicated in the 2010 MAR. Management rates, therefore, the implementation of this recommendation as Substantial. IEG RESPONSE 2012 IEG notes Management‘s response that it has revised The Good Practice Note on PSIAs and incorporated it into PSIA-related learning programs (such as Fundamentals of Bank Operations, PSIA e-learning, and PSIA Course), OPCS‘s DPL Academy now includes a dedicated session on how to use poverty and social impact analysis in the context of Development Policy Lending, the PSIA website has been improved to enhance the outreach of these materials, and more often and accessible learning activities have been offered to staff. As indicated in the previous MAR, a higher rating would call for data on: the proportion of relevant staff (those currently undertaking/reviewing or likely to undertake/review PSIAs or other forms of distributional analysis) availing of the learning programs/web-based resource materials; and views of those staff regarding the usefulness of the guidance in understanding what PSIAs are and when to use them. IEG rates the level of adoption as Substantial. IEG notes Management‘s response that ―In FY10, the Bank set up a new Multi-Donor Trust Fund (MDTF) to support PSIA work and assigned responsibility to the Regions, in terms of allocating funds, monitoring and quality review of the analytical work funded by the Trust Fund. The MDTF requires that the Regions detail in their concept notes how they will ensure that poverty, social and distributional impact analysis (supported by the TF) is treated as ESW, TA, or a FTD,‖ but finds that this does not adequately address IEG‘s Recommendation. A higher rating would call for data on the extent to which PSIAs are clear/explicit about their intended operational effect and the PSIA approach is tailored to achieve that effect. IEG rates the level of adoption as Medium. IEG recognizes that making the Regions responsible for managing the MDTF and requiring the Regions to provide matching resources from the country unit budget is an important step in the right direction. It also notes that The Good Practice Note encourages poverty, social and distributional impact analysis to be anchored in the CAS, that the FY10 CAS Retrospective (Country Assistance Strategy: Retrospective and Future Directions) confirmed that CASs, particularly in IDA countries, have included a comprehensive discussion and diagnosis of poverty, covering poverty incidence, trends, causes and correlates with links to the MDGs. As indicated in the previous MAR, a higher rating would call for data on the proportion of matching contributions from country unit budgets where earmarked PSIA funds are used as well as on the extent to which PSIA topics, scope, and approach are consistent with the country assistance program, and PSIAs ask policy-relevant questions. IEG rates the level of 47 adoption as Substantial. As Management has not provided any 2012 update of actions taken, IEG rates adoption of this Recommendation the same as 2011. The actions taken by Management so far (i.e., issuance of the DPL Retrospective and implementation of the PSIA MDTF reporting system) do not adequately address IEG‘s Recommendation. It is not clear if the AAA Review was undertaken and, if so, to what extent its findings were incorporated in the Bank‘s Knowledge Strategy, or to what extent that resulted in enhanced PSIA quality assurance. As indicated in the previous MAR, a higher rating would call for data on the extent to which the Regional management‘s review of PSIAs is focusing on the intended effects of PSIAs in addition to their technical quality, and there is a monitoring and self-evaluation system to assess if PSIAs are being undertaken where appropriate and are achieving their intended effects. IEG rates the level of adoption as Medium. 48 3. The World Bank's Country Policy and Institutional Assessment (CPIA) LIST OF RECOMMENDATIONS 3.1. Disclose ratings for IBRD countries. 3.2. Remove accounting for the stage of development in the CPIA rating exercise. 3.3. Undertake in-depth review of each CPIA criterion and revise as necessary. This should entail a detailed review of the literature for each criterion and should reflect the latest thinking on development and lessons learned. It should also take into account the recommendations of IEG on specific changes to the criteria that were derived from the evaluation. 3.4. Consider not producing an overall CPIA index while continuing to produce and publish the separate CPIA components. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG Difference of Difference of Disclose ratings for IBRD countries. Opinion/ Not Opinion/ Not tracked tracked Difference of Difference of Remove accounting for the stage of development in the Opinion/ Not Opinion/ Not CPIA rating exercise. tracked tracked Undertake in-depth review of each CPIA criterion and High Substantial* revise as necessary. Consider not producing an overall CPIA index while continuing to produce and publish the separate CPIA High Substantial* components. *: Will be retired MANAGEMENT RESPONSE 2012 Original Response: 3.1. Disclose ratings for IBRD countries. Difference of Opinion 3.2. Remove accounting for the stage of development in the CPIA rating exercise. Difference of Opinion 3.3. Undertake in-depth review of each CPIA criterion and revise as necessary. Agreed. 49 Periodic reviews of the content and methodology have been a fixture of the evolution of the CPIA, and going forward they should continue to be. As the IEG evaluation recognizes, these reviews create discontinuities, as some criteria are added, dropped, or revised. The last major revision took place in 2004, informed by the recommendations of an external panel that undertook an in-depth review of the CPIA. Consensus on development thinking moves slowly, and revisions should be undertaken with sufficient intervals so that the CPIA scores have some consistency over time. From the standpoint of country relations and aid volatility, it is also important to avoid situations where changes in scores result from modifications in the criteria rather than from a deterioration or improvement in country performance. The CPIA is used by other multilateral development banks and an extensive consultation process would be necessary. The IEG evaluation found that ―perhaps the time has come… for a thorough review of the CPIA‖ (chapter 2). Management broadly agrees but underscores that such a review needs to be carefully planned and done in the context of IDA 16. The specific suggestions provided in the IEG evaluation will inform this review, to be completed by the time IDA 16 is launched. 3.4. Consider not producing an overall CPIA index while continuing to produce and publish the separate CPIA components. Agreed. Management will take this IEG recommendation into consideration in the context of the review of the CPIA mentioned above. IEG‘s rationale for this recommendation is that producing the different components of the CPIA without assigning weights to them in order to arrive at an aggregate index would allow different weights to be applied according to country context and uses. In management‘s view, in the absence of robust evidence as to what these weights should be, there is value in applying a uniform weighting scheme across all countries and producing an overall index that summarizes the information contained in the different criteria and provides a clear reference point. Moreover, because the scores for all the criteria are disclosed, nothing prevents the users from creating an alternative index based on their preferred set of weights. As part of the review of the CPIA, management will consider whether or not to produce an overall index. MAR 2012. During 2011 the review of the CPIA criteria was completed. The review, which was informed by the conclusions of the IEG evaluation, also provided an opportunity to introduce some changes in the criteria based on the experience accumulated in past exercises. Throughout, special attention was given to ensure that the content of the revisions was commensurate with the availability of information and the Bank‘s ability to assess country performance, particularly for IDA countries; and that some degree of continuity was preserved in the criteria. Changes were introduced to address the IEG recommendations, to reduce the overlap among criteria, and to ensure the consistent treatment of topics across the different rating levels. Going forward, the process used to carry out the exercise will remain unchanged. A draft of the criteria was prepared by the different network anchors, discussed by the respective sector boards and reviewed by the Regions. The revised criteria were then submitted for review by the Council of Chief Economists and their comments incorporated. As agreed during the CODE meeting of September 2009, the new criteria are already being used in the ongoing CPIA exercise that was launched in September 2011. The CPIA review was guided by the IEG recommendations, relevant findings of the 50 development literature and the lessons gathered from the experience of carrying out previous exercises. A major objective throughout the review was to reduce the overlap among criteria, and ensure the consistent treatment of topics across the different rating levels. In addition the review created an opportunity to benefit from the rapid growth of new data, as new indicators have become available and for others the country coverage has increased. Changes in the criteria covering trade, financial management, and debt, for example, take into account the availability of new information for a larger number of countries. The broader availability of qualitative and quantitative information contributes to increase the robustness of the scores. The revision of the criteria also involved testing whether the new criteria would entail significant changes in country scores. This helped to ascertain whether the revised thresholds were set too high/low and, if needed, then make any necessary fine tuning. The new criteria was tested by using it to score a few 2010 benchmark countries on the basis of the write-ups and the information available at that time, and then comparing the distribution of the actual 2010 scores with those obtained under the new criteria to determine whether some marginal adjustment to the contents were warranted. Some changes in scores are unavoidable, but the testing was important to ensure that such changes are well grounded and could be justified. Throughout the review special attention was given to ensure that the content of the revisions was commensurate with the availability of information and our ability to assess country performance, particularly for IDA countries; and that an appropriate degree of continuity was preserved in the criteria. The review maintained the fundamental CPIA framework and process used to determine the country scores remained essentially unchanged. Changes were introduced in all the CPIA criteria, with the exception of Q6 (Business Regulatory Environment). The major changes are summarized below. Cluster A. In light of the links among the criteria in this cluster the importance of assessing them simultaneously in consistent manner to prevent double counting was emphasized in the staff guidance; the title of Q1 was changed from macroeconomic management to monetary and exchange rate policies to explicitly indicate its coverage and to clarify the boundaries with the other criteria in this cluster. The content of Q3 (Debt Policy and Management) was unbundled into two sub-components, debt policy, covering debt sustainability issues, and debt management; the role of DSAs in the assessment was underscored. Cluster B. In Q4 (Trade) trade policy and trade facilitation are now equally weighted; more emphasis is placed on the trade regime, not just imports; services are explicitly introduced; and the trade facilitation sub-component elaborated. The focus of Q5 (Financial Sector) was broadened beyond banking issues; the guideposts were revised to include a significantly simplified version of CPIA Stats; staff guidance clarified that the assessments will focus on a small set of core indicators for each of the sub-dimensions, but that this set can be complemented as appropriate by other indicators that may not be available for all countries. Cluster C. The content of Q7 (Gender) remains essentially unchanged but the spreadsheet used to guide the rating process, including the indicators, was revised. The expenditure component in Q8 (Equity of Public Resource Use) was revised and divided into two parts: measurement issues, and strategies and policies targeting poor and vulnerable groups. The 51 weights were adjusted accordingly. In Q9 (Building Human Resources), the sub-component covering HIV/AIDS, TB and malaria was merged with the health dimension. In Q10 (Social Protection and Labor) the coverage of social assistance programs including coordination, reach and targeting issues was strengthened. The spreadsheet used in Q11 (Policies and Institutions for Environment Sustainability) to guide the determination of the scores was drastically simplified. Cluster D. The overlap between Q12 (Property Rights and Rule–based Governance) and Q6 (Business Regulatory Environment) regarding the burden of regulation was substantially reduced; the sub component on crime and violence now explicitly covers organized crime. The mapping of the content of Q13 (Quality of Budgetary and Financial Management) with PEFA was strengthened, thus increasing the value of this indicator as a source of information. Q15 (Quality of Public Administration) was revised to include a stronger focus on the core public administration and, where relevant, a more explicit treatment of sub-national governments. The changes in Q16 (Transparency, Accountability and Corruption in the Public Sector) include adding a new dimension to cover aspects of financial corruption such as in public contracting that previously were not consistently assigned; a more explicit coverage of transparency of fiscal information; and a more consistent treatment of state capture and conflicts of interest as distinct forms of corruption. The overall CPIA is used in several activities in the Bank. In the Debt Sustainability Framework, for example, the indicative threshold for each debt burden indicator depends on each country‘s policy and institutional capacity as measured by the overall CPIA index. The conclusion of discussions with the regions and networks was that it was important to have an overall score that would be used as a reference point for internal and external purposes, such as discussions with clients, and that would ensure that country scores were used in a consistent manner in Bank documents and in a comparable form across countries. As some speakers noted at the September 2009 CODE meeting that discussed the IEG evaluation, it was ―inevitable to have one overall score‖. In the absence of conclusive empirical evidence about the weights that should be attached to each of its clusters, the CPIA follows the common practice of attaching equal weights to its components. This is an explicit and transparent approach that should not be interpreted as implying that each dimension of the policy and institutional framework is expected to equally contribute to growth and poverty reduction. Clearly, the contribution of specific policy and institutional changes will depend on the different country contexts and will vary across countries. The CPIA scores show a high correlation among its different components, and therefore the index is not significantly affected by moving from the equal weights approach to a set of weights derived from standard statistical methods such as principal components. Equal weights, however, has the added advantage of transparency. Areas where country performance is assessed as relatively strong (weak) can be highlighted by comparing the score of a given country in a specific criterion (cluster) with the equivalent average score for a representative group e.g., IDA countries or a regional average. The disclosure of all 16 criteria scores allows users to construct a new index to analyze a specific area of policy and institutional performance by selecting a sub-set of the criteria and employing their preferred set of weights. 52 IEG RESPONSE 2012 Recommendations 1.1 and 1.2 have not been tracked in the past due to differences of opinion between IEG and Management. With regards Recommendation 1.3, IEG confirms that Management has reviewed and revised CPIA criteria with a view to maintaining some continuity but reducing overlaps between components. IEG confirmed that this process has taken place through a bottom-up process across Networks. As a result, changes have been made, including for instance, an attempt to move away from accounting for stages of development and an attempt to focus on the ―function‖ of institutions rather than their particular organizational ―form.‖ It is envisaged that the updated criteria will be applied during the 2011 CPIA exercise include the revisions. IEG was unable to get a complete list of revisions made: these will need to be confirmed and reviewed again in light of the evolving development literature. Overall, IEG concurs with Management that the level of adoption is substantial and this recommendation will be archived. On Recommendation 1.4, Management is publishing separate CPIA components, but has decided to continue publishing the overall index (based on a non-weighted average of CPIA components). IEG agrees that there is no hard and fast evidence about how different clusters should be weighted. Yet, the publication of the non-weighted index de facto implies that all components are equally important in shaping a country‘s overall institutional endowment and therefore, its poverty reduction prospects. Qualitative, historical evidence suggests that some institutions do matter more than others at particular points in time. If the non-weighted index continues to be published, Management should provide a clearer rationale. Level of adoption is substantial and this recommendation will be archived. 53 4. Gender and Development: An Evaluation of World Bank Support, 2002–08 LIST OF RECOMMENDATIONS 4.1 Foster greater clarity and better implementation of the Bank's gender policy, notably by— t. Establishing a results framework to facilitate consistent adoption of an outcome approach to gender integration in the Bank's work. u. Establishing and implementing a realistic action plan for completing or updating country-level diagnostics, giving primacy to countries with higher levels of gender inequality. v. Extending implementation of the 2007 GAP while formalizing and strengthening its policy basis. An alternative would be to reinstate and strengthen provisions along the lines of OMS 2.20 to restore a sector- and/or project-level entry point for gender. 4.2 Establish clear management accountability for the development and implementation of a system to monitor the extent to which Bank work adequately addresses gender- related concerns, including effective reporting mechanisms. The pivotal role of country directors needs to feature centrally in the accountability framework. 4.3 Strengthen the incentives for effective gender-related actions in client countries by continuing to provide incentive funding through the GAP to strengthen the collection, analysis, and dissemination of gender-disaggregated, gender-relevant data and statistics. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG Foster greater clarity and better implementation of the Bank's High Substantial gender policy. Establish clear management accountability for the development and implementation of a system to monitor the extent to which Bank work adequately addresses gender-related concerns, Substantial Substantial including effective reporting mechanisms. The pivotal role of country directors needs to feature centrally in the accountability framework. Strengthen the incentives for effective gender-related actions in client countries by continuing to provide incentive funding through the GAP to strengthen the collection, analysis, and Substantial Substantial dissemination of gender-disaggregated, gender-relevant data and statistics. 54 MANAGEMENT RESPONSE 2012 4.1 Fostering greater clarity and better implementation of the Bank’s gender policy Original Response: Management agrees that the implementation of the Bank‘s gender policy needs improvement and will detail steps to be taken in the GAP transition plan to be presented to the Board in the fourth quarter of fiscal 2010. Management will also prepare and issue a guidance note to staff on the Bank‘s gender policy framework. Management notes that the Bank has a gender monitoring framework and reports regularly to senior management. Of course, it can be improved and made more results focused. The GAP transition plan will set out how the framework will be strengthened. Management sees the CAS as the link between diagnostics and implementation and as the right place to determine gender priorities in Bank support to all countries in which the Bank has an active program. As noted in the CAS Retrospective (World Bank 2009c), management will work to improve the treatment of gender in CASs and will further monitor that the gender assessment adequately informs the CAS, as required in the policy. Management will report on results in regular gender monitoring reports. Management and IEG agree that the GAP is filling an implementation gap in the Bank‘s gender policy framework. At the Board‘s request, management is preparing a transition plan that will extend the gains of the GAP once it ends. The policy basis for the GAP and future action plans is fully adequate. The Bank‘s relevant policies already determine project entry points for gender. Mar 2012: Management welcomed IEG‘s recommendation that the implementation of the Bank‘s gender policy needs improvement. In addition to the steps outlined in the 2011 MAR, and in response to IEG‘s comments, Management identified a series of concrete steps in the GAP transition plan, ―Applying Gender Action Plan Lessons: A Three-year Roadmap to Gender Mainstreaming (2011 – 2013)‖, presented to the Board in the fourth quarter of fiscal 10, to this end. Further, as part of the IDA16 replenishment, the Bank committed to new gender-related targets, indicators to monitor and processes. Subsequent to that, corollary to the launch of the 2012 World Development Report, management sought and received the Development Committee‘s endorsement for a series of new steps laid out in a paper presented to the DC at the 2011 Annual Meetings: ―Implications of World Development Report 2012: Gender Equality and Development for the World Bank Group‖ Finally, a Corporate Scorecard was established, with additional targets and indicators to track. Through these mechanisms, which all seek to support OP4.20 and other gender-relevant OPs, a clear results framework for Bank support for gender and development has been established and presented in a manner that facilitates consistent adoption of an outcome approach to gender integration in the Bank's work—as called for in IEG‘s evaluation. New and comprehensive commitments, indicators and monitorable actions are time-bound, measurable and all are attributed to specific monitoring units. They include the following: 1. Monitorable Actions for IDA16 (Report to be presented at IDA16 MTR, November 55 2012) , including eight actions and indicators in seven priority areas 2. Tier 1: Country Progress—Indicators, Targets and Monitoring Unit (pre-IDA16, IDA16, Transition Plan and Corporate Scorecard); six indicators, of which two have specific targets 3. Tier 2: Development Results (IDA16 and Transition Plan); three priority areas and targets 4. Tier 3: Operational Effectiveness--Indicators, Targets, Actions and Triggers (IDA16, Transition Plan and Corporate Scorecard); five priority areas, six indicators and four targets. CASs and Operations. An important part of the commitments relate to the Country Assistance Strategies. A set of expanded and operationally focused CAS guidelines are being developed to support compliance with OP4.20, and have been agreed upon by PREM and OPCS. Management now peer reviews all CASs at concept note-stage. In addition, we continue to monitor all CASs to see the extent to which these are adequately gender-informed. We are now reporting progress internally on a quarterly basis. The latest monitoring figures show that all FY11 CASs were in full compliance with the OP requirements, and the same is true for CAS presented during the first quarter of FY12. We continue to monitor the lending product portfolio, rating lending products gender- informed or not and presenting the results in an annual monitoring report. To flag potential issues in the portfolio as early as possible, to provide a chance to redress problems in the portfolio, we now implement a system of quarterly internal reporting. Implications of World Development Report 2012 The launch of the WDR 2012 offers a significant opportunity to increase awareness especially among economic policymakers in client countries. Responding to the priorities for domestic and international action, five major strategic implications follow for the WBG from the WDR findings: - Using the WDR to inform country policy dialogue on gender equality; - Enhancing country-level gender diagnostics; - Scaling up lending for domestic priorities identified by WDR 2012; - Increasing the availability of gender-relevant data and evidence, and; - Leveraging partnerships. 4.2 Monitoring and Mgmt Accountability Original Response: Management agrees with the recommendation to strengthen accountability for implementation of the Bank‘s gender policy, including country directors and operational vice presidents. Management notes that it has monitoring systems in place, but agrees that further work is needed to improve their impact. Starting with the current fiscal year, management commits to an annual Managing Director–level discussion of the comprehensive annual progress report, drawing on inputs from operational vice presidencies. MAR2012: To monitor implementation against the new results framework, and to strengthen 56 accountability mechanisms, and in addition to the steps identified in the 2011 MAR, Managing Directors are now tracking progress on a monthly basis. By January 25, 2012, four MD-level reporting meetings have been held, monitoring trends and issues with comprehensive and senior and Bank representation. Gender is a key topic at the first Corporate Scorecard in late February 2012 4.3 Incentive-funding for collection, analysis, and dissemination of gender-disaggregated data Original Response: Management agrees that incentive funding continues to be needed for gender disaggregated data and statistics, but adds that transitional incentive funding for analytical and operational work has proven to be effective, as demonstrated by GAP results. MAR2012: Since the 2011 MAR, much international support has been galvanized for the improvement, collection and availability of sex-disaggregated data as a tool to support policy making (for instance both in the APEC Women and the Economy declaration, Sep 2011, and in the outcomes of the 4th High-Level Forum in Busan, South Korea). The World Bank has been a party to this effort. The global initiative on gender statistics under the Busan Action Plan for Statistics promises to bring much-needed attention to lagging areas. Through DEC, the World Bank is a part of the Interagency and Expert Group on Gender Statistics (IAEG-GS) that has put together a minimum list of gender indicators covering 5 domains: economic structures and access to resources; education; health and related services; public life and decision-making; and, human rights of women and girl children. This will raise the focus on areas where there has been less progress in the availability of gender-relevant data and statistics. But national statistical agencies will need greater support. The IAEG-GS has identified indicators where methodological investments are needed (eg access to credit, firms and land ownership). IEG RESPONSE 2012 4.1 Fostering greater clarity and better implementation of the Bank’s gender policy: IEG notes that WDR 2012 provides a clear results framework for gender equality. It will now be important that Management provide guidance to staff on how this framework can be adapted at the operational level. IEG commends management for strengthened monitoring of gender integration both into CASs and into the Bank‘s portfolio. IEG appreciates that some policies ((OP 4.10 on Involuntary Resettlement, OP 4.12 on Indigenous Peoples, BP 4.30 on Forests, and OP 2.30 on Development Cooperation and Conflict) require addressing gender issues at the project level. However, the thrust of the evaluation‘s recommendation (4c: to restore a sector- and/or project level entry point for gender integration) continues to remain unaddressed. 4.2 Agreed. 4.3 Agreed. 57 58 5. Water and Development: World Bank Support, 1997–2007 LIST OF RECOMMENDATIONS 5.1 Work with clients and partners to ensure that critical water issues are adequately addressed.  Seek ways to support those countries that face the greatest water stress. The mid-term strategy implementation review should suggest a way to package tailored measures to help the Bank and other donors work with these clients to address the most urgent needs, which will be far more challenging as water supply becomes increasingly constrained in arid areas.  Ensure that projects pay adequate attention to conserving groundwater and ensuring that the quantity extracted is sustainable.  Find effective ways to help countries address coastal management issues.  Help countries strengthen attention to sanitation. 5.2 Strengthen the supply and use of data on water to better understand the linkages between water, economic development, and project achievement. • In project appraisal documents, routinely quantify the benefits of wastewater treatment, health improvements, and environmental restoration. • Support more frequent and more thorough water monitoring of all sorts in client countries, particularly the most vulnerable ones, and help ensure that countries treat monitoring data as a public good and make them broadly available. • In the design of WRM projects that support hydrological and meteorological monitoring systems, pay close attention to stakeholder participation, maintenance, and the appropriate choice of monitoring equipment and facilities. • Systematically analyze if environmental restoration will be essential for water-related objectives to be met in a particular setting. 5.3 Monitor demand-management approaches to identify the aspects that are working or not working and to build on these lessons of experience going forward. • Clarify how to cover the cost of water service delivery in the absence of full cost recovery. To the extent that borrowers must cover the cost of water services out of general revenues, share the lessons of international experience with them so they can allocate partial costs most effectively. • Identify ways to more effectively use fees and tariffs to reduce water consumption. • Carefully monitor and evaluate the experience with quotas as a means to modulate agricultural water use. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG Work with clients and partners to ensure that critical High Medium water issues are adequately addressed. 59 Level of Adoption Recommendations Mgmt IEG Strengthen the supply and use of data on water to better understand the linkages between water, High Medium economic development, and project achievement. Monitor demand-management approaches to identify the aspects that are working or not working and to Substantial Medium build on these lessons of experience going forward. MANAGEMENT RESPONSE 2012 5.1 Work with clients and partners to ensure that critical water issues are adequately addressed: Original Response: Ongoing/Agreed. Management agrees with the recommendation, which is at the core of the 2003 Water Resources Sector Strategy. The Bank has been responsive to government priorities on water in the most waterstressed countries and in those that will face problems in the future. Using a range of instruments (finance, knowledge, and reputation), the Bank has worked toward ensuring that its assistance adds value, especially vis-à-vis other development banks and donors. The Water Resources Sector Strategy Mid-Cycle Implementation Progress Report will highlight (i) how the World Bank has addressed client needs, differentiating by income group; (ii) the growing importance of addressing water issues at the river-basin level; (iii) areas of the 2003 strategy where the Bank has sequenced its approach, starting with studies, technical assistance, capacity building, and pilot projects to address complex issues, such as sustainable groundwater management and coastal management; and (iv) how the development community has been actively working toward meeting the sanitation MDG targets. MAR 2012 The Bank has been working with the SDN Chief Economist in ensuring that the green growth work clearly incorporates the contribution of water aspects in growth. It has also been working with the Special Envoy for Climate Change on the role of water in the mitigation and adaptation agendas. A database with more than 8,000 basins has been incorporated in the Bank‘s climate change portal with information on potential climate change impacts. The Water Unit is also working with ARD on agricultural water management and with SEG/SDV on scaling up hydropower. A community of practice on Sustainable Hydropower is being re established. The Bank has continued to address water issues at the river-basin level with work starting in the Mekong and continuing in several parts of the world, including the Zambezi, Nile, Sava, Niger, Ganges, Amu Darya, Mekong river basins.. Case studies on groundwater governance have been completed in India, Morocco, Kenya, Tanzania South Africa and Peru and a report highlighting the main lessons for other parts of the world is under preparation. The Water Partnership Program (WPP) has organized capacity building events in addressing complex hydrology in operations and in including climate change in water resources management. The Water Expert Team is providing short term support directly to task teams to help them incorporate sustainable groundwater management, sanitation and hydrological analysis throughout the world. 60 WSP has played a pivotal role by bringing the development community in piloting and testing at scale the most effective approaches to sanitation and supporting national and local governments towards meeting the sanitation MDG targets. During FY11-15 WSP is working with partners and stakeholders to help up to 50 million people to gain access to improved rural sanitation. 5.2 Strengthen the supply and use of data on water to better understand the linkages between water, economic development, and project achievement: Original Response: Ongoing/Agreed. Management agrees with the recommendation, and the principle that more and better data would help to support efforts to improve the performance and accountability of the water sector, the results of Bank-financed water projects, and the impact of alternative water policies. Several global initiatives are under way (for example, IBNET, Hydrological Expert Facility), and efforts are ongoing as part of the standard evaluation analysis of projects to quantify the costs and benefits (and externalities). Better management and use of data will take place when the investment lending reforms are implemented. More specifically: • The Water Anchor will develop further core indicators for water projects (for example, sanitation/sewage, irrigation/drainage). • Regions will pilot new approaches to take advantage of new sources of information (such as remote sensing), tackling these with existing data sources. • Regions will scale up projects, building detailed information systems and benchmarking systems. • The Water Anchor and Water Sanitation Program will conduct an impact evaluation of sanitation and hygiene interventions at scale in achieving health and income outcomes. • As part of the development impact evaluation initiative, in collaboration with the Development Economics Department, the water sector will conduct further impact evaluations on health impacts of water and wastewater interventions. The Water Resources Sector Strategy Mid-Cycle Implementation Progress Report will outline how progress toward strengthening the supply and use of data will be monitored. Update 2012. Water sector monitoring continues to be a core activity of the water anchor and its support in development of the sound water sector monitoring continues to advance. In Vietnam, water sector benchmarking is becoming institutionalized through the World Bank loan using information and early pilots supported by WSP. Similar work is being done in Bangladesh. In India, the WSP sustainable local benchmarking project resulted in a massive water sector information collection covering 1600 cities and towns for the first time in India. Information from these studies is already used for the sector strategic planning and development. In connection with the sector benchmarking, the Country Sector Overviews, CSOs based on objective information are initiated in SAR region and expected to be initiated for EAP. 61 The Water Anchor contributed with WSP, SDN Chief Economist Office and HDN in the formulation of a proposal to set up a water ‗window‘ in an HDN trust fund on impact evaluation. The allocation to water could be approximately USD 10 million over 4 years. Once approved, the water practice will select a series of loans that could be subject to impact evaluation and provide support in implementation. The Water Partnership Program (WPP) in cooperation with SDN Chief Economist is organizing regional workshop on impact evaluation to train TTLs and clients in the tools. A first workshop was organized in Bangladesh covering SAR; the next regional workshop is expected to take place in Moldova and it will cover the entire ECA region. The Water Anchor is starting an initiative on the water and energy nexus to better understand the linkages between water and sector economic development and to quantify the economic and social tradeoffs of different allocation decisions. The Water Partnership Program (WPP) is supporting work on the combination of hard and soft infrastructure and green growth activities in ECA and EAP. Remote sensing is becoming an increasingly prominent aspect of the Banks lending and analytical work related to water. Several learning events have taken place led by the water thematic groups. In addition, under the Memorandum of Understanding with the United States Government on water signed by Secretary of State Clinton and President Zoellick in 2011, teams are working on a series of learning events to showcase how remote sensing can be scaled up and highlighting expertise available to Bank clients from the United States government. In 2011 the Board approved a GEF project for five countries in MENA which will bring NASA models and data to help the MENA countries improve their water resource monitoring. In October 2011 TWI (WSP, water anchor and ICT) sponsored a Water Hackathon to bring non-traditional actors to solve water problems related to software technology. This process led to the development of 50 prototype software solutions, many of which related to water monitoring and open data. For example, one of the winners of the Hackathon in Peru was an app to map water quality in Peru, overlaying water quality data with Google Earth maps. This led to the government making water quality data publicly available for the first time. The practice worked with OPCS in the formulation of core indicators for irrigation/drainage projects and the Water Anchor prepared a study to assess the beneficiaries of Bank lending in water supply and sanitation. The Water Partnership Program established the Water Expert Team (WET) to provide just-in- time technical advise to the regions to improve the design and supervision of Bank lending. WET is providing support in information and data by strengthening technical capacities and incorporating lessons learned in lending in hydrometeorological services including in Mexico, Albania and Morocco. WSP supported the Ministry of Public Works in Liberia by piloting a new, more efficient mapping and surveying tool to create a digital waterpoint map. The survey was completed in less than six months and more than 10,000 water points were mapped. The resulting report provided a useful baseline on water quality testing conducted in FY12 which would be used by the Government of Liberia to develop water security and investment strategies for the 62 sector. The water Hackathon successfully conducted in October 2011 engaged wide groups of water activists and young computer users in stating the problems of the sector and their successful solutions. Nearly 1000 software developers worked to address some of the 1000 submitted questions, and more than 50 prototype solutions were developed. 5.3 Monitor demand –management approaches to identify the aspects that are working or not working and to build on these lessons of experience going forward : Original Response: Ongoing/Agreed. • Regions and the Water Anchor will examine financing of services delivery as part of Public Expenditure Reviews and other country-specific economic and sector work. • The Water Anchor and Regions will conduct a study on lessons learned about government payment for water services. • Regions will continue to explore fees, tariffs, and other options (metering, water rights, and the like) for demand management in Bank projects. • Regions will pilot evapotranspiration (ET)-based rights and community-based approaches to water resource management. A key priority of the Thematic Group on Water Resource Management, with the support of the Water Anchor, will be to document lessons learned on demand-management approaches. 2012 Update. The decision meeting for a draft report Public Expenditure Reviews was conducted in December 2011 and will be finalized shortly. The Water Partnership Program (WPP) supported the formulation of PERs in Sierra Leone, Democratic Republic of Congo (DRC), Central African Republic (CAR), Togo, and the Republic of Congo (RoC). WPP also financed analytical work that allowed evapotranspiration (ET) based rights to be piloted in China The Water Anchor is currently preparing an ESW report template on financial sustainability in the water practice to be completed in 2012. The work will present a systematic approach to assessing and documenting the current situation in the water practice, the minimum requirement for financial sustainability, and the desired future scenario. The Regions continue to focus their efforts in ensuring the financial sustainability of water projects and different approaches are being tested including output based aid and public private partnerships in irrigation. Water has built significant partnerships with non-traditional partners: (1) Largest grantee of the Bill and Melinda Gates Foundation on water and sanitation to improve access to rural sanitation; currently in discussions with Coca Cola Foundation, (2) Water Innovation Task Force with leading global sector entrepreneurs & water utilities, (3) Mainstreaming ICT into water applications (mobile; remote sensing), (4) engaged in global partnerships with the biggest donors and players in water and sanitation: UNICEF, UK, Netherlands, Germany, and US on Sanitation and Water for All (SWA). Bill Gates and Hillary Clinton potential speakers at the upcoming SWA meeting during World Bank Spring Meetings. The MOU on Water with US government is the first of its kind: ―whole of government‖. If the MOU is successful, the US will apply this model to engage with all other sectors and the World Bank. (5) The World Bank 63 and WSP sponsored a water ―hackathon‖ to engage software developers in solving water problems. An extensive tariff database, solution for which was suggested by a Hackaton participant is being tested by WSP team. The solution will allow Internet users directly upload tariff information of the IBNET database. IEG RESPONSE 2012 5.1 Work with clients and partners to ensure that critical water issues are adequately addressed.  IEG notes that the Bank is paying increasing attention to incorporate the contribution of water aspects in the green growth work and the related mitigation and adaptation agendas in climate change. While the knowledge about the potential climate change impacts at the basin level may have been built up, it is not clear how this is translating into the implementation of the recommendations provided in IEG‘s Water Evaluation, which was to work with countries that face the greatest water stress. As commented last year, more evidence needs to be provided on the tailored measures to help the Bank and other donors work with the water stressed clients to address the most urgent needs.  IEG welcomes the completion of the case studies on groundwater governance in a few countries, and the preparation of a report highlighting the main lessons for other parts of the world. IEG also notes that there have been capacity building events in addressing complex hydrology in operations. IEG seeks to understand how these activities have turned into specific actions to ensure that projects pay adequate attention to conserving groundwater and that the quantity extracted is sustainable.  With regards to IEG‘s recommendation for the Bank to help countries address coastal management issues, more evidence is required on the progress to date.  IEG welcomes the increasing focus placed by WSP to bring the development community in piloting and testing at scale the most effective approaches to sanitation and supporting national and local governments towards meeting the sanitation MDG targets. IEG seeks to understand in which countries these activities have taken place, and what progress have been made in achieving the MDG targets. IEG rates the level of adoption as Medium. 5.2 Strengthen the supply and use of data on water to better understand the linkages between water, economic development, and project achievement.  IEG welcomes the proposal to set up a water ―window‖ in an HDN trust fund on impact evaluation, and the initiative on water and energy nexus to better understand and quantify the economic and social tradeoffs of different allocation decisions. While these initiatives may help assess the development impact of water sector projects, IEG seeks to understand what actions have been taken to routinely quantify the benefits of wastewater treatment, health improvements, and environmental restoration in project appraisal documents. 64  IEG welcomes the Water Anchor‘s efforts to mainstream water sector monitoring as its core activity and the institutionalization of water sector benchmarking through the World Bank loan in Vietnam, Bangladesh, and India. Remote sensing, digital water point mapping in Liberia, and the use of software technology have also been presented in MAR 2012 to be relevant and useful initiatives for resolving water related problems. Good evidence has been provided on the results of Liberia and Peru initiatives and how they are adding value to the countries. The collected monitoring data should also be treated as a public good and be made broadly available.  IEG welcomes the establishment of Water Expert Team and its just in time technical advice to the regions that contain strengthening of technical capacities and incorporation of lessons learned in hydrometeorological services in Mexico, Albania, and Morocco. More evidence would be required on how these projects are paying close attention to stakeholder participation, maintenance, and the appropriate choice of monitoring equipment and facilities.  More information from the region would be required to assess whether a systematic analysis or monitoring has been adopted in the client countries of e.g. the need for environmental restoration. IEG rates the level of adoption as Medium. 5.3 Monitor demand –management approaches to identify the aspects that are working or not working and to build on these lessons of experience going forward.  IEG welcomes the completion of public expenditure reviews primarily in the Africa region, and evapotranspiration (ET)-based rights piloted in China. IEG expects to see the results of the ongoing work by the Water Anchor to prepare a template on financial sustainability in the water practice that is expected to help enhance the capacity of the countries to better plan and estimate the funding needs to cover the sector deficit.  IEG also welcomes that new partnerships have been developed in the water sector with non-traditional partners, which will contribute to scaling up and harmonize approaches to sanitation, mainstreaming of ICT, and soliciting and implementing innovative solutions to the water sector issues.  According to the original management response, the regions had agreed to continue to explore fees, tariffs, and other options (metering, water rights, and the like) for demand management in Bank projects, but no update has been provided in the MAR 2012. Similarly, a key priority of the Thematic Group on Water Resource Management, with the support of the Water Anchor, was stated as the documentation of lessons learned on demand-management approaches but this has not yet been reported in the MAR 2012. IEG rates the level of adoption as Medium. 65 6. Poverty Reduction Support Credits: An Evaluation of World Bank Support MAR 2012 LIST OF RECOMMENDATIONS 6.1 Phase out the PRSC as a separate brand name for development policy lending or clarify when it is appropriate to use it. • Convergence in the design and content of PRSCs and other development policy lending suggests that there is no rationale for the separate existence of the PRSC today. However, there are also implicit criteria backing the PRSC brand name. If the PRSC brand name is seen to still be important, clear guidelines (which are currently lacking) and criteria for eligibility should be spelled out and applied. 6.2 Simplify the language of conditionality for PRSCs/DPLs by eliminating the term ―triggers‖ and by transferring program benchmarks to the monitoring framework  In line with its use of the term ―prior actions,‖ the Bank could further simplify its lending framework by dispensing with the term ―triggers‖ and substituting the term ―indicative prior actions for future lending.‖ Lending would then be based simply on prior actions already achieved and indicative prior actions for future lending. This would exhibit greater flexibility and improve understanding. To clearly delineate legally binding conditions from program benchmarks, which are still referred to as binding and nonbinding conditions by clients and others in the aid community, program benchmarks should be removed from the policy matrix/Performance Assessment Framework and, instead, combined with the program monitoring framework. 6.3 Enable more effective participation of the Bank in a multidonor budget support lending framework by better synchronizing Bank internal process with donor processes.  At present, Bank financial commitment to support, in a multidonor framework, must sometimes be made before the Bank‘s internal review of the PRSC. This can limit the Bank‘s substantive contributions and comments on program content. Synchronizing the Bank‘s internal processing schedule with country and donor processes would ensure Bank input in PRSC/DPL formulation. 6.4 Underpin operations with comprehensive diagnostics  PRSCs (and DPLs) should reflect country-specific growth diagnostics, which are undertaken based on analytic underpinnings that identify an overall growth strategy reflecting the linkages among growth, poverty-reduction, and broader social development. 6.5 Strengthen PRSC/ DPL results frameworks and link them with the underlying PRS / national development strategy and increase their poverty focus.  Results frameworks of PRSCs should be consistently linked to those in the PRS or national development strategy and its annual reviews, and simplified to a small set of core outcomes. Adequate baseline and intermediate indicators and pro-poor results indicators should be required and built on country monitoring systems to the extent 66 feasible. 6.6 Focus sector content in policy loans on high level or cross-cutting issues.  PRSC/DPL sector content should focus on areas where it has been consistently effective: cross-sectoral or central ministry issues critical to facilitating key sectoral reforms and strengthening sector budget processes. Complementary parallel sector lending, linked to PRSC/DPL, remains necessary to address detailed technical issues and facilitate program ownership by line ministries. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG Phase out the PRSC as a separate brand name for development policy lending or clarify when it is High Substantial appropriate to use it. Simplify the language of conditionality for PRSCs/DPLs Difference of Difference of by eliminating the term ―triggers‖ and by transferring Opinion/ Not Opinion/ Not program benchmarks to the monitoring framework tracked tracked Enable more effective participation of the Bank in a multidonor budget support lending framework by better Substantial Negligible synchronizing Bank internal process with donor processes. Underpin operations with comprehensive diagnostics High Medium Strengthen PRSC/ DPL results frameworks and link them with the underlying PRS / national development High Medium strategy and increase their poverty focus. Focus sector content in policy loans on high level or High Medium cross-cutting issues. MANAGEMENT RESPONSE 2012 6.1 Original Response: Agreed. All operational policy and guidance on development policy lending were unified with the introduction of OP/BP 8.60 in 2004. OP/BP 8.60 and operational guidance do not list PRSCs as a separate option. Since 2004, therefore, PRSCs have not had a separate existence. Poverty Reduction Support Credit (or Grant) is simply a title given to operations, usually programmatic, to signal their alignment with a PRSP. Management will emphasize this in training activities and guidance to staff to eliminate any remaining misunderstandings on this matter. Management is not prepared to object, however, if a government wants to call the development policy operation that it receives from the World Bank a PRSC. Management considers this action complete, as subsequent to the evaluation we have clarified this in DPO guidance. 67 MAR 2012: As indicated in the Management Response, all operational policy and guidance on development policy lending were unified with the introduction of OP/BP 8.60 and operational guidance do not list PRSCs as a separate option. Since 2004, therefore, PRSCs have not had a separate existence. Management has revised its guidance on DPOs to eliminate any remaining misunderstandings on this matter and clarify that a Poverty Reduction Support Credit (or Grant) is a title given to an operation, usually programmatic, to signal their alignment with a PRSP. 6.2 Difference of Opinion/ Not tracked 6.3 Original Response: Agreed in principle. More effective participation of the Bank in multidonor budget support (MDBS) groups would be valuable in helping countries achieve their development goals. Management agrees that it would be optimal to achieve such synchronization. However, MDBS lending frameworks are prepared and implemented in tandem with other development partners, and there can be tradeoffs and limits to Management‘s ability to commit to a particular operation in the absence of Senior Management and Board approval. As indicated in the 2009 DPL Retrospective, Management agrees to review its experience with MDBS frameworks and derive lessons from these collaborative engagements and to use these lessons to identify ways to do better in the future. This action will be completed when the proposed review of experience with MDBS frameworks agreed to in the DPL Retrospective has been concluded and lessons disseminated. MAR 2012: Management has agreed in principle with this recommendation. Management agrees that it would be optimal to achieve such synchronization and most programmatic DPOs to IDA countries in the Africa Region are embedded in a MDBS lending framework. However, full synchronization is not always possible or feasible as MDBS lending frameworks are prepared and implemented in tandem with other development partners, and there can be tradeoffs and limits to Management‘s ability to commit to a particular operation in the absence of Senior Management and Board approval. As indicated in the 2009 DPL Retrospective, Management is reviewing its experience with MDBS frameworks with the objective of deriving lessons from these collaborative engagements. On the basis of these lessons, a Good Practice Note will be issued in FY 12 to provide guidance on how to engage in effective and productive MDBS partnerships. The 2012 DPL retrospective will also present and distill lessons from the experience with MDBS collaborative arrangements. 6.4 Original Response: Agreed. As discussed in the 2009 DPL Retrospective, Management fully agrees that the policy and institutional actions supported by a DPO should be underpinned by comprehensive analytic work. However, Management would like to note that diagnostic work needs to be related to the content of the operation. For example, a DPO that focuses on the health sector should be underpinned by comprehensive analytic work on health. Management will monitor and report on progress in strengthening the diagnostic underpinning of DPLs in the context of its periodic DPL Retrospectives. MAR 2012: Management agreed with this recommendation and considers that it has been completed. As highlighted in the 2009 DPL Retrospective, the policy and institutional actions supported by a DPO should be underpinned by comprehensive analytic work that needs to be related to the content of the operation. Guidance to staff has been revised to clarify how to explain the extent to which analytic work has informed the design and the content of a DPO. Furthermore, Management has agreed to continue monitoring and reporting on progress in 68 strengthening the diagnostic underpinning of DPOs in the context of its periodic DPL Retrospectives. 6.5 Original Response: Agreed. As discussed in the 2009 DPL Retrospective, DPOs have become increasingly results focused, but there is scope for further improvement in their results frameworks. Management agrees with the recommendation to strengthen DPL results frameworks with the adequate use of baseline and results indicators that are linked to the actions supported by the operation. Management notes, however, that the results frameworks of DPOs need to be more specific than those prepared for a PRSP or a national development strategy. In the context of its 2009 DPL Retrospective, Management has agreed to update guidance to staff on how to design results frameworks. This action will be considered completed when the revised guidance to staff agreed to in the DPL Retrospective has been completed and disseminated. MAR 2012: Management agreed with this recommendation and considers that it has been completed. As indicated in the Management Response, Management agreed to undertake this recommendation by updating its guidance to staff on how to design results frameworks. A revised Good Practice Note on Results in Development Policy Lending reflecting the findings of the 2009 DPL Retrospective has been posted in OPCS‘s website. 6.6 Original Response: Agreed. Management agrees with this recommendation to focus sector content of DPOs on areas that can enhance the effectiveness of the reforms supported by the Bank and with the provision of parallel sector lending necessary to address specific sector issues that cannot be addressed by the DPL instrument in isolation. Management will continue to monitor the content of DPOs in the context of its periodic DPL Retrospectives. MAR 2012: Management agreed with this recommendation and considers that it has been completed. Management has reviewed the experience with sector-specific DPOs in its 2009 DPL Retrospective which concluded that DPOs with sector-specific content support policy and institutional actions considered necessary to address specific sector issues to enhance the effectiveness of the reforms supported by the Bank. Management has agreed to continue to monitor the content of DPOs in the context of its periodic DPL Retrospectives, including in the upcoming 2012 DPL Retrospective. IEG RESPONSE 2012 6.1 Phase out the PRSC as a separate brand name for development policy lending or clarify when it is appropriate to use it.  Convergence in the design and content of PRSCs and other development policy lending suggests that there is no rationale for the separate existence of the PRSC today. However, there are also implicit criteria backing the PRSC brand name. If the PRSC brand name is seen to still be important, clear guidelines (which are currently lacking) and criteria for eligibility should be spelled out and applied. Level of Adoption: Substantial. IEG considers that there has been progress towards the clarification of what PRSCs are, reflecting their programmatic nature and links to a PRSP. Current guidelines (2010-11-18) state that ―Programmatic development policy operations in IDA-eligible countries that support the implementation of a country's Poverty Reduction Strategy (PRS) are called poverty reduction support operations (credits or 69 grants)‖. Management has stated that in two recent training activities: Nuts and Bolts, and Customization of DPOs, guidance has been given to staff to emphasize this. The OPCS website points out that the PRSC is a title given to programmatic development policy operations in IDA-eligible countries that support the implementation of a Poverty Reduction Strategy. IEG notes however that there is no change in the Management Response as of last year. It would therefore like to repeat that the revised wording does not comment on other implicit attributes of PRSCs that have been associated with the ‗brand name‘: their orientation towards better performing countries, their broad scope, the focus on social sectors and poverty related issues. It is to be noted that the notion of a PRSC as being a part of a programmatic series reserved for better performing countries and signaling long term Bank support and engagement still remains. It has a signaling content not only within the Bank but also to external audiences. Changing the practice within the institutions as well as changing external perceptions is a difficult process though the need for such a more broad based concept of response to the recommendation should be kept in mind. If PRSCs are now indistinguishable from other DPLs, evidence of this, comparing recent PRSCs with other DPLs from management would be welcome. A look at recent PRSC operations over the past two years, underscores this. While Bank guidance states that the PRSC is a title given to Bank programmatic DPL operations in IDA eligible countries that support the implementation of a PRSP, the implicit characteristics of a PRSC as well as its associations with long term support to better performers still remains. Several examples are provided below. IEG would like again to invite management to comment on how these are indistinguishable from other DPLs. Appro IDA val FY Proj ID Project name Date, Approval Country Commit Amt ($ million) 1 2010 P117345 TZ PRSC 7 SUPPLEMENTAL 12/22/2009 Tanzania 170 2 2010 P117287 BJ-PRSC 6-Sixth Poverty Reduction Suppor 4/29/2010 Benin 30 3 2010 P117278 BF-PRSC 10 - DPL 6/29/2010 Burkina Faso 90 4 2010 P117273 SN-PRSC 4-DPL 6/1/2010 Senegal 43 5 2010 P117270 ML-PRSC 4 - DPL 6/3/2010 Mali 70.5 6 2010 P117238 MW-PRSC 3 6/8/2010 Malawi 54 7 2010 P117234 MZ-PRSC 6 (first in new series) 11/12/2009 Mozambique 110 8 2010 P113306 DPL 1/PRSC V 12/17/2009 Cape Verde 15 9 2010 P112817 LS - PRSC II 3/30/2010 Lesotho 25 10 2010 P111182 VN - PRSC 9 6/24/2010 Vietnam 150 11 2010 P107218 ZM-First Poverty Reduction Support Cr. 3/30/2010 Zambia 20 12 2011 P125114 BJ-PRSC 6 Supplemental Credit 4/21/2011 Benin 22 13 2011 P122783 LS-PRSC III 6/28/2011 Lesotho 18 14 2011 P122483 ML-PRSC 5 - DPL 4/28/2011 Mali 70 15 2011 P122470 MZ-PRSC 7 (second in series) 12/16/2010 Mozambique 85 16 2011 P121812 CV-DPL-PRSC VI 12/16/2010 Cape Verde 10 17 2011 P121178 SN-PRSC 5-DPL 5/26/2011 Senegal 42 18 2011 P117924 Poverty Reduction Support Credit (PRSC-7 1/20/2011 Ghana 215 19 2011 P117370 ZM-PRSC 2 3/31/2011 Zambia 30 20 2011 P116666 TZ-PRSC 8 (5th and last in 2nd series) 9/28/2010 Tanzania 115 21 2011 P101232 UG-PRSC 8 9/30/2010 Uganda 100 22 2012 P122805 BF-PRSC 11 - DPL (last of series) 7/26/2011 Burkina Faso 125 23 2012 P122803 BJ-PRSC 7-Seventh Poverty Reduction Supp 12/19/2011 Benin 30 24 2012 P111183 VN - PRSC 10 12/15/2011 Vietnam 150 1789.5 6.2 Difference of Opinion/ Not tracked 6.3 Enable more effective participation of the Bank in a multi-donor budget support 70 (MDBS) lending framework by better synchronizing Bank internal process with donor processes.  At present, Bank financial commitment to support, in a multi-donor framework, must sometimes be made before the Bank‘s internal review of the PRSC. This can limit the Bank‘s substantive contributions and comments on program content. Synchronizing the Bank‘s internal processing schedule with country and donor processes would ensure Bank input in PRSC/DPL formulation.  Level of Adoption: Negligible. IEG notes that in their response for the MAR 2012, Management response has again alluded to its review of Bank experience with MDBS frameworks, which Management has flagged is a prerequisite to the adoption of specific steps in this direction. IEG points out that there is no indication of such a report and requested an update on progress. IEG notes that Management has now committed itself to issuing a Good Practice Note in FY12 to provide guidance on how to engage in effective and productive MDBS partnerships. It has also committed to a presentation of distilled lessons from the experience of MDBS collaborative arrangements in the 2012 DPL Retrospective. IEG looks forward to the Good Practice Note and to the write up in the DPL retrospective. 6.4 Underpin operations with comprehensive diagnostics  PRSCs (and DPLs) should reflect country-specific growth diagnostics, which are undertaken based on analytic underpinnings that identify an overall growth strategy reflecting the linkages among growth, poverty-reduction, and broader social development. Level of Adoption: Medium. IEG notes that there is no change in the MAR 2010 relative to the MAR 2011 with regard to this recommendation, as Management considers it to be completed. IEG therefore points out again that while the 2009 DPL retrospective indeed highlights the need for a comprehensive analytical underpinning of a DPL, IEG notes that the 2009 DPL Retrospective describes the need for an appropriate macroeconomic framework and spells out its core elements. Management notes also that guidance to staff has been revised to clarify how to explain the extent to which analytic work has informed the design and the content of a DPO. If there are additional Guidance documents to staff on the analytical framework for a DPL they are requested. There is also a need to demonstrate the results; i.e. the extent to which there is a real improvement in the macroeconomic framework or analytical underpinning since the revision of guidance. In the absence of such evidence it is difficult to judge the outcome. 6.5 Strengthen PRSC/ DPL results frameworks and link them with the underlying PRS / national development strategy and increase their poverty focus.  Results frameworks of PRSCs should be consistently linked to those in the PRS or national development strategy and its annual reviews, and simplified to a small set of core outcomes. Adequate baseline and intermediate indicators and pro-poor results indicators should be required and built on country monitoring systems to the extent feasible. Level of Adoption: Medium. IEG notes that there is no change in the MAR 2010 relative to the MAR 2011 with regard to this recommendation, as Management considers it to be completed. IEG therefore points out again that while Management has revised its guidance 71 to staff, what is needed is a demonstration of achievement of better results frameworks, and that revised guidance is just a first step towards this. IEG had suggested that an assessment of achievement could be incorporated in the upcoming DPL Retrospective, and would now hope to see this in the 2012 DPL Retrospective. 6.6 Focus sector content in policy loans on high level or cross-cutting issues.  PRSC/DPL sector content should focus on areas where it has been consistently effective: cross-sectoral or central ministry issues critical to facilitating key sectoral reforms and strengthening sector budget processes. Complementary parallel sector lending, linked to PRSC/DPL, remains necessary to address detailed technical issues and facilitate program ownership by line ministries. Level of Adoption: Medium. While Management has indeed included a review of sector specific content of DPLs in its 2009 DPL Retrospective, IEG does not perceive a discussion of the need for, or consideration of, a focus on high level or cross-cutting issues with regard to sectors, in broad based DPLs. IEG notes that Management has flagged explicitly, in the MAR 2012, that such a review will be included in the 2012 Retrospective/ 72 7. Safeguards and Sustainability Policies in a Changing World – An Independent Evaluation of World Bank Group Experience LIST OF RECOMMENDATIONS 7.1 Revise the policy frameworks to harmonize thematic coverage and guidance across the WBG and enhance the relevance of those frameworks to client needs. IFC, MIGA, and the World Bank should jointly adopt and use a shared set of objective criteria to assess social and environmental risks to ensure adequacy and consistency in project categorization across the WBG, using the more inclusive criteria for category A, and refining the categorization system to address the bunching of higher- and lower-risk projects within the current category B. The World Bank should:  Ensure adequate coverage of social effects—integrating community and gender impacts, labor and working conditions, and health, safety, and security issues not currently covered by its safeguard policies—by consolidating existing social safeguards with other WBG policies on social risks as requirements under one umbrella policy on social sustainability.  Consolidate the environmental policies as requirements under one umbrella policy on environmental sustainability.  Revise the current approach to safeguards pilots on use of country systems to focus on strengthening country institutions and systems to manage environmental and social risks. 7.2 Enhance client capacity, responsibility, and ownership. The World Bank should:  Increase the synergies between safeguards work and broader Bank engagement on environmental and social sustainability by investing in upstream analytical work, technical assistance, and lending to strengthen country and sector institutions and capacities in client countries.  Require regular reporting by the borrower on implementation and outcomes of safeguards in Bank-supported projects, and work with clients to develop instruments and indicators to help in such monitoring. 7.3 Revise guidelines, instruments, and incentives to strengthen supervision arrangements. The World Bank should:  Assign responsibility and budget for safeguards oversight and reporting to environmental and social units in each operational region—in line with IFC practice— in place of the delegation of safeguards processing and supervision to sector management units.  Introduce a certification program to expand the pool of staff qualified to undertake social and environmental preparation and supervision while ensuring quality and consistency, and provide orientation training on environmental and social 73 sustainability to all task team leaders.  Develop and implement an action plan to ensure regular supervision of FI projects and investment projects that use social and environmental policy frameworks through third party or community monitoring for higher-risk projects, and disclosure of monitoring and supervision reports. 7.4 Strengthen safeguards monitoring, evaluation, and completion reporting. The World Bank should:  Include performance indicators on environmental and social outcomes in project results frameworks and ensure systematic collection of data to monitor and evaluate safeguards performance.  Ensure that Implementation Completion Reports and IEG reviews of those reports rate and report effectively on the outcomes of safeguards and, for all projects with significant environmental and social effects, ensure the results are incorporated as an essential dimension when assessing achievement of the project‘s development objective, as has already been done for IFC and MIGA. 7.5 Improve systems and instruments for accountability and grievance redress. IFC, MIGA, and the World Bank should:  Seek greater symmetry in the structure of WBG accountability and grievance redress mechanisms. For the World Bank this would entail creation of a grievance redress and conflict resolution mechanism to complement the IPN. For IFC and MIGA this would entail a more independent compliance review process, ensuring that the CAO submits its audits directly to the Board. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG Revise the policy frameworks to harmonize thematic coverage and guidance across the WBG and enhance the Substantial Medium relevance of those frameworks to client needs. Enhance client capacity, responsibility, and ownership. Medium Medium Revise guidelines, instruments, and incentives to strengthen Substantial Medium supervision arrangements. Strengthen safeguards monitoring, evaluation, and Medium Negligible completion reporting. 74 Level of Adoption Recommendations Mgmt IEG Improve systems and instruments for accountability and Medium Medium grievance redress. MANAGEMENT RESPONSE 2012 7.1 Revise the policy frameworks to harmonize thematic coverage and guidance across the Bank Group and enhance the relevance of those frameworks to client needs Original Response: 7.1 a) World Bank, IFC, and MIGA: Agreed. Bank, IFC, and MIGA Management will convene within the first half of fiscal 2011 a small group of senior-level environmental and social specialists to discuss approaches to either a shared set of objective criteria or alternative approaches to categorization that are more refined in scope and clearer to teams. The recommendations of this review will be factored into Bank Management‘s review of global good practice, which will be carried out in preparation of an overall update of Bank policies on project safeguards. IFC will consider these recommendations in its revised Sustainability Framework, which will be presented to the Board. IFC Management will provide internal guidance to staff regarding categorization as part of the update of its Environmental and Social Review Procedures, which will be finalized at the completion of the ongoing process of updating IFC‘s Sustainability Framework. MIGA will review its Policy on Social and Environmental Sustainability to make necessary changes and bring its categorization more in line with IFC and the Bank, after IFC revises its Sustainability Policy and proposed changes are endorsed by the Board. Timeline: In parallel with the update of Bank safeguards (see below) and following Board approval of the updated IFC Sustainability Policy and Performance Standards. MAR 2012 Senior level environmental specialists of the World Bank, IFC and MIGA held meetings to discuss this matter, supported by LEGEN. This has resulted in an alignment between the World Bank and IFC on classification of Category A projects. The Bank also anticipates, as part of the safeguard updating process, adopting the recently introduced (January 1, 2012) subcategorization system for Financial Intermediary operations adopted by IFC. The World Bank has prepared and disseminated Guidelines for Environmental Screening and Classification which summarizes coverage. This note is currently under a revision process being coordinated by QACU and LEGEN, in order to provide improved guidance on classification of projects in Category B and Category C For purposes of internal coordination the Bank will provide the draft revised guidance note to IFC and MIGA for their review and then have discussions as appropriate. The World Bank, through discussions with other multilateral development banks as part of the updating process for the safeguard policies, is also examining the issue of how to improve the use of Category B as an element of our review 75 of global good practice. Original Response 7.1.b) Disagreed. While Bank Management recognizes, as does IEG, the importance of undertaking a comprehensive updating and consolidation of its safeguard policies, it is not yet ready to agree in this detail on the final outcome of that process. Instead, taking into account IEG‘s analysis and consideration of IFC‘s Performance Standards for its private sector support in the context of the Bank‘s public sector support, Bank Management plans to engage in a learning and consultative process with a diversity of shareholders and stakeholders on global good practice (in developing countries as well as industrial countries). Bank Management plans to complete this process in the next 24 months and then report to CODE and the Board on how it intends to further strengthen environmental and social sustainability in its projects, including presentation, for their consideration, of a policy paper setting out its updated and consolidated approach. A period of 24 months has been proposed to provide adequate time to undertake an interactive review process, develop a draft umbrella safeguard policy, prepare translations, and conduct consultations within and outside the WBG. Periodic briefings are planned to be held on a regular basis with representatives of CODE and the Board to keep them apprised of developments and to seek their guidance to the joint team as needed. Timeline: 24 months. During this process, on an interim basis, Bank Management will address concerns related to the balance between environmental and social issues by preparing and issuing guidance on the scope and coverage of social issues in the context of the preparation and implementation of environmental assessments. Timeline: Guidance issued by the end of the third quarter of fiscal 2011. MAR 2012 Although the Bank has disagreed with the IEG recommendation on outcomes, the Bank has initiated a process for a comprehensive update and consolidation of the Bank‘s environmental and social safeguard policies. While initially planned to be undertaken over a 24-month period, following a variety of discussion with a variety of discussions with external and internal parties, Management will present an Approach Paper for discussion by CODE in late September 2012 which will include a proposal for an extending the schedule for completion of this process in 2014. Progress continues to be made on the pilot program for the use of country systems, with additional projects at the national and subnational level adopting this approach. Management does not plan to have a freestanding process for the revision of OP/BP 4.00. Management has included the revision of OP/BP 4.00 as an integral element of the ongoing process for a comprehensive update and consolidation of the Safeguard Policies. Management anticipates mainstreaming the use of country systems in a modified form as an element of this process. 7.2 Enhance client capacity, responsibility, and ownership Original Response 7.2.a) Agreed. Bank Management agrees and will work among SDN, OPCS, LEG, and the Regions to promote this approach. This issue will also be an element of the global good practice review discussed above. For example, as part of the updated Environment Strategy process, SDN is developing guidelines on how to incentivize analytical work, technical 76 assistance, and lending that strengthens environmental governance, institutions, and capacity in client countries. While Bank Management agrees, it suggests that it not be included in future Management Action Records for monitoring because there is no clear way of demonstrating its implementation. MAR 2012 No action as Management considers that there is no clear way of demonstrating implementation as stated in the Original Response. 7.2.b) Not Agreed. Instead, this issue will be included in the process outlined above in response to Recommendation 1. MAR 2012 As noted in the Original Response, as part of the process outlined in Recommendation 1, measures will be proposed to improve monitoring by borrows on implementation and outcomes of safeguards in Bank-supported projects and adoption of standardized indicators. The Bank is reviewing the approach to environmental and social monitoring indicators recently adopted by the IFC to determine if they could be adapted for Bank use, This would allow for a more consistent approach to monitoring and use of comparable indicators. 7.3 Revise guidelines, instruments, and incentives to strengthen supervision arrangements Original Response 7.3.a) Disagreed. Bank Management agrees with IEG that there is a need to strengthen supervision of medium- and low-risk projects. How this will be done may need to differ from region to region, depending on country capacity and project type and mix. Bank Management does not agree with the specific recommendation on giving the responsibility and budget for safeguard oversight and reporting to environmental and social units in each operational Region and this will need to be dropped from further monitoring by IEG. Bank Management plans to undertake a review by the second quarter of fiscal 2011 concerning current practices with respect to responsibility, accountability, incentives, staffing, and budgeting for safeguard processing and supervision. This review will also cover the issue of financial intermediary projects and projects that use environmental and social policy frameworks (see below). Based on this review, practices will be updated with the objective of enhancing effectiveness and efficiency and maximizing the synergies between safeguard work and broader Bank engagement on environmental and social sustainability. Timeline: Bank Management action, based on the review, by the third quarter of fiscal 2011. Bank Management notes that, as part of Investment Lending Reform process, it has actions ongoing to enhance the effectiveness and efficiency of implementation support. These include: (a) the assignment of staff and budget in line with the level of risk associated with an operation, using the new risk assessment and management procedures; and (b) the embedding of grievance redress mechanisms more broadly into projects. Original Response 77 7.3.b) Agreed/Ongoing. OPCS is developing, in coordination with SDN and LEG, a mandatory Operational Core Course for task team leaders which includes modules on safeguard policies and their implementation. Bank Management also has several ongoing and planned initiatives to expand the pool of qualified environmental and social staff that can provide support on safeguards and sustainability issues. Bank Management supports the initiation of a certification/ accreditation program for environmental and social staff working on sustainability and safeguard issues starting in fiscal 2011. SDN is working on the design of a core environmental and social sustainability and safeguards course, which will act as a mentoring and certification/ accreditation program for environmental and social staff, selected staff of other sectors, and safeguard consultants. The certification/ accreditation program will commence by the end of fiscal 2011. SDN also has launched several complementary initiatives to improve the staffing and skills mix for sustainability and safeguards, and to align incentives with the mainstreaming of environmental and social sustainability throughout the portfolio. These include: (a) a Bank-wide analysis of staffing for environmental and social sustainability and safeguards; (b) the development of competencies that emphasize skills in sustainability and safeguards, on both the environment and social issues; (c) consistent management signaling regarding the importance of working on sustainability and safeguards; and (d) the organization of field-based training sessions on sustainability and safeguards. Timeline: Processes in place (subject to cost considerations) by the beginning of fiscal 2012. MAR 2012 Operational Course for Task Team Leaders – Completed. Three E-Learning Modules11 on Safeguards have been prepared cooperatively by OPCS, LEG and SDN, to supplement the face-to-face safeguards training program offered by QACU. They have become part of the Operational Core Curriculum (OCC) developed by OPCS, with Module 3 targeted at TTLs. Accreditation Process / Environment – Ongoing ENV is implementing an Environmental Safeguards Accreditation process for environmental specialists involved in project preparation and supervision. The process includes the designation of suitable staff by their respective managers, a one week compulsory course, face- to-face peer review and operational mentoring in the field, leading to accreditation by the Sector Board. Environmental Accreditation material has been developed. A pilot training course was conducted with a core safeguards team on January 9-13, 2012 and 24 specialists fro, across the Bank were trained on May 14-18, 2012. Training courses and field mentoring will be done in all 6 regions in FY13 for field based staff and local consultants. Accreditation Process / Social – Ongoing A Social Development Training and Accreditation program has been developed and began in June 2011. The program covers for basic competency areas including social safeguards. The 11 The OPCS OCC Webpage http://intranet.worldbank.org/WBSITE/INTRANET/UNITS/INTOPCS/0,,contentMDK:22772354~pagePK:5145532 4~piPK:3763353~theSitePK:380832,00.html The Description of the Safeguards Learning http://intresources.worldbank.org/INTOPCS/Resources/380831-1294415423637/Fullcoursedescriptions_latest.pdf 78 program consists of 4 components: (a) e-learning; (b) face-to-face training; (c) mentoring; and (d) practical field experience. Under this program 56 staff from both headquarters and field has been trained to date. The next session is scheduled for February/March 2013. Community of Practice. Both ENV and SDV have initiated the establishment of communities of practice for environmental and social specialists, respectively. The SDV community of practice has been active since 2009 and the ENV community of practice since 2011. Original Response 7.3.c) Agreed/Ongoing. Bank Management is currently engaged in a Bankwide review of the use of frameworks that will examine these types of projects and identify good practices. The review will include an examination of a variety of means to strengthen monitoring of such projects, including, in appropriate situations, the use of third-party or community monitoring for selected higher-risk projects. The review is expected to be completed by the third quarter of fiscal 2011 and will provide the basis for guidance to be issued for use by Bank staff and borrowers by the fourth quarter of fiscal 2011. Disagreed. See above on supervision. To be clear, Bank Management does not agree and will not be held accountable in future Management Action Records for asking clients to implement third-party or community monitoring. Timeline: Action completed by the end of fiscal 2011. MAR 2012 World Bank Management reviewed the use of frameworks and made a decision to do a detailed examination of the performance of projects using environmental and social management frameworks (ESMFs) in the Africa and South Asia Regions. These reviews covered over 80 active projects across sectors and themes. A standard methodology was applied with extensive documentation that reviewed all aspects of preparation of the frameworks and implementation. Recommendations were provided on actions to improve the practical use of frameworks across the Bank, including the identification of good practice approaches. A guidance note on the use of Environmental and Social Management Frameworks (ESMFs) is being prepared by QACU, LEGEN and AFTSG, QACU sponsored training on the use of safeguard frameworks in January 2012, using resource staff from QACU, LEGEN and ECA. There will be further sessions in February and April 2012. QACU is in the process this fiscal year of creating a new webpage called Frameworks on its anchor website that will contain Bank wide information in one central repository for the community of practitioners. This site will hold examples of good ESMF products, sample of good sections in operational manuals, ToRs for contracting the development of frameworks, samples of checklists, screening forms and other guidance and tools. As stated in the Original Response, Bank Management does not agree and will not be held accountable in future Management Action Records for asking clients to implement third-party or community monitoring. 7.4 Strengthen safeguards monitoring, evaluation, and completion reporting Original Response 79 7.4.a) Partially Agreed/Ongoing. Bank Management agrees with the need to strengthen monitoring and evaluation arrangements. To address this issue, the Bank will collaborate on developing guidelines on monitoring and evaluating safeguard performance by the third quarter of fiscal 2011. These guidelines will focus on more systematically measuring outcomes, including through the use of core environmental and social performance monitoring indicators, and on evaluating impacts. Reporting will be integrated in the ISR, building on the new risk framework under the Investment Lending Reform, which is already enhancing the monitoring of environmental and social risk mitigation measures in ISRs. The proposed guidelines on monitoring and evaluation will further emphasize the need for the ICR to evaluate the achievement of the safeguard-related objectives and identify lessons for future projects. Timeline: Guidelines issued by the end of the third quarter of fiscal 2011. MAR 2012 The World Bank has examined options for improving safeguards monitoring and evaluation arrangements. In light of the recent operationalization of the IFC Sustainability Framework and Performance Standards, the Bank is reviewing a proposed alignment of its approach with that used by the IFC in order to have standardized environmental and social performance monitoring indicators and evaluation of impacts that are consistent between the two institutions. It is anticipated that the adoption of such standardized guidelines for monitoring and evaluation would be linked to the ISRs and ICRs to allow for better reporting and comparative review. Original Response 7.4.b) Not Agreed. Bank Management does not agree and will not be held accountable in future Management Action Records for asking clients to use performance indicators on environmental and social outcomes in all project results frameworks. 7.5 Improve systems and instruments for accountability and grievance redress Original Response 7.5.a) World Bank: Agreed/Ongoing. Bank Management agrees with IEG that there is value in creating a grievance redress mechanism for which Bank Management will take responsibility that is complementary to, but separate from, the Inspection Panel. Bank Management wishes to underscore that establishing this mechanism would not alter the responsibility of borrowers and recipients for implementing projects, and that in many cases, the grievances are not necessarily with the Bank, but between our clients and project-related stakeholders. Nevertheless, these grievances are often brought for resolution to the Bank. Therefore, by the end of the third quarter of fiscal 2011, Bank Management intends to complete a survey and review of a wide range of potentially analogous existing grievance redress mechanisms as a basis for designing one for the Bank. The study will include a review of the cost implications and potential cost savings that could be engendered by using a system similar to the IFC CAO or other multilateral financial institutions. Bank Management will present the results of this study to the Board to ensure that any decisions emerging from the study will be consistent with the Board Resolution and related Board decisions concerning the Inspection Panel, and in a manner which takes fully into account the current requirements and experiences with 80 project-based grievance mechanisms (including as required under OP 4.12, Involuntary Resettlement, and OP 4.10, Indigenous Peoples). This study will be coordinated among Bank units with considerable experience in this field to ensure institutional coherence and efficiency. Timeline: Bank Management will (subject to cost considerations) establish a grievance mechanism by the first quarter of fiscal 2012, and provide to the Board a detailed report on the initial operation of the grievance mechanism by the end of fiscal 2012 MAR 2012 A small facility to lead this work has been established in OPCS, within the Operations Risk Management Department, with expertise in operational dispute resolution. The development of this facility has involved a review of practices used by a variety of both public and private sector parties to address operational disputes. The facility is tasked with supporting Management efforts to address potential and actual operational disputes from a risk based perspective – to focus on prediction and prevention as well as resolution. Underway is work within Bank policies and structures to provide in-house guidance to Bank staff in order to improve outcomes at the early stages of complaints and potential disputes. IEG RESPONSE 2012 IEG welcomes Management‘s first update on the follow up to the Safeguards evaluation. We will give our rationale for each rating below. Our ratings reflect those recommendations on which management committed actions to the Board in its original management response. 7.1 Revise the policy frameworks to harmonize thematic coverage and guidance across the WBG and enhance the relevance of those frameworks to client needs. Management has initiated harmonization of classification for Category A, but action on Category B and C is still pending. The World Bank ―Guidelines Environmental for Screening and Classification‖ (dated February 2007) basically reiterates the categorization definitions offered in OP 4.01 and indeed is in need of revision. 7.1b) The World Bank should: Ensure adequate coverage of social effects—integrating community and gender impacts, labor and working conditions, and health, safety, and security issues not currently covered by its safeguard policies—by consolidating existing social safeguards with other World Bank Group policies on social risks as requirements under one umbrella policy on social sustainability. 7.1c) Consolidate the environmental policies as requirements under one umbrella policy on environmental sustainability. IEG acknowledges that the 24 month period beginning in July 2010 has not yet ended and we look forward to seeing the final proposal from management. The ―Interim Guidance Note: Assessing Social Impacts and Risks Under OP/BP 4.01- Environmental Assessment‖ makes no mention of (a) labor and working conditions; and (b) 81 health and safety and security. QACU indicated that these issues are currently being addressed as follows: (a) Labor and Working Conditions. Having high level discussions within the Bank to determine how and through which instruments Management considers that labor and working conditions would most appropriately be addressed in the context of IBRD/IDA supported operations. This ongoing discussion includes how these issues should be addressed within the context of the Standard Bidding Document used for Procurement by the Bank. However, several regional safeguard units (e.g. EA, ECA) have already started this process by simply requiring the OSH guidance presented in the WBG EHS Guidelines be incorporated into or cross referenced in the EMPs of Category A and B projects. (b) Health, Safety and Security. OPCS has already initiated new training on this matter and are covering this issue in our updated guidance materials such as the one discussed immediately above concerning screening and classification The current EA Policy certainly is an adequate framework to address Health, Safety and Security. However, explicit mention would act to reinforce to both staff and our clients the priority which the Bank places on this issue. 7.1d) Revise the current approach to safeguards pilots on use of country systems to focus on strengthening country institutions and systems to manage environmental and social risks. IEG‘s review of Safeguards had found that the policy OP4.00 in its current form had not been successful and recommended a revision. Management did not comment on this sub- recommendation. The new Bank Policy OP/BP 9.00 (for the Program-for-Results financing Instrument) appears to have more of the elements of a genuine use of country systems. However, it is too early to assess the effectiveness of this policy. (See BP 9.00 paras. 28-31) 7.2 Enhance client capacity, responsibility, and ownership 7.2a) The World Bank should: Increase the synergies between safeguards work and broader Bank engagement on environmental and social sustainability by investing in upstream analytical work, technical assistance, and lending to strengthen country and sector institutions and capacities in client countries. Agreed but no follow up 7.2b) Require regular reporting by the borrower on implementation and outcomes of safeguards in Bank-supported projects, and work with clients to develop instruments and indicators to help in such monitoring. Not agreed, no follow up 7.3 Revise guidelines, instruments, and incentives to strengthen supervision arrangements 7.3a) The World Bank should: Assign responsibility and budget for safeguards oversight and reporting to environmental and social units in each operational Region—in line with IFC practice—in place of the delegation of safeguards processing and supervision to sector management units. Disagreed, no follow up 7.3b) Introduce a certification program to expand the pool of staff qualified to undertake social and environmental preparation and supervision while ensuring quality and 82 consistency, and provide orientation training on environmental and social sustainability to all task team leaders. Agreed/ongoing One of the IEG recommendations that has received a lot of ―traction‖. 7.3c) Develop and implement an action plan to ensure regular supervision of financial intermediary projects and investment projects that use social and environmental policy frameworks through third-party or community monitoring for higher-risk projects, and disclosure of monitoring and supervision reports. The actions taken by the Bank (Africa/South Asia Review, training, webpage etc.) are noted. However, they do not address the IEG recommendation, namely: Develop and implement an action plan to ensure regular supervision of financial intermediary projects and investment projects that use social and environmental policy frameworks. The Bank actions indicated are primarily focused on the preparation of quality Frameworks, which is important. However the recommendation focuses on the supervision of projects using Frameworks, which is not addressed and is of equal or greater importance. 7.4 Strengthen safeguards monitoring, evaluation, and completion reporting 7.4a) The World Bank should: Include performance indicators on environmental and social outcomes in project results frameworks and ensure systematic collection of data to monitor and evaluate safeguards performance. (Partially Agreed/Ongoing) No significant progress noted. 7.4b) Ensure that Implementation Completion Reports and IEG reviews of those reports rate and report effectively on the outcomes of safeguards and, for all projects with significant environmental and social effects, ensure the results are incorporated as an essential dimension when assessing achievement of the project’s development objective, as has already been done for IFC and MIGA. (Not agreed) 7.5 Improve systems and instruments for accountability and grievance redress IFC, MIGA, and the World Bank should: Seek greater symmetry in the structure of Bank Group accountability and grievance redress mechanisms. For the World Bank this would entail creation of a grievance redress and conflict resolution mechanism to complement the Inspection Panel. For IFC and MIGA this would entail a more independent compliance review process, ensuring that the CAO submits its audits directly to the Board. Management has not established a separate Grievance Redress Mechanism (GRM) redress but has created a Dispute Resolution and Prevention Facility (DRP) in OPCS to assist project teams utilize existing GRMs at the project or country level. Explicit guidance on how to access the GRMs and use the DRP Facility has not yet been issues, and the absence of a grievance mechanism for affected people from projects that lack GRMs is a critical omission leading to unequal treatment of people affected by different projects. 83 8. Phase II: The Challenge of Low-Carbon Development - Climate change and the World Bank LIST OF RECOMMENDATIONS 8.1 The World Bank and IFC should—Create incentives and mobilize resources to support effective pilot, demonstration, and technology transfer projects that have a clear logic of demonstration and diffusion. This will include mobilizing Global Environment Fund and other concessional funds to mitigate World Bank borrower risk; reshaping incentives for staff and managers; providing adequate resources for the design and supervision of complex projects; and making available specialized expertise in technology transfer and procurement through a real or virtual technology unit. 8.2 The WBG should—Place greater emphasis on large-scale energy efficiency scale-up, as measured by energy saved and generating capacity avoided. This includes support for efficient lighting and exploring the scope for accelerating the global phase-out of incandescent light bulbs. It includes continued and expanded support for reductions in transmission and distribution losses. And it includes proactive search by IFC for large- scale, catalytic investments in energy efficiency. There is scope to coordinate World Bank support for demand-side energy efficiency policies with IFC support for more efficient manufacturing and more efficient products. 8.3 The WBG should—Help countries find alternatives to coal power while retaining a rarely used option to support coal power, strictly following existing guidelines (including optimal use of energy efficiency opportunities) and being restricted to cases where there is a compelling argument for poverty or emissions reductions impacts that would not be achieved without WBG support for coal power. 8.4 The WBG should—Continue to explore, in the REDD context, ways to finance and promote forest conservation and sustainable use, including support for indigenous forest areas and maintenance of existing protected areas. 8.5 The World Bank should take the necessary steps to enhance the delivery of its guarantee products by taking actions to improve policies and procedures, eliminate disincentives, increase flexibility, and strengthen skills for the deployment of the products. It should assess the potential for greater use of partial risk guarantees to mobilize long-term financing for renewable energy projects, particularly in the context of feed-in tariffs or other premiums to support investment in renewable energy. 8.6 The Carbon Partnership Facility and other post-Kyoto carbon finance efforts should focus on demonstrating effective technical and financial approaches to boosting low-carbon investments. Funds and facilities should have clear exit strategies. 8.7 The WBG should—Measure projects‘ economic and environmental impact during execution and after closure and aggregate this information for analysis. For instance, 84 renewable energy projects should monitor capacity utilization, and energy efficiency projects should monitor energy savings. This may require the use of concessional funds to defray additional costs of monitoring by staff, clients, and project proponents. 8.8 The WBG should—Link these measures to a results framework that shifts the SFDCC toward a focus on outputs such as power produced, power access, forest cover, transit share of urban trips, rather than money spent. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG 8.1 The World Bank and IFC should—Create incentives and mobilize resources to support effective pilot, demonstration, and technology transfer projects that have a clear logic of demonstration and diffusion. This will include mobilizing Global Environment Fund and other concessional funds to mitigate World Bank borrower risk; High Substantial reshaping incentives for staff and managers; providing adequate resources for the design and supervision of complex projects; and making available specialized expertise in technology transfer and procurement through a real or virtual technology unit. 8.2 The WBG should—Place greater emphasis on large-scale energy efficiency scale-up, as measured by energy saved and generating capacity avoided. This includes support for efficient lighting and exploring the scope for accelerating the global phase-out of incandescent light bulbs. It includes continued and expanded support for High Substantial reductions in transmission and distribution losses. And it includes proactive search by IFC for large-scale, catalytic investments in energy efficiency. There is scope to coordinate World Bank support for demand-side energy efficiency policies with IFC support for more efficient manufacturing and more efficient products. 8.3 The WBG should—Help countries find alternatives to coal power while retaining a rarely used option to support coal power, strictly following existing guidelines (including optimal use of energy efficiency opportunities) Medium Medium and being restricted to cases where there is a compelling argument for poverty or emissions reductions impacts that would not be achieved without WBG support for coal power. 85 Level of Adoption Recommendations Mgmt IEG 8.4 The WBG should—Continue to explore, in the REDD context, ways to finance and promote forest conservation High High and sustainable use, including support for indigenous forest areas and maintenance of existing protected areas. 8.5 The World Bank should take the necessary steps to enhance the delivery of its guarantee products by taking actions to improve policies and procedures, eliminate disincentives, increase flexibility, and strengthen skills for the deployment of the products. It should assess the Substantial Substantial potential for greater use of partial risk guarantees to mobilize long-term financing for renewable energy projects, particularly in the context of feed-in tariffs or other premiums to support investment in renewable energy. 8.6 The Carbon Partnership Facility and other post-Kyoto carbon finance efforts should focus on demonstrating effective technical and financial approaches to boosting Substantial Substantial low-carbon investments. Funds and facilities should have clear exit strategies. 8.7 The WBG should—Measure projects‘ economic and environmental impact during execution and after closure and aggregate this information for analysis. For instance, renewable energy projects should monitor capacity Difference of Difference utilization, and energy efficiency projects should monitor Opinion of Opinion energy savings. This may require the use of concessional funds to defray additional costs of monitoring by staff, clients, and project proponents. 8.8 The WBG should—Link these measures to a results framework that shifts the SFDCC toward a focus on outputs such as power produced, power access, forest High Medium cover, transit share of urban trips, rather than money spent. MANAGEMENT RESPONSE 2012 8.1 Original Response: Ongoing/Agree This recommendation is consistent with the SFDCC. Key ongoing work consistent with the recommendation include: i) WBG expects to expand its partnership with GEF, which is well positioned to take on early research and development risks, through the recently established Technology Transfer Program and GEF/IFC Earth Fund; ii) Through the Clean Technology Fund (CTF), Scaling-up Renewable Energy Partnership (SREP) and Forest Investment Partnership (FIP), WBG is supporting technology scale-up with the help of innovative financing; iii) A Climate Technology Program, launched in September 2009, is exploring 86 the feasibility of climate technology innovation centers in developing countries as a way to stimulate locally relevant climate technologies and harness economic opportunities at the small and medium enterprise (SME) level (first centers already under development in Brazil, India, and Kenya); iv) A new MDTF has been established for supporting the introduction of CCS technologies and providing TA to clients; v) IFC‘s cleantech investment practice will be housed in the newly-created Climate Business Group. CLEANTECHNET is a practice group that meets virtually and in person to share knowledge and issues in the technology space; vi) The potential for additional initiatives to support these objectives will also be explored in the energy strategy. Mar 2012 response: The Clean Technology Fund (CTF) offers countries incentives to move forward with the demonstration, deployment, transfer, and replication of clean technologies that reduce greenhouse gas emissions. The CTF Trust Fund Committee has endorsed 13 CTF investment plans with funding allocation (and two additional investment plans without funding allocation), which are projected to leverage an additional $36 billion in co-financing on a $4.5 billion CTF investment. This projected leveraging ratio of more than 1:8 represents a significant investment in climate change mitigation in CTF partner countries. It is now proven that in underdeveloped but promising niche markets, even limited CTF resources can significantly increase low baseline investments and lay the groundwork for transformational change. For example, in Mexico, the CTF is accelerating the development of Mexico‘s private wind market by targeting the untapped wind power potential in the state of Oaxaca (more than 5,000 megawatts of world-class wind potential, of which only 88 megawatts are currently operational). The government plans to use the CTF funds to build a wind power generation plant and attract private commercial banks to provide debt financing for the construction and implementation of wind projects. The CTF investment in the MNA regional project is expected to support the deployment of about one gigawatt of CSP generation capacity, almost tripling current global levels. In addition, with support from the Scaling Up Renewable Energy Program in Low Income Countries (SREP), six low income countries (Ethiopia, Honduras, Kenya, Maldives, Mali, and Nepal) announced their intention to invest in renewable energy services as a means to grow their citizens‘ much-needed energy access and leapfrog into climate-friendly development. Kenya has submitted its investment plan for endorsement by the Trust Fund Committee. In providing support through CTF, SREP, and other funds for technology transfer, the World Bank has been stepping up technical assistance for elements of integrated resource planning, which evaluates the full range of alternatives—including such supply options as cogeneration, supply efficiency measures, and renewable energy resources as well as end-use energy conservation and efficiency—to provide adequate and reliable service to customers while minimizing long-term costs. This approach encourages governments to consider a new way of planning investments and expansion, including new technology options. The analytical support also enables a better understanding of trade-offs in opting for cleaner technologies. Accordingly countries have developed low carbon investment plans that allow them to adapt and apply newer technologies to their particular business context. 8.2 Original Response: Partially Agree WBG agrees with the general emphasis proposed, but does not agree with the specific action areas proposed. WBG does focus on large-scale or bundled EE projects to avoid high 87 transaction costs associated with small-scale investments. Estimates of energy saved are computed as part of the appraisal of projects with EE components. However, World Bank finds the recommendations on efficient lighting and T&D to be rather prescriptive and limited in terms of scope and impacts. They do not account for a variety of untapped EE scale up opportunities and available long-term EE potential in other sectors, on both the supply and demand sides, which could have much larger impacts, such as in district heating, industry, and municipalities. IFC intends to increase its climate-related lending from 10 percent of annual commitments in FY09 to 20-25 percent in FY13, and will undertake a proactive search for suitable investments. EE is expected to be a significant contributor to meeting this target. IFC will define an approach to estimating avoided emissions associated with its climate- related activity. IFC agrees with the potential for investments in manufacturing of more efficient products and is actively seeking such opportunities, having made several such investments in FY10. Mar 2012 response: The Bank continues to expand its non-lending (policy, institutional, and advisory) and lending portfolio in the areas of supply-side and demand-side energy efficiency, supported by efforts to improve knowledge management, cross-sectoral collaboration and sharing of best practices across the energy and other practices. The share of total WBG energy financing for low carbon projects rose from 42 percent to 51 percent over this period. FY11 marked an all-time record in WBG low carbon financing, with $5.9 billion in new commitments. Compared to a target average annual increase of 30 percent in financing for new renewable energy (new RE) and energy efficiency (EE) to FY11, amounting to total financing of about $6 billion over FY09-11, the WBG's financing for new RE and EE actually increased by about 48 percent per annum, and totaled some $9.2 billion. Energy efficiency intervention has seen large scale energy efficiency projects launched in middle-income countries, such as China and Mexico. In the case of China, highlights of projects include Shandong Energy Efficiency Project—a $150 million IBRD loan to support Energy Service Companies for energy efficiency investments. The Bank introduced market- based mechanisms of delivery and financing mechanisms to help the government achieve its ambitious energy intensity reduction targets. Lighting Africa, a joint IFC and World Bank program, is helping develop commercial off-grid lighting markets in Sub-Saharan Africa as part of the World Bank Group‘s wider efforts to improve access to energy. In the case of Mexico, the Mexico Efficient Lighting and Appliances Project is a catalyst for the transformation of the residential lighting and refrigeration subsectors, by supporting the replacement of 45 million inefficient incandescent light bulbs and 1.7 million old and inefficient refrigerators and air conditioners with more efficient models across a range of beneficiaries, mostly lower income groups. In the context of low income countries, innovative solutions have also been implemented. Lighting Africa is mobilizing the private sector to build sustainable markets to provide safe, affordable, and modern off-grid lighting to 2.5 million people in Africa by 2012 and to 250 million people by 2030. Since 2007, over 190,000 solar portable lamps which had passed Lighting Africa quality tests were sold in Africa, providing more than 950,000 people with cleaner, safer, better lighting and improved energy access. The Bank is providing technical and financial support to electricity distribution companies in several countries in the design and implementation of projects aimed at improvement of 88 commercial performance, particularly reduction of non-technical losses increase in collection rates. Several projects involving ICT are either under implementation or under preparation in electricity utilities in Brazil, Dominican Republic, Ghana, Haiti, Honduras, India, Malawi, Mozambique, Uzbekistan and Vietnam. In the Dominican Republic, an AMI-based revenue protection project designed and implemented by the three distribution companies in the country with technical assistance provided by the Bank resulted in the reduction of total losses by 4 percentage points in less than one year. Project investments (meters, communication devices and software package to remotely collect and analyze metering data) were paid back in 7 months. More importantly, the design of the project assures sustainability of the improvements achieved through its implementation. 8.3 Original Response: Ongoing/Agree The policy proposed is consistent with SFDCC criteria for coal investments. SFDCC criteria for coal investments have been clarified in the ―Operational Guidance for the World Bank Group Staff: Criteria for Screening Coal Projects under the Strategic Framework for Development and Climate Change‖ which took effect on April 15, 2010. The specific stress on ―optimal use of energy efficiency opportunities‖ presented in the IEG recommendation seems unnecessary as there are no priority criteria either in SFDCC or the Operational Guidance and all required criteria must be adequately addressed. The Operational Guidance makes clear that coal investments should focus on cases where there is a compelling argument for poverty reduction and a clear need for WBG support. Mar 2012 response: The draft energy sector strategy sets as its two goals the expansion of access to reliable modern energy services and a shift to more sustainable energy sector development. Building upon the principles set forth in Development and Climate Change: A Strategic Framework for the World Bank Group (SFDCC), an operational guidance note for screening coal projects has been issued. The WBG's renewable energy portfolio more than doubled from $2.9 billion in FY06-08 (18 percent of total WBG energy financing in the period) to $6.6 billion in FY09-11 (22 percent). Low carbon projects likewise more than doubled from $6.8 billion to $14.9 billion over the same period, and the share of total WBG energy financing for low carbon projects rose from 42 percent to 51 percent over this period. 8.4 Original Response: Ongoing/Agree By definition, REDD+ includes reducing emissions from deforestation and forest degradation and addressing the role of conservation, sustainable management of forests and enhancement of forest carbon stocks. World Bank is assisting countries to engage in REDD+ activities through two programs: FCPF and FIP. Both of these will contribute to financing and promoting forest conservation and sustainable use, including support for indigenous forest areas and forest conservation. FCPF involves 37 countries, and has mobilized US$160 million for capacity building and performance-based payments to pilot projects which aim to open financial flows for sustainable management of forests and land. FIP, funded at approximately $600 million, will pilot programmatic investments to reduce deforestation and forest degradation, promote sustainable forest management, and conserve forests in Brazil, Burkina Faso, Democratic Republic of Congo, Ghana, Indonesia, Lao PDR, Mexico, and Peru. 89 Mar 2012 response: IEG‘s and other reviews are finding that forests under indigenous management and protection area status contribute most effectively to protection and therefore emission reductions. These findings have been duly noted and efforts are being made to reflect them in policy processes, including those of the FCPF (now capitalized at $433 million) and FIP (now capitalized at $624 million). For example, the FCPF is increasing its capacity building program for Indigenous Peoples so they engage more fully in the national and international debates on REDD+, is conducting regional and global dialogues with Indigenous Peoples, has made room for five Indigenous Peoples‘ representatives to participate in the Participants Committee meetings, and is advocating and supporting the inclusion of indigenous representatives in the management arrangement structures for REDD+ in each country and the formulation and implementation of national Readiness Preparation Proposals. The FIP is putting in place a $50 million Dedicated Grant Mechanism for Indigenous Peoples and Local Communities, which will support investments in indigenous areas by Indigenous Peoples that contribute to REDD+. This mechanism is expected to become operational mid-2012 and to complement the national efforts presented in the countries‘ Investment Plans. A number of these Investment Plans will themselves include support for on-the-ground programs that benefit Indigenous Peoples, such as land titling of indigenous areas. Finally, the FCPF Carbon Fund is expected to provide performance-based payments to a number of national or sub- national REDD+ programs, some of which are co-financed by the FIP, that entail support to Indigenous Peoples. 8.5 Original Response: Partially Agree In response to IEG‘s evaluation of WBG guarantees, Management has been engaged in ongoing discussions on opportunities to optimize the delivery of WBG guarantee instruments and has taken action to introduce greater flexibility in the use of Bank guarantee instruments in response to dynamic country and client needs and market developments. An MOU was recently signed between World Bank and MIGA to provide incentives to staff to collaborate and a similar agreement is being worked on with IFC. World Bank is working to increase potential for greater use of partial risk guarantees for RE projects and is allocating more staff and resources accordingly. World Bank feels that the delivery of RE guarantee products should not single out the feed-in-tariff instrument, as the effectiveness and efficiency of its application varies across market structures and varies across countries depending on their energy access levels, with the potential to result in high energy costs that will need to be borne by consumers. Mar 2012 response: During FY11 and FY12, there has been significant progress towards meeting the recommendations. An Approach Paper on ―Modernizing the World Bank‘s Operational Policy on Guarantees‖ was prepared and has been discussed with the Bank‘s Committee on Development Effectiveness (CODE) on October 26, 2011. The Bank is carrying out a three month global consultation process beginning in December 2011 and will develop a policy paper including revised Operational Policy and Bank Procedure statements (draft OP and BP 14.25, Guarantees) – planned for presentation to the Board during FY12. SDNVP has actively engaged with senior management to alter performance reviews for Bank staff to increase incentives linked to leveraging private sector financing. 90 FEUFS has developed and rolled out a training program for Bank staff focused on improving project financing skills within the Bank – including guarantees training. FEUFS is working with SEGEN and regional energy teams to carry out greater assessment of potential for use of PRGs for renewables. There are new projects with approved PCNs in Tanzania (PRG for 100MW wind farm), Kenya (PRG for 300MW wind farm) and Tanzania (PRG for 358MW hydro power). Several more are being worked on. FEUFS and SEGEN are collaborating on a new PRG instrument to support a feed-in tariff regime coordinated with KfW and Deutsche Bank. First pilot country being considered is Uganda. 8.6 Original Response: Ongoing/Agree CPF and FCPF were clearly established for the purposes described. Beyond these facilities, World Bank is invited to explore how to facilitate developing countries‘ further access to the carbon market and expand the reach of market mechanisms in land use, including in agriculture. Work is underway to develop successor facilities to CDCF and the BioCarbon Fund. Each fund and facility has its own clear exit strategy corresponding to when its capital has been fully committed. Regarding CPF, each tranche is to be established based on an assessment of the needs for further methodology development and piloting of new approaches to scale up the use of market mechanisms. Mar 2012 response: The work undertaken by the CPF continues to be very focused on supporting countries‘ efforts to accelerate their transition to a lower carbon development path, by developing national or sub-national mitigation programs in renewable energy, methane capture, and energy efficiency. Likewise, the more recent facilities launched in December 2011, the Carbon Initiative for Development and a new tranche of the BioCarbon Fund, are geared to lending their support to scale up investments in mitigation in low-income countries, including through increasing access to low-carbon energy and expanding land-based mitigation activities. These carbon finance activities are more and more closely linked to strategic low-carbon development programs supported by the Bank through multiple instruments, including DPLs. 8.8 Original Response: Ongoing/Agree A long-term results framework for SFDCC is under development, as stated in the Interim Progress Report for SFDCC, May 2010. The SFDCC results framework will be outcome- oriented and is currently being developed in a consultative manner as envisioned in SFDCC. In addition to tracking WBG actions at the input level, the new results framework is being designed to track outputs, outcomes and impacts related to WBG actions. Potential indicators at all of these levels (including the suggested output indicators: power produced, power access, forest cover, transit share of urban trips) are currently being assessed for their feasibility, simplicity, and suitability in communicating results at different levels and scales. Separate results frameworks are under development for the CIF with an emphasis on impact, outcome and output indicators. Results chains link projects to the CIF final outcomes through pilot country outputs and outcomes, program replication outcomes, and transformative impact. The Multilateral Development Banks are currently in a process to identify reliable indicators to measure results and achievements at each level. 91 Mar 2012 response: A long-term results framework has been prepared as part of the SFDCC Close-out report. It has been developed in a consultative manner and is tailored to track results at three tiers. The first and highest tier sets the global context for the results framework organized around the four areas – climate resilience, climate mitigation, climate finance, and knowledge /capacity / innovation / partnerships. In the second tier, the focus is on outputs and outcomes supported by WBG operations which include support to country programs, policies, and knowledge activities as well as to global public goods. The third tier focuses on operational effectiveness of WBG programs to achieve low carbon and climate resilient development results. For lending activities this includes the quality and implementation performance of the climate-related portfolio, as defined by a new system that the World Bank has adopted for tagging projects at board approval as providing climate mitigation or climate adaptation co-benefits. For non-lending activities, this includes the integration of climate change considerations into Country Assistance Strategies (CASs) and sector strategies. Finally, Tier III includes the integration of CC considerations into various knowledge products including AAA, guidelines, and tools. Results frameworks for the CTF, FIP, PPCR and SREP have been approved by the Trust Fund Committees. The results frameworks are designed to operate: (i) within existing national monitoring and evaluation systems; and (ii) the MDBs‟ own managing for development results (MfDR) approach. With the approval of the results frameworks as living documents, the Trust Fund Committees established the basis for an adaptive M&E approach. The data generated through the M&E system should allow countries to take corrective action based on information/evidence. The adaptive management approach requires a constant and sustained feedback mechanism which allows countries to reflect on measures, approaches, methodologies etc. and initiate change when data or observations point towards the need to adapt to changing circumstances. IEG RESPONSE 2012 8.1 The recommendation focused not on the sheer volume of low carbon projects, but rather on the extent to which they effectively demonstrate technologies and catalyze their diffusion, or address client risk aversion through grant funding. The CTF investment plans potentially give scope for this approach. Some of the CTF-supported projects are consistent with this, notably the large-scale effort to promote concentrating solar power. The recommendation refers also to reshaping incentives and shifting resources to encourage projects and programs with a clear logic of demonstration -- that is, projects which target specific kinds of knowledge creation and behavioral change leading to technology adoption. Change of this sort is less evident. 8.2 The Mexican project exemplifies the kind of high-return energy efficiency investment advocated by the Evaluation. And the expression of goals and progress in terms of people served (Lighting for Africa) and appliances replaced (Mexico) is in line with the recommendation that progress be measured in outcomes rather than expenditure – an approach that IEG would like to see applied to the entire portfolio. There is still 92 tremendous scope for expanding these lines of actions, as outlined in a Bank study which found that efficient lighting could free up a quarter of Sub-Saharan Africa‘s generating capacity, and in the Bank-supported Low Carbon study for India, which advocates reducing transmission and distribution losses from 29% to 15%. 8.3 Treatment of coal power is a sensitive issue. The adoption of the Guidance Note on coal is an important step forward, but its application needs to be informed by specific guidelines on how to shadow-price carbon and local environmental damages, and how to assess system-wide options for energy efficiency. IEG acknowledges that the WBG has boosted commitments to renewable energy and energy efficiency to high levels, and is devoting staff and analytic attention to natural gas, a potential coal alternative. Further progress on this recommendation will be shaped by the way that the issue is addressed in the still- pending Energy Strategy. 8.4 Although yet to be evaluated, the Bank‘s support for the FCPF and for FIP represent a serious effort to use REDD to promote forest conservation, and as noted there has been outreach to indigenous communities, 8.5 The Bank has continued to make important steps toward enhancing its guarantee products, streamlining processing, and strengthening staff skills. IEG notes that an approach paper on modernizing guarantees has been discussed with the Board and that the Bank is presently undertaking broad consultations on the agenda. Following the consultations, the implementation phase can be expected to contribute to better realizing the potential of the Bank‘s guarantee instruments. IEG also favorably notes the crop of renewable energy guarantee projects under preparation and discussion. 8.6 The Carbon Finance Unit is demonstrating or planning to demonstrate new technical approaches (soil carbon, forest carbon) and financial approaches (programs of activities under the CPF; Partnership for Market Readiness). IEG acknowledges complementary efforts by the World Bank to shape the regulatory and market environment for carbon projects. 8.8 At this writing, the draft long-term results framework for the SFDCC has not been disclosed, so IEG is unable to rate it. The Environment Strategy has a results framework which includes tracking the GHG impact of World Bank projects, a notable step forward. It also will track the number of people newly gaining access to renewable electricity, and generation capacity of renewable energy due to Bank interventions, but does not track energy efficiency. 93 9. An Independent Assessment – The World Bank’s Involvement in Global and Regional Partnership Programs LIST OF RECOMMENDATIONS STRATEGIC AND POLICY FRAMEWORK 9.1 The Bank should continue to work with its global partners to develop shared understanding and information about the role and nature of GRPPs in the new aid architecture, from the inception and establishment of new programs through their independent evaluation and impact assessment. 9.2 The Bank should develop a formal policy on engaging with GRPPs, including among other things:  Standard approval processes for Bank engagement with new partnership programs, independent of how they are financed.  A policy for hosting the management units (secretariats) of GRPPs inside the Bank. FINANCING 9.3 The Bank should revise the eligibility criteria for receiving DGF grants, taking into account the role of GRPPs and the DGF in the new aid architecture and the Bank‘s mixed experience with the existing DGF criteria. SELECTIVITY 9.4 The Bank should not formally engage in new GRPPs that do not have well-articulated governance arrangements, theory of change, monitoring and evaluation framework, and resource mobilization strategy at the outset. 9.5 The Bank should have an explicit engagement strategy for each GRPP in which it is involved, including the following elements:  The expected roles of the Bank in the program at both the global and country levels, along with the expected duration of these roles  How the program‘s activities are expected to be linked with the Bank‘s country operations  How the risks to the Bank‘s participation will be identified and managed, including conflicts of interest among the Bank‘s roles in the program. 9.6 The approval of new programs should include criteria against which programs will subsequently be evaluated, including:  Evidence of an international consensus  Evidence of developing country demand  Subsidiarity  The absence of alternative sources of supply. OVERSIGHT AND RISK MANAGEMENT 9.7 The Bank should strengthen its oversight and risk management of GRPPs by:  Establishing and maintaining a definitive, continuously updated, and searchable database of the GRPPs in which the Bank is currently engaged  Requiring standard terms of reference for Bank staff serving on partnership boards  Preparing Bank-wide guidelines for task team leaders of GRPPs.  Providing sufficient budgetary resources for effective oversight and risk management.  Requiring each vice presidency to produce an annual report on its involvement in GRPPs, including new entrants and exits. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG 9.1 The Bank should continue to work with its global partners to develop shared understanding and information about the role and nature of GRPPs in the Medium Medium new aid architecture, from the inception and establishment of new programs through their independent evaluation and impact assessment. 9.2 The Bank should develop a formal policy on engaging with GRPPs, including among other things:  Standard approval processes for Bank engagement with new partnership programs, Substantial Substantial independent of how they are financed.  A policy for hosting the management units (secretariats) of GRPPs inside the Bank. 9.3 The Bank should revise the eligibility criteria for receiving DGF grants, taking into account the role of GRPPs and the DGF in the new aid architecture and Substantial Substantial the Bank‘s mixed experience with the existing DGF criteria. 9.4 The Bank should not formally engage in new GRPPs that do not have well-articulated governance arrangements, theory of change, monitoring and Medium Medium evaluation framework, and resource mobilization strategy at the outset. 95 Level of Adoption Recommendations Mgmt IEG 9.5 The Bank should have an explicit engagement strategy for each GRPP in which it is involved, including the following elements:  The expected roles of the Bank in the program at both the global and country levels, along with the expected duration of these roles Substantial Substantial  How the program‘s activities are expected to be linked with the Bank‘s country operations  How the risks to the Bank‘s participation will be identified and managed, including conflicts of interest among the Bank‘s roles in the program. 9.6 The approval of new programs should include criteria against which programs will subsequently be evaluated, including:  Evidence of an international consensus Medium Medium  Evidence of developing country demand  Subsidiarity  The absence of alternative sources of supply. 9.7 The Bank should strengthen its oversight and risk management of GRPPs by:  Establishing and maintaining a definitive, continuously updated, and searchable database of the GRPPs in which the Bank is currently engaged.  Requiring standard terms of reference for Bank Substantial Substantial staff serving on partnership boards.  Preparing Bank-wide guidelines for task team leaders of GRPPs.  Providing sufficient budgetary resources for effective oversight and risk management.  Requiring each vice presidency to produce an annual report on its involvement in GRPPs, including new entrants and exits. MANAGEMENT RESPONSE 2012 Original Response Strategic and Policy Framework 9.1 Partially Agreed. Management will continue to work with global partners to develop a shared understanding of the role of GRPPs, in line with the Paris and Accra agendas. 96 Management notes that this work will not translate into a specific product such as an over-arching global partnership strategy, but rather complement and inform planned work on GRPP selectivity and improving links to country-level objectives (see below). 9.2 Partially Agreed. Management does not consider it would be appropriate to develop a formal operational policy at this time. Instead, Management plans to develop a partnership management framework that will entail guidelines and best practices for Bank units involved in partnership programs, including governing bodies and secretariats. This framework will include strengthened standard approval processes for new and restructured partnership programs and expected to be completed in FY12. Financing 9.3 Agreed. Management plans to review the eligibility criteria for receiving grants as part of the ongoing strategic reorientation of the DGF, which is increasingly focused on support for innovative partnerships. Management will report to the Board on this issue in June 2012 through the FY13 DGF Board document. While agreeing with the recommendation, just to be clear as to Management‘s commitments going forward, Management does have concerns and disagreements regarding some of the more detailed suggestions outlined in Chapter 8. Specifically, the suggestions in Table16 are problematic as revisions to specific criteria at this stage may prejudge the ongoing reorientation process. Nevertheless, we recognize the importance of the underlying principles of most of the suggestions and will consider how the objectives might be met in a manner appropriate to the DGF reorientation. Management will explore the recommendation to eliminate the arm‘s length relationship given the challenges in applying the arrangement consistently. However, the original objective behind the arm‘s length relationship— minimizing risks of conflict of interest—remains of paramount importance and would need to be met through other means. Management believes that the suggestion to require a formal charter would not be applicable in all cases. As has been indicated in the past, Management does agree that depending on the nature of the partnership, clearly articulated roles and responsibilities should be a requirement whether in a charter or another instrument. As part of this, Management would further define the circumstances under which the Bank should or should not be a voting member of the partnership governance structure. Selectivity 9.4 Partially Agreed. Management agrees that all partnerships should have well- articulated monitoring and evaluation frameworks and appropriate governance arrangements at the outset. These issues are currently being addressed in the Partnership Review Note (PRN) prepared for new GRPPs and that process will be strengthened in line with the partnership management framework noted above (see response to Recommendation 2). Management does not agree that resource mobilization strategies should be required for every partnership. Instead, the requirement would depend on the nature of the partnership. We agree resource mobilization strategies should be required at the outset where financial exit While Management recognizes the intent behind the recommendation, it does not agree with the requirement for well-articulated theories of change. As international priorities and the aid architecture evolve, partnerships will need to retain need some degree of flexibility to evolve in response. Requiring elaborate ex-ante plans is 97 impractical given the number of actors involved in many partnerships and may actually undermine one of the objectives behind partnerships: timely response to global challenges. Instead, Management will explore how partnerships might implement regular internal reviews, linked to results frameworks; to achieve the underlying objective from the partnership is an end objective, as is the case of the DGF. Explicit resource mobilization strategies are less appropriate in other cases. Some partnerships may be created to respond to a very specific issue and should sunset when the objective has been met, or be mainstreamed into other existing funds. Resource mobilization requirements would also need to be considered in the context of subsidiarity principles, increasing fragmentation of the aid architecture, and competition for resources. 9.5 Partially Agreed. Management sees the need for a more holistic approach to partnerships as envisioned by explicit engagement strategies and will work to provide guidance to task team leaders of new GRPPs, informed by the work on the partnership management framework noted above and the work of the Matrix Leadership Team noted below in response to recommendation 6. However, Management would modify the recommendation on country-level linkages. Instead of an explicit ―link with the Bank‘s country operations,‖ the design of country-level linkages within the engagement strategy should be tailored to cases where the Bank will be the primary interface at country level, versus others where the Bank‘s operational role is limited, or where the program objective, such as global knowledge or research, has an indirect linkage to Bank country operations. 9.6 Partially Agreed. Management agrees that approvals of new programs should include clear criteria against which the programs will subsequently be evaluated. However, Management does not agree in the entirety with the recommended criteria. Management recognizes the intent behind the criteria—most of which are already applied to new partnership approvals, though not always consistently. Past reports suggest documentation at approval is quite thorough but follow-through monitoring and reporting remains the real issue. The Matrix Leadership Team work program includes the selection and monitoring of GRPPs. It will develop and implement a set of actions to address this issue and report on progress in the context of the regular updates to the Board on internal reforms. While international consensus has been used in the past, Management would call for the need for ―collective action‖ to replace ―international consensus‖ as consensus may be increasingly open to interpretation as the aid architecture evolves. Management disagrees with the specific requirement for evidence of developing country demand as many partnerships are created to address global and regional public goods, stimulating incentives (and demand) for action at the country-level. Instead, the link to the ultimate country beneficiaries should be defined in terms of how the partnership programs will link to country-level priorities where clear evidence of demand is not available exante. Management agrees in principle with the subsidiarity and absence of alternative sources of supply, though it would not ask for strict adherence to these two criteria (which are in practice difficult to prove or disprove). Management agrees that partnerships should not duplicate what can be achieved by other existing instruments. Nevertheless, coordination among implementers is sometimes the objective behind partnerships driven by broader 98 international calls for action. In this context, some degree of flexibility is called for. In terms of alternative sources of financial ―supply,‖ some partnerships are created to help leverage other sources of finance. As part of the efforts going forward, Management will look at defining better guidelines so that duplication and complementarity (versus substitution) can be assessed more readily in the selectivity and approval processes. Oversight and Risk Management 9.7 Partially Agreed. Management agrees that strengthening oversight and risk management is needed. Guidelines for GRPP task team leaders will be prepared as part of a broader exercise to strengthen the overall partnership management framework (see response to recommendation 2 above), linking selectivity, quality assurance, resources and risk management, monitoring and reporting, and governance (including clearly defined roles and responsibilities). Management agrees that Bank staff serving on partnership boards should have terms of reference, following standard Bank policies; however, Management disagrees that all terms of reference have to be ―standard.‖ Management does not agree to mandating an annual report on GRPPs from each VPU. Instead, the immediate focus is better placed on developing better systems for information flows, and for management reporting and decision-making. Establishing a separate, definitive, searchable database is not warranted. However, Management will explore ways to improve current information systems and reporting so that relevant information on partnerships can be incorporated into and then easily retrieved from other existing reporting mechanisms. MAR 2012 This report, which is an assessment of the Bank‘s engagement in a subset of partnership programs of global and regional scope, was issued in spring 2011. The Management Response expresses appreciation for IEG‘s work in this area, and indicates that most of the recommendations are partially to substantially agreed. Some of IEG‘s recommendations contain sub-recommendations, some of which Management plans to fully adopt and some not. Hence, the actions to be taken under these recommendations in the coming years may respond more to the spirit than the letter. Overall, Management agrees with IEG‘s message that stronger oversight over partnership programs is warranted, including upfront selectivity and ongoing management of both individual programs and wider portfolios. To achieve this, Management is currently preparing a Partnership Program Management Framework paper, to be discussed by the Board in H2FY12. However, Management‘s view of these programs is also hat they are closely related to their funding instruments and other Bank operational work. Hence, systems, databases, business process, and strategy development needs to be linked to the ongoing work on Trust Fund Reform and on the Bank‘s operational and strategic agendas, as opposed to freestanding approaches. The population of GRPPs identified by IEG includes both internally and externally managed programs, and they are supported by Trust Funds, Financial Intermediary Funds, and the Bank‘s Development Grant Facility. Intensive work has been carried out on all three types of funding instruments in recent years. The DGF Reform process 99 has resulted in a more dynamic portfolio in line with DGF‘s intended catalytic role, with exits of longstanding recipients. IEG has recommended revisiting the DGF criteria (the one recommendation to which Management fully agreed), and Management will raise this topic with the DGF Council. The next phase of DGF reform is focusing on better results frameworks for programs receiving DGF grants. A FIF Framework paper is currently under preparation by Management, and FIFs provide the largest volume of funding to the GRPP portfolio. Finally, the Board is regularly briefed on the Trust Fund Reform process, which is also increasingly focused on mainstreaming trust funds into Bank operations and on efforts to improve results and reporting. The forthcoming new Partnership Program Management Framework will build on this work by creating guidelines for program-level aspects of GRPPs and other similar programs. These guidelines will cover selectivity criteria, the applicability of Bank policies and procedures, managing partnership structures (including secretariats), providing appropriate Terms of Reference for Bank staff involved in governing bodies, and promoting global-country linkages. While much of this work pertains to internal Bank procedures, external partners are also being consulted to prepare this Framework. IEG RESPONSE 2012 Management‘s actions in response to IEG‘s evaluation are a work in progress. IEG agrees with Management‘s ratings as indicative of where more work is currently being done – those areas rated ―substantial‖ as opposed to ―medium‖. IEG reserves judgment on the outcomes of this work, such as the preparation of the new Partnership Program Management Framework (PPMF) and the revision the DGF eligibility criteria, until these are completed and start to be implemented by the Network and Regional VPUs responsible for partnership programs and by the programs themselves. Overall, IEG is pleased to see that Management is doing a substantial amount of work on implementing its policy agenda to promote effective partnership arrangements in comparison with the previous 2–3 years. IEG also interprets these ratings against what Management agreed in the cases where Management did not fully agree with IEG‘s recommendations. For example, IEG recommended that the Bank should develop a formal policy on engaging with GRPPs — a recommendation with which the majority of EDs who commented on the recommendation agreed. IEG hopes that the preparation of the Partnership Program Management Framework will end up being the first step towards a formal policy. IEG remains doubtful that the Bank‘s accountability for governance, management, and results of the GRPPs in which it is involved will be sufficiently accepted and exercised Bank-wide in a decentralized Bank in the absence of a formal policy, like, for example, OP/BP 14.40 on trust funds. IEG sees a particular and pressing need for a formal policy on hosting the management units (secretariats) of GRPPs in the Bank in order to address the existing confusions regarding their status. IEG agrees that systems, databases, and business processes for engaging with GRPPs should be linked to the ongoing work on Trust Fund and DGF reforms. There should be one reliable 100 Bank-wide system for tracking GRPPs and the sources of funds that finance their activities. However, trust funds and DGF grants are not programs in and of themselves, although they have often been labeled as such. They are dedicated sources of funds to support GRPPs and other activities agreed between the donors, the Bank, and other partners. They are like the fuel in an automobile. And like a car and its driver who steers the car towards a destination, it is the programs, their governance and their management that produce the results with the aid of the fuel provided by trust funds, DGF grants, and other resources dedicated to the programs. Partnerships are first and foremost about building relationships among the partners, and IEG found in its recent evaluation that past systems, databases, and business processes had put excessive emphasis on the funding of the programs to the relative neglect of forging strong institutional relationships among the program partners. The essence of partnership programs, as opposed to other activities that trust funds and DGF grants support, is shared governance. IEG continues to recommend that the PPMF should focus, at least in the first instance, on partnership programs with shared governance, for the three reasons given on pages 12–14 of our evaluation: (a) such programs challenge the Bank‘s traditional programmatic and financial accountability mechanisms; (b) the Bank is dedicating increasing amounts of senior management time to their governance; and (c) the Bank expects the programs‘ activities to be appropriately linked to the Bank‘s country operational work. In addition, the Bank can reasonably expect more requirements from such programs when they are established, such as a charter or some other constitutive document which defines the partnership. Everyone in the Bank, including IEG, is dependent on the systems and databases that Bank Management establishes to keep track of the Bank‘s activities, in this case GRPPs. Since Bank Management appears to have decided to include partnerships without shared governance in the Bank-wide database, IEG would ask that those partnership programs with shared governance should be readily identifiable, and extractable from the database. This involves each program answering a simple question: Is the consultative group, program council, or steering committee, etc. that has been established to oversee the program a collaborative or a consultative body? Do the members of the body have shared responsibility for programmatic oversight and shared accountability for results (collaborative), or does one partner, i.e. the Bank in the case of in-house programs, simply listen to what the other partners have to say while continuing to make the major governance and management decisions and to accept accountability for results (consultative). Authority and accountability should be aligned. Each partnership should have to declare on which side of this collaborative-consultative fence it stands, so that the Bank can avoid the continuation of situations in which it has little or no authority but most of the accountability. IEG also hope that processes will finally be put in place to ensure that TTLs provide both CFP and IEG with electronic copies of recently completed evaluations within one month of their completion, and that copies of these evaluations will be easily accessible on the Bank‘s intranet. IEG has recently shared its own database of recently completed evaluations of GRPPs with CFP as a foundation on which to build this system. These are some of the things that IEG will be looking for when the new Partnership Program Management Framework is presented for Bank-wide review and to the Board. 101 102 10. Growth and Productivity in Agriculture and Agribusiness: Evaluative Lessons from the World Bank Group Experience LIST OF RECOMMENDATIONS 10.1 Synergies and complementarities  Step up IFC‘s engagement in Sub-Saharan Africa, including support for public-private partnerships and adapting the integrated trader-processor model for more effective use with small-to-midsize indigenous companies in the agriculture-based economies.  Set up a knowledge network that links agriculture and agribusiness supply-chain specialists across the World Bank Group to strengthen communication and collaboration among sector departments within the Bank and IFC, as well as across the World Bank Group.  Work with partners to ensure that CGIAR research and other global and regional efforts are translated into benefits in client countries, and facilitate partnerships among countries to encourage south-south knowledge exchange. 10.2 Knowledge and capacity building  Ensure sufficient quantity and quality of Bank AAA and IFC advisory services in agriculture-based economies, link them closely to lending, and use them to build counterpart commitment and to address constraints along the production chain.  Establish mechanisms to confirm ex ante if project M&E frameworks are adequate, with clear, relevant, and realistic objectives; thorough cost-benefit analysis; appropriate indicators; and adequate baseline data.  Review the human resource base and skill gaps (also in light of the increased lending) and develop and implement a strategic plan to enhance the technical and policy skills of Bank and IFC staff working in the agriculture sector, particularly in agriculture- based economies. 10.3 Efficiency and sustainability  Increase World Bank Group support to medium-term expenditure planning to help ensure the adequacy of funding for operations and maintenance, and work with clients to ensure sustainable financing—including cost recovery where appropriate—for irrigation, transport, and research and extension services.  Take stock of experience in water management and crop technologies in rain-fed areas to inform future World Bank Group support.  Ensure that gender concerns are adequately mainstreamed and monitored in World Bank and IFC agriculture operations.  Expand the application of IFC Performance Standards to material biodiversity and other environmental and social aspects along the supply chain for primary suppliers (and for secondary suppliers to the extent the client has leverage), and enhance IFC support to the development and application of internationally accepted commodity certification systems. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG 10.1 Synergies and complementarities High Medium 10.2 Knowledge and capacity building Medium Negligible 10.3 Efficiency and sustainability High Substantial MANAGEMENT RESPONSE 2012 10.1 Synergies and complementarities (HIGH) Original Response  Ongoing/Agree IFC‘s business model in the Region is evolving to be more active in IDA countries and IFC is extending its reach to small-scale farmers and small and medium enterprises through: (i) direct investments in larger companies with a significant development reach to farmers and SMEs; (ii) indirect financing of medium-size companies and cooperatives through financial intermediaries; and (iii) indirect financing of smaller size agribusiness farms and enterprises through risk-sharing facilities with financial intermediaries. IFC now has an agribusiness anchor in Africa with a dedicated team of 9 full-time and 4 part- time staff, which is double that of the previous year, and IFC has more than doubled its investments in African agribusiness8 in the past three fiscal years, reaching a record $270 million in fiscal 2010.  Ongoing/Agree Management will broaden the existing informal World Bank Group thematic group on food 104 safety to include agribusiness supply-chain specialists in the Bank and IFC. Bank-IFC collaboration on agriculture has increased since the 1998–2008 IEG review period. Recent examples of joint work include: (i) preparation of the World Bank Group Agriculture Action Plan 2010–2012; (ii) inclusion of both a public and private sector window in the recently launched Global Agriculture and Food Security Program (GAFSP); (iii) the Responsible Agro-Investment (RAI) initiative, including the recently launched RAI Toolkit, and; (iv) preparation of a new Bank Group framework for engaging in oil palm. In addition, there is increasing Bank-IFC in-country coordination. Examples include: (i) a review of smallholder participation in the Liberian rubber sector; (ii) possible interventions in the cocoa sector in Côte d‘Ivoire; (iii) commercial agricultural development in Ghana; iv) a first loss agribusiness finance facility in Cambodia; and (v) an agribusiness logistics study in the Philippines.  Ongoing/Agree Since the end of 2008 (i.e., after the IEG evaluation period), there has been a significant reform of the CGIAR. A major mechanism to improve the effectiveness and relevance of the CGIAR‘s science and facilitate partnerships is the establishment, currently under way, of a relatively small number of high-impact ―mega-programs,‖ to replace the many, often fragmented, research programs of the past. Criteria which will guide approval of the mega programs include quantity and quality of partnerships included in design and implementation. Institutionally, the major reform is to separate the CGIAR ―doers‖ from the ―funders,‖ including the recent creation of the Consortium of CGIAR Centers (i.e., the ―doers‖), and the CGIAR Fund Council (i.e., the ―funders‖), supported by a Fund Office, which, like the previous CGIAR Secretariat, will be hosted by the Bank. MAR 2012 IFC’s engagement in Sub-Saharan Africa (AFR). IFC RESPONDING SEPERATELY Collaboration among sector departments within the Bank and IFC. Management has broadened informal links with the IFC. In addition to the examples provided in the original response, new examples of joint Bank-IFC work include: (i) Preparation and implementation of the IFC Agribusiness Strategic Action Plan FY12-14, (ii) Preparation of the forthcoming World Bank Group Agriculture Action Plan:2013-2015; (iii) Completion of The World Bank Group Framework and IFC Strategy for Engagement in the Palm Oil Sector; (iv) Regular joint meetings on the IFC‘s agribusiness strategic action plan and implementation practice which includes Bank staff and staff from both the advisory and investment arms of the IFC; (v) Cross institution learning between staff from the agriculture and private sector development sectors 105 of the Bank and staff from IFC in the development of several new Bank agriculture commercialization projects (for example in Ghana12, Burkina Faso13, Senegal14 and Tanzania15) (vi) Inclusion of both a public and private sector windows in the recently launched Global Agriculture and Food Security Program (GAFSP); (iv) The Responsible Agro-Investment (RAI) initiative, including the recently launched RAI Toolkit; and (v) Preparation of the Ukraine Agriculture Investment Policy Note. CGIAR research translated into benefits in client countries. The CGIAR continues its reform process with key components being: (i) the formulation of a research portfolio consisting of larger CGIAR Research Programs (CRPs) to reduce fragmentation of effort; (ii) promoting partnerships to draw on comparative advantage in research efforts; and (iii) improving research relevance by better alignment to country. Implementation of an agreed Strategy and Results Frameworks for each CRP is underway. As of end of 2011, a total of 11 CRPs have been approved by the CGIAR Fund Council. Each approved CRP involves a broad range of partners that include not only research institutions, but also national agencies that are engaged in development activities. The Global Rice Science Partnership Program (or Rice CRP), the first CRP to be implemented, involves over 300 research and extension agencies as partners in developing countries. The program not only generates technologies but also serves as a hub for knowledge exchange on rice. The Bank is supporting or participating in other global and regional research programs that benefit client countries. World Bank projects are supporting regional research in West Africa, through implementation of the West African Agricultural Productivity Program), and in Southern Africa, through the preparation of the Agricultural productivity program for Southern Africa. South-South cooperation and knowledge exchanges in agriculture between Brazilian and African organization is being catalyzed and supported through the Development Grant Facility (DGF), DIFID, IFAD and the Bill and Malinda Gates Foundation. 10.2 Knowledge and capacity building (MEDIUM) Original response  Agree Management agrees with this recommendation and notes that the number of Bank analytical products declined from a peak of 183 in 2000 to 131 by 2008, which was a period when lending more than doubled. As the number of agricultural sector staff has remained relatively constant, more time spent on project preparation and supervision to meet rising demand and improve portfolio quality has reduced the resources available for analytical work. This trend has actually worsened since the end of the IEG review period and requires further attention. The issue will be addressed in part through the staff review highlighted in response to item 10.3. 12 P114264, FY12 13 P11962, FY11 14 P124018, FY13 15 P125728, FY13 106  Ongoing/Agree Management agrees that this is clearly an area where it is possible to do better, while noting that considerable progress has been made. Bank management has introduced core implementation status and result (ISR) indicators which serve to both standardize and improve indicator quality. Core ISR indicators for agricultural research and extension, irrigation and drainage, and land administration have also been prepared and will soon be introduced. Furthermore, baseline data are now required to be included in the first ISR and, finally, economic analysis courses are being offered regularly. As for IFC, the tracking of financial, economic and environmental outcomes has progressively been built into the appraisal of new investments and monitoring of the portfolio since IFC deployed its Development Outcome Tracking System (DOTS)9. For new investments, coverage of IFC‘s standardized agribusiness indicators is at 100 percent. Monitoring and evaluation of development impacts is an area of priority for IFC, and indicators will continue to be reviewed and improved continuously for their relevance. In particular, agribusiness indicators will be adjusted as part of IFC‘s forthcoming implementation of its Corporate Development Goals.  Agree Management shares the concerns about the proportion of Bank ―subsector technical specialists‖ relative to ―broader agricultural sector generalists‖ within the Bank‘s agriculture and rural development sector family. One factor driving this shift has been the increase in decentralizing staff to country offices where the need for ―generalists‖ is often considered higher (to engage on multiple sector topics with government and other in-country development partners). The trend toward decentralization has increased since the end of the IEG review period, and will remain a challenge unless overall staff numbers increase and/or changes are made in terms of how staff are recruited and managed. This balance between generalists and technical staff in a decentralized organization is being addressed by the Board paper ―New World, New World Bank Group: (II) The Internal Reform Agenda; March 22, 2010.‖ To mitigate these challenges in the shorter term, the Regions and the anchor will: (a) conduct a Bank-wide ARD strategic staffing exercise to determine more precisely staffing gaps, especially in the context of current 3-5-7 senior staff rotation exercise, and; (b) consider targeted cluster recruitment, based on lessons learned from the current Social Development pilot cluster recruitment exercise. MAR 2012 Bank AAA advisory services. The annual number of related Bank analytical products has increased since the end of the IEG review period, from 131 in FY08, to 139 in FY10 and 145 in FY11. Management continues to note that, as the number of agricultural staff has remained relatively constant, more time spent on project preparation and supervision to meet rising demand and maintain portfolio quality has reduced the resources available for analytical work. There is ongoing analysis of the Bank‘s AAA on agriculture. M&E frameworks and cost benefit analysis. Core ISR indicators, which serve to both standardize and improve indicator quality, will be released for two sub-sectors in March 2012 – for agriculture research and extension, and for irrigation and drainage. The sector 107 participates in the Bank‘s Development Impact Evaluation Initiative (DIME) program on impact evaluations for Agriculture Adaptations (AADPT). Countries participating in AADPT include, Afghanistan, Angola, Azerbaijan, Bangladesh, Benin, Brazil, Burkina Faso, Cambodia, CAR, DR Congo, Ethiopia, Haiti, India, Malawi, Mongolia, Mozambique, Niger, Nigeria, Pakistan, Rwanda, Senegal, Sri Lanka, Tanzania and Zambia. The Bank offered a well-attended and highly rated learning event, ―Introduction to Quantitative Risk Analysis‖ of agricultural projects in FY11. The course addressed the risk from uncertainty in project variables, as part of cost benefit analysis of project interventions. IFC RESPONDING SEPERATELY ON M&E Human resource base and skill gaps. Management continues to share the concerns about the proportion of Bank ―subsector technical specialists‖ relative to ―broader agricultural sector generalists‖ within the Bank‘s agriculture and rural development sector family. One factor driving this shift continues to be the increase in decentralizing staff to country offices where the need for ―generalists‖ is often considered higher (to engage on multiple sector topics with government and other in-country development partners). The trend toward decentralization has continued over the review period, and will remain a challenge unless overall staff numbers increase and/or changes are made in terms of how staff are recruited and managed. To mitigate these challenges in the shorter term, the Regions and the Anchor are: (a) conducting a Bank-wide ARD strategic staffing exercise to determine more precisely staffing gaps, especially in the context of current 3-5-7 senior staff rotation exercise, and; (b) undertaking targeted cluster recruitment, based on lessons learned from the current Social Development pilot cluster recruitment exercise. IFC RESPONDING SEPERATELY ON HR ISSUES 10.3 Efficiency and sustainability (HIGH) Original Response  Ongoing/Agree Management agrees with the need for medium-term expenditure planning and sustainable financing. In order to assist governments do so, the Bank is currently finalizing an agriculture public expenditure toolkit10 to help better guide medium-term expenditure planning.  Ongoing/Agree Management fully supports this recommendation. For example, the Bank has recently undertaken analytical work on ―Improving Water Management in Rainfed Agriculture,‖11 and carried out a portfolio review of rainfed agriculture projects. This ongoing stock taking has focused on gathering lessons learned from past experience to better inform future lending on rainfed agriculture. In addition, several analytical studies examining the impact of climate change on agriculture, including rainfed agriculture, are being undertaken by the Regions.  Ongoing/Agree Although the integration of gender equality into agriculture and rural development projects was already higher than in other sectors, and has improved since the period covered by the IEG report, management fully agrees that more needs to be done. The percentage of rural projects with ―gender responsive design‖ (as defined by the PREM Gender Department) in the 108 Africa Region, which already met the 50 percent target in fiscal 2008, increased even further— from 59 percent in 2008 to 65 percent in 2009. In April 2010 the ARD Sector Board set a target of 75 percent for gender responsive design in agriculture and rural development projects in all Regions by fiscal 2014. After the period covered by the IEG review, the Bank compiled a good practice sourcebook with FAO and IFAD, the ―Gender in Agriculture Sourcebook,‖12 and completed a study on access to extension services in India, Ghana, and Ethiopia13 (World Bank & IFPRI 2010).These efforts complement the Gender Action Plan, which specifically provides operational support to agriculture. Going forward it is envisioned that these actions will significantly improve mainstreaming of gender into agriculture operations. Gender is mainstreamed across sectors and industries through explicit requirements in IFC‘s Sustainability Policy and Performance Standards (PS).14 The social assessment process required in PS1 provides guidance on disaggregating stakeholder groups by key social identities, including gender, and ensuring that any potential adverse impacts are addressed. Throughout the PS, there is explicit reference to addressing risks, ensuring opportunities, and providing appropriate consultation for women. As examples, PS2 addresses non- discrimination; PS4 addresses gender-disaggregated aspects of health and vulnerability; PS5 addresses women‘s tenure and livelihoods in cases of resettlement; and PS7 & and PS8 both include specific attention to women‘s views in decision-making processes. As part of its PS review and update, IFC proposes to strengthen client‘s requirements not to employ trafficked persons as part of a revised version of the Performance Standards (version 1). IFC has been tracking the implications of its investments on employment by gender through its DOTS system. It agrees with IEG that more can be done to mainstream and monitor gender concerns in agribusiness. This will be done as part of the next revision of the sector‘s standardized indicators. Management agrees that IFC policies did not contain a requirement to examine supply chains in the early years of the review period; this was only addressed in the IFC Performance Standards (PS) which became effective April 30, 2006. PS 1 requires that the impacts associated with supply chains be assessed in two cases: (1) where the resource utilized by the project is ecologically sensitive (e.g., wood products) or (2) in the case where low labor cost is a factor in the competitiveness of the item supplied (e.g., textile and some agribusiness activities); then child and forced labor should be examined for supply chains. IFC‘s approach to supply chains has been to focus client actions on the most immediate and serious risks in their supply chains—such as child labor, forced labor, and potential clearing of critical habitats. As part of its PS review and update, IFC proposes to: (i) strengthen its supply chain assessment methodology as part of project appraisal; (ii) make changes to the PS by adding significant occupational health and safety issues as a new risk factor to be considered in the supply chain assessment; and, (iii) to continue supporting certification schemes, through investments and advisory services, including engagement in a number of global commodity roundtables. The ongoing review and update of IFC‘s Sustainability Policy and Performance Standards will address climate change, supply chains, and biodiversity issues among others. IFC is generally supportive of international certification schemes and has an active engagement in a number of global commodity roundtables. In addition, IFC has been actively working with its clients (especially producers and traders) to increase the traceability and certification of their products in their respective supply chains. Efforts in the coffee, cocoa, and 109 cotton sectors, among others, are under way to develop corporate policies that emphasize purchasing of sustainable products, and compliance with national environmental legislation and IFC‘s PS requirements. MAR 2012 Medium-term expenditure planning and sustainable financing. A ―Practitioners‘ Toolkit For Agriculture Public Expenditure Analysis‖ was developed and finalized by the World Bank with support from the UK Department for International Development (DFID) as part of the partnership on ―Public Expenditures for Pro-Poor Agricultural Growth‖. The toolkit contributes to a broader effort to enhance the focus, quality, and appropriate scaling of public spending in the sector. More specifically it provides checklists for practitioners conducting various kinds of agriculture public expenditure analyses, and presents selected examples on aspects of the checklist to help guide analysis. Specific examples on cost recovery for different sectors were included to promote a more efficient use of limited resources and to promote practices that contribute revenues to sustain public expenditures. A related public-website was also developed as part of a process to improve the analysis of public expenditures in agriculture and to increase awareness of the process and political economy of public spending in the sector to enhancing understanding of the linkages between spending in agriculture and outcomes. In FY10 and FY11, agriculture public expenditure reviews were completed for Azerbaijan, Bolivia, Lebanon, Kazakhstan and Uganda, and is underway in for Nicaragua. The toolkit is being used to guide ongoing agriculture expenditure reviews in Latin America and Africa. In addition, a review and good practice guidance note on Forest Sector Public Expenditure Reviews was published in FY11 in collaboration with the Program on Forests (PROFOR.) The 2010 ESW ―Designing and Implementing Agricultural Innovation Funds: Lessons from Competitive Research and Matching Grant Projects‖ provides guidance on the use of competitive and matching grants for agriculture research on innovations. These instruments can promote partnerships, leverage significant resources and help develop a more efficient and pluralistic research system. These instruments are being used for example in the FY11 Ethiopia Agricultural Growth Program16 project and the FY11 West Africa Agricultural Productivity Program APL17. Water management and crop technologies in rainfed areas. As noted in the original management response, the Bank undertook analytical work on ―Improving Water Management in Rainfed Agriculture‖ and carried out a portfolio review of rainfed agriculture projects. This ongoing stock-taking has focused on gathering lessons learned from past experience to better inform future lending on rainfed agriculture. The World Bank Institute has recently established a three year program to support "climate- smart agriculture", of which sustainable management of agriculture water and land-water management are an integral component. The program will work through knowledge instruments such as south-south knowledge sharing, structured learning with 3-4 days in short courses, and coalition building on the subject areas. The South-South learning topics and 16 P113032 17 P117148 110 learning modules will cover both irrigated and rainfed agriculture. Gender concerns are adequately mainstreamed and monitored. Management continues to emphasize this issue. The percentage of agricultural projects with ―gender responsive design‖ (as defined by the PREM Gender Department) in the Africa Region, met the 50 percent target in FY08 and in April 2010 the ARD Sector Board set a target of 75 percent for gender responsive design in agriculture and rural development projects in all regions by FY14. From a baseline of 46 percent over FY06-08, satisfactory gender responsive design of agriculture and rural development projects has increased to 72 percent over FY10-11. Since the last response, the Bank has focused on operationalizating the ―Gender in Agriculture Sourcebook‖ which has proven to be a very effective tool used by Bank staff and development practitioners globally. A Sr. Gender Specialist has been recruited by the Agriculture and Rural Development anchor to support the mainstreaming process, and a ―gender in agriculture‖ focal person system has been instituted comprising Bank staff from the regions and staff from the IFC. Some large investment projects (for example in Azerbaijan18, Ethiopia19 and Zambia20) have been strategically selected and support has been provided to mainstream gender in the project design, implementation and M&E. In addition the Bank has developed more tools to assist in the gender mainstreaming process. For instance a ―Gender in Agriculture‖ e-platform was launched in early FY12 -- the site has approximately 150 visits weekly from all over the world. In addition, preparation of a 16-17 module e-learning course based on the Sourcebook has been initiated. IFC RESPONDING SEPERATELY ON GENDER ISSUES IFC Performance Standards to material biodiversity and other environmental and social aspects. IFC RESPONDING SEPERATELY IEG RESPONSE 2012 10.1 Synergies and complementarities (Medium) • Set up a knowledge network that links agriculture and agribusiness supply-chain specialists across the World Bank Group to strengthen communication and collaboration among sector departments within the Bank and IFC, as well as across the World Bank Group. Management provides a number of examples of collaboration between IFC and World Bank staff in the preparation of specific projects and through input provided in the development of specific programs and strategies. The spirit of the IEG recommendation, however, was to establish a mechanism to enhance communication and collaboration across the WBG in a systematic rather than ad hoc manner. No evidence is provided on this front. Adoption of this sub-recommendation is rated medium. • Work with partners to ensure that CGIAR research and other global and regional efforts are 18 P107617 19 P113032 20 P070063 111 translated into benefits in client countries, and facilitate partnerships among countries to encourage south-south knowledge exchange. Efforts have been made to reform the CGIAR with the aim of improving the relevance and effectiveness of its research. Notably the formulation of larger CGIAR Research Programs to reduce the fragmentation of past research effort and the promotion of partnerships to draw on comparative advantage in research. The Bank is also supporting regional research programs and has promoted knowledge exchanges between Brazilian and African organizations. However, it isn‘t clear to what extent these efforts have led to greater use of research results on the ground to provide benefits in client countries.. Adoption of this sub-recommendation is rated medium. 10.2 Knowledge and capacity building (Negligible) • Ensure sufficient quantity and quality of Bank AAA and IFC advisory services in agriculture-based economies, link them closely to lending, and use them to build counterpart commitment and to address constraints along the production chain. Management notes a slight increase in the number of analytic products produced in the years following the IEG evaluation but it is not clear what the trend is for AAA in agriculture-based economies. No information is provided on the quality of Bank AAA. IEG shares management‘s concern that resources available for AAA have declined as the demand for lending has increased, while the number of agriculture staff has remained constant. There is ongoing analysis of the Bank‘s AAA on agriculture but it is too soon to assess action from this exercise. No evidence is provided to show that AAA products are being utilized to build counterpart commitment or address constraints along the production chain. Adoption of this sub-recommendation is rated negligible. • Establish mechanisms to confirm ex ante if project M&E frameworks are adequate, with clear, relevant, and realistic objectives; thorough cost-benefit analysis; appropriate indicators; and adequate baseline data. Progress towards adoption of this recommendation has been limited to the release of core indicators for two subsectors in an effort to standardize and improve indicator quality, and a learning event offered in FY11, entitled ―Introduction to Quantitative Risk Analysis‖, which addressed risk from uncertainty in project variables in cost benefit analysis of agriculture projects. The sector also participated in the Bank‘s Development Impact Evaluation Initiative program on impact evaluations for Agriculture Adaptations. While these efforts may contribute to enhancing individual segments of an M&E framework, there is still a long way to go towards the establishment of mechanisms that confirm that overall project M&E frameworks are adequate. No evidence has been provided that M&E frameworks increasingly have relevant and realistic objectives, that cost-benefit analysis has become more thorough, or that and increasing share of projects have adequate baseline data. Adoption of this sub- recommendation is rated negligible. • Review the human resource base and skill gaps (also in light of the increased lending) and develop and implement a strategic plan to enhance the technical and policy skills of Bank and IFC staff working in the agriculture sector, particularly in agriculture-based economies. 112 Management notes that the trend toward decentralization has continued over the review period and this has been driving a shift towards a greater proportion of agricultural sector ―generalists‖ relative to subsector technical specialists. The trend is not expected to change unless overall staff numbers increase or changes are made in recruitment and HR management. Two efforts are ongoing to mitigate staffing challenges in the short term: (1) a Bankwide ARD strategic staffing exercise to determine staffing gaps in the context of the 3-5-7 senior staff rotation, and (2) implementation of targeted cluster recruitment, based on the experience of the Social Development pilot cluster recruitment exercise. Other than through recruitment, however, no evidence has been provided of a plan to enhance the technical and policy skills of Bank staff working in the agriculture sector, particularly in agriculture-based economies. Adoption of this sub-recommendation is rated medium. 10.3 Efficiency and sustainability (Substantial) • Increase World Bank Group support to medium-term expenditure planning to help ensure the adequacy of funding for operations and maintenance, and work with clients to ensure sustainable financing—including cost recovery where appropriate—for irrigation, transport, and research and extension services. Progress has been made to enhance public expenditure planning in the sector. Notably, a practitioners‘ toolkit for Agriculture Public Expenditure Analysis has been developed and is being used to guide ongoing public expenditure reviews in the LCR and AFR regions. In FY10 and FY11, agriculture public expenditure reviews were completed for 5 countries. A public website was also launched as part of a process to improve the analysis of public expenditures in agriculture and raise awareness of the linkages between spending in agriculture and outcomes. Less evidence is provided on efforts to work with clients to ensure sustainable financing. In FY10 an ESW was completed on ―Designing and Implementing Agricultural Innovation Funds‖ which provides guidance on the use of competitive and matching grants for research on agriculture innovations. Competitive and matching fund grants are being used in some projects in the Africa Region. However, no evidence is provided of effective efforts to ensure sustainable financing (including cost recovery) for irrigation or transport. Adoption of this sub-recommendation is rated medium. • Take stock of experience in water management and crop technologies in rain-fed areas to inform future World Bank Group support. Two stock-taking excises have been carried out to gather lessons on rain-fed agriculture: an analytical work on ―Improving Water Management in Rain-fed Agriculture‖ and a portfolio review of rain-fed agriculture projects. An additional effort to enhance knowledge of rain-fed agriculture is underway through WBI‘s three year program on climate smart agriculture. This program includes sustainable management of agriculture water and land-water management components as well as South—South learning modules on rain-fed agriculture. However, it is not clear how this information will inform future lending. Adoption of this sub- recommendation is rated substantial. • Ensure that gender concerns are adequately mainstreamed and monitored in World Bank and IFC agriculture operations. 113 Several steps have been taken to operationalize the Gender in Agriculture Sourcebook. The anchor has hired a gender specialist to support the gender mainstreaming process and a gender in agriculture focal person system has been developed comprising Bank and IFC staff. New tools to assist with gender mainstreaming have also been developed: a gender in agriculture e –platform and an e-learning course based on the gender sourcebook. The Bank has also made progress in monitoring agriculture projects with respect to ―gender responsive design‖ (as defined by the PREM Gender department). In FY08 the AFR region met its target of 50 percent. In April 2010 the ARD set a target for 75 percent of all ARD projects Bankwide to have a gender responsive design. Management reports that the percent of ARD projects with a ―gender responsive design‖ has increased to 72 percent in the period FY10-11, from a baseline of 46 percent in the period FY06-08. The IEG evaluation noted that ―greater attention has been paid to gender issues during the design of projects than in their implementation, and both need to be stepped up.‖ IEG acknowledges the positive steps taken with respect to gender design but progress on implementation is less clear. Adoption of this sub- recommendation is rated substantial. 114 11. Social Safety Nets - An Evaluation of World Bank Support, 2000–2010 LIST OF RECOMMENDATIONS 11.1 Engage during stable times to build SSNs that can help countries respond effectively to shocks. This requires steady country dialogue and support for developing SSNs, whether by lending, country-specific AAA, or engagement in global knowledge and learning. As such shocks are transitory in nature, an important characteristic of an SSN is its ability to expand and contract to reach different population groups as needed. Access to reliable poverty data, crisis monitoring systems, and flexible targeting systems are elements to develop appropriate SSNs. 11.2 Support the development of SSN institutions and systems. The Bank can further accelerate institution building, particularly in LICs, where capacity constraints are often severe and the building blocks for SSN administrative systems may need to be built from scratch. In MICs, the approach will require continuing the effort to harmonize programs within the broader social protection framework. 11.3 Increase SSN engagement in LICs. The Bank needs to maintain special efforts (financing and internal incentives) for LICs that permit countries to develop SSNs that will protect their poorest and prepare for shocks. Depending on the country context, these may include improving country capacity, adapting SSN programs to the institutional environment, improving poverty data and analysis to identify the particularly vulnerable groups, and assuring donor coordination for SSNs (for financing and technical assistance) to increase efficiency of government programs. 11.4 Improve the results frameworks of Bank supported SSN projects to (1) more clearly identify and address the needs of specific groups of poor or vulnerable, and (2) identify how project objectives fit within longer-term objectives for development of country SSNs. This involves improvements in the quality of objectives, design, and monitoring within projects, as well as development of a longer-term results framework for building effective SSN programs and systems 11.5 Clearly define objectives and assess benefits, costs, and feasibility of policy alternatives to ensure the most appropriate use of SSNs. Comparing alternative options for reaching the specific objective(s) is particularly important in contexts of high poverty and tight budgets and encourages specific SSN objectives. 11.6 Improve internal coordination of SSNs. It will help to review budget systems to see if the incentives they create for managers and task team leaders constrain cooperation between Bank units. Expertise across sectors and networks needs to be shared to enhance Bank support. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG 11.1 Engage during stable times to build SSNs that can Substantial Substantial help countries respond effectively to shocks. 11.2 Support the development of SSN institutions and Substantial Medium systems. 11.3 Increase SSN engagement in LICs. Substantial Medium 11.4 Improve the results frameworks of Bank supported Medium Medium SSN projects 11.5 Clearly define objectives and assess benefits, costs, and feasibility of policy alternatives to ensure the Medium Medium most appropriate use of SSNs. 11.6 Improve internal coordination of SSNs. High Medium MANAGEMENT RESPONSE 2012 11.1 Original Response: Management agreed with the recommendation: The Bank‘s client countries need broad-based and flexible SSN systems that address chronic poverty and its consequences but can also be scaled up to protect the poor from economic shocks—whether at global, country or household level. During the 2008-09 crisis, increased demand from clients and the extraordinary amount of resources deployed by the World Bank have helped to build a strong pipeline of activities that will persist into the immediate future, including in many countries that did not have prior SSN dialogue. The Bank will continue to provide client- driven assistance to respond to crisis situations, and to help countries build systems that alleviate chronic poverty, through its full menu of products—investment and policy lending (including new instruments, such as P4R), grants, and analytical activities. A key element of this would be capacity building and knowledge exchanges. These all will be financed through normal Bank budget allocations and supplemental seed funding from donors. It will also launch new activities focused on helping countries build greater capacity to respond to future crisis. 2012:  The proposed 2012-2022 Social Protection and Labor strategy emphasizes continuing the move towards building country SP (including SSN) scalable and flexible systems to address both shocks and chronic poverty. Extensive consultations on the new strategy continued through-out 2011 and into 2012. The new SP&L strategy (including results framework) has been endorsed by CODE on March 14 2012. 116  The Bank has been using demand-driven grant financing from the first round Rapid Social Response (RSR 1) multi-donor TF. RSR provides catalytic resources in relatively small amounts, with grants ranging from US$40,000 to US$3 million, half going to sub-Saharan Africa. This level of relatively small funding can effectively support system building efforts, which lead either to increased capacity to borrow for larger projects or to direct domestic resources to establish the ―building blocks‖ of safety nets administration. The progress in RSR was rapid. In May 2011, the RSR approved 17 activities for a total amount of $10.44 million (of which 23% in AFR) in 13 countries together with regional and global activities. In October 2011, the RSR approved 17 activities under the final round of expression of interest, for a total amount of $6.7 million (of which 19% in AFR) in 10 countries. As of March 2012 the number of countries supported by RSR reached 44 and its full resources have been committed. RSR grants have also laid the foundations for larger country investments in safety nets. In two years, just over $23 million in RSR grants have been associated with IDA support for safety nets totaling over US$1 billion. During the year new countries that never had the dialogue on safety nets started activities with RSR support. To meet unfulfilled country demand, the Bank will also seek additional donor support for a second round of RSR (RSR 2). Donor consultations for second round of RSR (FY12-13) are on-going.  The Bank continued to build capacity to enhance crisis resilience of client countries (LICs and MICs) through a South-South learning event in Ethiopia in May 2011 (between MICs and LICs clients), and through offering the Core Social Safety Nets training program targeted to the representatives of client countries and staff of donor agencies in December 2012 (attended by 90 participants). Following the training program, the network launched a Community of Practice, linking practitioners in different client countries with periodic videoconferences and on-line exchanges on key aspects of building blocks in SSN. In addition, the Bank, jointly with FAO, has launched a new distance learning course for practionners on Social Safety Nets. Another South learning event is planned for the second half of 2012.  Several briefings on crisis preparedness led to discussions with senior management on building safety nets to improve future crisis resilience- an effort which culminated in the preparation of the Development Committee paper on Safety Nets : ―Safety Nets Work: During Crisis and Prosperity‖ for the Spring meetings (March 2012). Management has committed to continued managerial support to assist all countries to build safety nets in a way that is appropriate and affordable. 11.2 Original Response: Management agreed with the central role of the systems approach for safety nets and for social protection in general, as reflected by its support for the shift in project designs noted by IEG. The Bank will continue supporting the systems approach in both LICs and MICs, adapted to country needs and capacity levels. 2012  The 2012-2022 Social Protection and Labor strategy puts systems as a central theme for the SP&L practice in World Bank in its engagements with clients and partners. The strategy was adopted by CODE with strong support by all CODE members on March 14, 2012. The strategy emphasizes deepening the Bank‘s knowledge and practice for building effective SP&L systems in different country contexts and capacity levels. 117  The World Bank has developed new CPIA measures, with guidelines for benchmarking SP, including a new measure for SP systems. This has become part of the regular process of monitoring progress towards system building. The process of revisions of the new CPIA with the new guidelines and benchmarking tools is close to completion (March 2012).  The Bank is drawing lessons on developing the building blocks for SSNs in LICs through its current activities (supported by RSR and other crisis-related instruments) focused on building administrative systems, institutions and delivery capacity for SSNs. These lessons are being disseminated internally and externally through the RSR monthly report and the social safety nets quarterly report (latest March 2012). The systems agenda of safety nets is featured prominently at the 2012 Spring Meeting of the WBG, with a number of events around dissemination of good practices.  The Bank is seeking donor funding to develop new knowledge tools and data products to assess performance of SSN/SP&L systems through a global inventory of social programs, their rules and their interactions in delivering benefits at the household level. A global inventory of SSNs is being developed and existing SP&L systems mapped with the new ―SP Atlas‖ which was launched in November 2011. The SP Atlas, also called ASPIRE, was revamped and launched with w serious media efforts in Spring meetings 2012. The data on systems from ASPIRE has become part of the World Bank indicators. 11.3 Original Response: Management agreed with the importance of continuing focusing efforts on improving capacity of LICs to deliver SSNs for the poor and also to protect access to basic services by the poor through all available instruments: lending, grants, AAA, training and knowledge-sharing activities. 2012  The proposed 2012-2022 Social Protection and Labor strategy identifies increased engagement in LICs as the key action to increase coverage of the poor by social protection. The strategy emphasizes deepening the Bank‘s knowledge and practice for supporting work in LICs, adapted to different contexts and capacity levels. The new SP&L strategy was approved by CODE with strong support by all CODE members on March, 14 2012 and will be launched during Spring meetings (April 2012).  To implement this strategic direction, the Bank is continuing to enhance capacity building and knowledge sharing activities for operational staff of the Bank and client countries focused on LICs. A South learning event in Ethiopia in May 2011 focused on building resilient safety nets. As part of the follow-up to this event, the Bank is supporting the on- going implementation or the creation of regional safety net communities of practice (the communities were launched in ECA, Africa, and South Asia).  The Bank is seeking additional donor resources to supplement BB funding for SSNs capacity building in LICs. Donor consultations for a second round of the RSR , which is focused on LICs, have been on-going  The Bank is continuing its yearly coordination meetings on social protection together with the ILO, UN agencies and all key bilateral and multilateral donors, as well as actively 118 participating in country-specific donor coordination meetings. The consultations with ILO, Trade Unions and IMF in March 2012 helped to prepare for the next meeting. 11.4 Original Response: Management partially agreed with this recommendation. Management agrees with and fully appreciates the need for improving results frameworks and of identifying project impacts on beneficiaries, as was borne out by our 2010 internal HDNSP results-readiness review. However, as long-term results and outcomes go beyond the scope of individual projects, management will focus on strengthening the application and consistency of medium term indicators in the results chain to monitor performance of projects. The HD network will also promote strengthened and consistent quality across the Bank through tools, analysis of outputs, impact evaluation and support to task team in results. 2012  The Social Protection and Labor Sector Board has introduced the use of core sector indicators, as appropriate, for all new investment projects. Training in the use of the new indicators is on-going.  The HDNSP anchor has prepared technical guidance notes on results readiness in safety net operations and a technical note on result- based lending. It has also developed TORs for assessing results readiness in projects and put in place a consultant roster that includes technical specialists in this area.  HDNSP has supported operational staff by organizing multiple quality enhancement reviews that, among other activities, help task teams develop improved results frameworks.  A major conference on impact evaluation of CCT took place in November 2011 (sponsored by the office of the Chief economist of HDN) and was widely attended by development practioners, staff from regions. Over 20 evaluations, majority linked to the Bank‘s projects, have been presented. 11.5 Original Response: Management only partially agreed with this. In its proposed actions, Management follows IEGs‘ suggested focus on the formulation lending operations, but with the understanding that at that stage of decision-making the range of options under consideration by government is rather narrow, essentially focusing on alternative features of a given type of program or more commonly, further development of an existing program. In the tools to improve consideration at this stage, the Bank will focus its efforts on the core objectives of poverty reduction through cash or in-kind transfers and building effective institutions. 2012  The issue of effectiveness of SP was a key theme of one Background paper for the strategy: Productive role of Social Protection. The paper is published and is being disseminated (Marc h 2012).  The SP ADEPT module reports on targeting effectiveness of SSN transfers. It is being applied across countries to produce summary outcome indicators made available in SP 119 Atlas. The manual for users has been finalized and will be published and disseminated by the end of FY12. The new version 5 is about to be launched (March 2012)  The SP Atlas (ASPIRE) was launched on a pilot basis in November 2011 and officially at the Spring meetings. It contains information to benchmark results in SSN across 50+ countries and regions. SP&L anchor staff are working on analytical notes on the results,to be posted on the dedicated web site FY12-13.  A workshop on assessing cost-effectiveness took place in February 2012 for Bank economists to improve project team capacity to carry out the necessary analysis. 11.6 Original Response: Management agreed with this recommendation. Bank management is committed to continued strengthening of the SSN community of practice and its inter-sectoral linkages 2012  On-going review by Social Protection Sector Board of thematic coding system for new social protection lending and AAA activities, which will improve capacity to monitor portfolio and cross-sectoral linkages in operations with Social Protection components  Maintaining cross-Sector Board membership and ongoing coordination and dialogue between HDN Social Protection and Labor and PREM Poverty and Equity.  Continuing consultation and collaboration in topical and key knowledge products by Social Protection and Labor, Education, HNP, PREM poverty, ARD, SDN as topical, e.g. Social Protection and Labor staff are involved in on-going products managed by other networks including:  Fuel subsidy reform and the role of safety nets—with the Energy anchor, SDN.  Climate change adaptation and safety nets—with SDN.  Nutrition and SP/scaling up nutrition—with HNP, ARD and PREM.  Food price monitoring—with PREM poverty Labor market work on informality—with PREM poverty and FPD. IEG RESPONSE 2012 IEG‘s evaluation of the Bank‘s support for SSNs spanned the decade of FY 01-10. IEG notes that Management conducted its own self-evaluation of this period and reviewed IEG results in preparation for its Social Protection Strategy 2012-2022. This strategy now addresses, to varying degrees, all of the issues raised in IEG‘s evaluation and reflects the new wave of effort undertaken in the past few years by the Bank and client countries. While the strategy does focus on the right issues (according to IEG‘s independent evaluation), progress over the coming years will assessed in terms of implementation and results. In addition, some aspects of the recommendations have not been addressed in the management response. 11.1 Engage during stable times to build SSNs that can help countries respond effectively to shocks. IEG recognizes the strong focus on system development including systems that can be flexible to respond to shocks; the use of RSR for financing demand driven 120 support to enable countries to focus more on crisis preparedness and its effort to mobilize more donor support for these efforts; and ongoing learning events to improve capacity and crisis resilience in countries. IEG‘s SSN evaluation pointed out that in countries with very little Bank engagement in SSNs, opportunities to improve country capacity and stimulate demand for SSNs is weak. The RSR has served an extremely important role breaking this cycle by providing grant support to countries that otherwise would lack county demand. Experience with the RSR has indicated that when countries are provided this support, they become more interested in further developing the capacity to respond to shocks. The steady engagement envisioned would require not only RSR to provide demand driven support, but greater Bank country support (via BB) for involvement on SSN development. This would enable a steady effort to build capacity that could then be relied upon in times of crisis. While the SP Sector has done an excellent job addressing crisis preparedness through a systems approach, a higher ranking for this recommendation would require greater buy-in from county directors as demonstrated by medium term programs of support for SSN engagement. IEG rates adoption of this recommendation as substantial. 11.2 Support the development of SSN institutions and systems. The development of SSN institutions and systems is central to the new strategy. IEG acknowledges Management‘s efforts to deepen knowledge and practice for developing SP&L systems in different country contexts and to disseminate the lessons from experience, both internally and externally. Establishing a new CPIA measure for SP systems is a concrete way of encouraging this approach. Increasing knowledge on what systems mean for LICs and MICs and disseminating this is important for advancing the agenda. Further improvements in this ranking would depend on evidence of results of these efforts in terms of accelerated institution-building in LICs and harmonization of SSN progress within the broader social protection network in MICs. IEG rates adoption of this recommendation as medium. 11.3 Increase SSN engagement in LICs. IEG recognizes management‘s greater focus on LICs as articulated in the new SP Strategy as well as its capacity building efforts tailored for LICs through south-south learning events. IEG also acknowledges the stated emphasis on donor coordination, through greater financing of the RSR (critical for kick-starting SSN development in LICS) and global coordination meetings of key development partners and particularly, through country specific donor coordination meetings. Given the array of donors and the disparate range of support to LICS, coordinating strategy, funding and activities is important in LICs with very constrained budgets and often inefficient use of donor funds. Improving the ranking in future years will require evidence that these extensive efforts have resulted in better donor coordination efforts (as illustrated in Ethiopia), better poverty data analysis to identify vulnerable groups to improve the efficiency of SSN programs, and continued engagement in new LICs. IEG rates adoption of this recommendation as medium. 11.4 Improve the results frameworks of Bank supported SSN projects. IEG notes the considerable emphasis that the Human Development network placed on improving results frameworks of social protection projects, including the use of core sector indicators, the preparation of technical notes on results readiness, quality enhancement reviews to help task teams prepare results frameworks. This is commendable, however, evidence that the results frameworks have in fact improved due to these efforts would be useful. IEG emphasized that 121 results frameworks need to be expanded beyond the duration of the specific projects when objectives are longer term, even if intended results requires the involvement of different actors. Accountability for this set of SSN recommendations is for the World Bank management, and not only for the SP sector board. IEG agrees with management rating of ―medium‖ (i.e. adopted in some operational and policy work but not to a significant degree in key areas) and recommendations working beyond the SP sector to measure progress towards intermediate development outcomes. IEG rates adoption of this recommendation as medium. 11.5 Clearly define objectives and assess benefits, costs, and feasibility of policy alternatives to ensure the most appropriate use of SSNs. IEG notes management‘s greater emphasis on improving targeting effectiveness, of disseminating information on SSN effectiveness in general, and of improving the capacity of project teams to conduct cost- effectiveness analysis, including as part of impact evaluations. Cost effectiveness analysis was recommended by IEG because it requires a clear definition of specific objectives and assessment of alternatives. IEG‘s analysis of the results frameworks indicated very general objectives related to poverty reduction (or even reaching the poor). IEG agrees with management that cost-effectiveness analysis can be undertaken to clarify project design; however, it also notes the importance of using this tool more upstream in the country team deliberations of its poverty reduction strategy and defining clearly how SSNs fit into that. Unfortunately, management provides no evidence that the cost-effectiveness of policy alternatives are actually being assessed, which is the thrust of the recommendation, IEG rates adoption of this recommendation as medium. 11.6 Improve internal coordination of SSNs. IEG recognizes the Social Protection sector board‘s efforts to improve coding of social protection activities to enable better monitoring and identify cross sectoral linkages. However, no evidence has been provided that the budget systems have been reviewed with respect to staff incentives or that coordination between Bank units has improved. IEG rates adoption of this recommendation as medium. 122 12. Trust Fund Support for Development - An Evaluation of the World Bank’s Trust Fund Portfolio LIST OF RECOMMENDATIONS 12.1 For trust funds other than Financial Intermediary Funds. In order to better align trust funds with recipient, donor, and Bank strategic priorities and improve their effectiveness, efficiency, and accountability for results, IEG recommends that the Bank adopt a three-pillar structure for trust funds, consisting of country-specific trust funds, partnership programs, and umbrella facilities.  Country-specific trust funds: The Bank should continue to accept trust funds created to support operations in a single country, because these funds have generally worked well in filling financing gaps and deploying donor funds in line with recipient priorities. They have allowed donors to target priority issues or countries, while at the same time helping mitigate limits of bilateral aid expertise and enhance aid coordination. The funds should be managed and accounted for using the same processes as those used for Bank budget or IDA/IBRD lending, and the relevant VPU should be accountable for their use and results in the context of country assistance strategies. If the existing trust fund portfolio were mapped to this proposed pillar, the pillar would account for nearly two-thirds of total IBRD/IDA trust fund disbursements.  Global and Regional Partnership Programs: For trust fund supported multiple- recipient country programs in which donors want to be actively involved in governance and implementation, a formally structured partnership can foster stakeholder voice, transparency in government operations, and accountability for results. When partners select this option, the Bank should continue to participate in them and require that each partnership program has a charter, a governing body, a management unit, and terms of reference to guide the Bank‘s participation. If the existing trust fund portfolio were mapped to this proposed pillar, the pillar would account for about one quarter of total IBRD/IDA trust fund disbursements.  Umbrella Facilities: The Bank and donors should phase out the other multiple recipient country funds (including those executed both by recipients and the Bank) and establish instead a small number of multi-donor, multi-recipient umbrella facilities to mobilize and deploy trust fund resources. This approach would help to solve the problems identified in the evaluation, namely operational inefficiency, inadequate accountability for results, and lack of objective and transparent allocation criteria. 12.2 For FIFs. In light of the distinctive nature of the Bank‘s role in relation to FIFs and the FIF portfolio‘s considerable size, heterogeneity, varied experience and risks, the Bank should strengthen its framework for guiding its acceptance and management of FIFs going forward. To do so, Bank management should:  Review experience to date on the development effectiveness of the funds and their synergy with the Bank‘s own operations and, based on that review, revisit Operational Policy 14.40‘s adequacy for guiding acceptance and management of FIFs as a distinct business line.  Seek Board approval for each proposed new FIF.  Report to the Board regularly on FIFs‘ delivery of intended results and the implications for the Bank‘s pursuit of its development mandate and strategies. The review of FIF‘s and any resulting revisions of the Bank‘s framework should be presented to the Board by the end of 2011. 12.3 Implications for aid architecture. Trust funds are helping to address bilateral aid limitations and fill operational gaps in traditional multilateral aid mechanisms, including IDA, notably by providing pooled grant financing for specific countries, targeted development issues, and provision of global public goods. They also serve to coordinate governmental and nongovernmental sources of aid, and support programs with new governance arrangements. But their potential added value, their aid effectiveness, and their coherence with other elements of the international aid architecture varies considerably across the many ways they are currently used. It would be useful, therefore, for the international aid community to reflect on the reasons for the gaps in the multilateral system that lead donors to use trust funds, and to assess the comparative advantages of the trust fund and other aid vehicles. Such reflection would help to identify opportunities for reforms in the multilateral aid architecture, including the World Bank, while allowing trust funds to specialize on situations where the multilateral institutions alone cannot be fully effective. The Bank should initiate such an assessment, as well as a discussion among shareholders to explore the comparative advantages of multilateral and trust fund aid modalities before the 2012 Annual Meetings. STATUS OF IMPLEMENTATION Level of Adoption Recommendations Mgmt IEG 12.1 For trust funds other than Financial Intermediary Funds. In order to better align trust funds with recipient, donor, and Bank strategic priorities and improve their effectiveness, efficiency, and Difference of Difference accountability for results, IEG recommends that the Opinion of Opinion Bank adopt a three-pillar structure for trust funds, consisting of country-specific trust funds, partnership programs, and umbrella facilities. 12.2 For FIFs. In light of the distinctive nature of the Bank‘s role in relation to FIFs and the FIF portfolio‘s considerable size, heterogeneity, varied experience and Substantial Medium risks, the Bank should strengthen its framework for guiding its acceptance and management of FIFs going forward. To do so 124 Level of Adoption Recommendations Mgmt IEG 12.3 Implications for aid architecture. Trust funds are helping to address bilateral aid limitations and fill operational gaps in traditional multilateral aid mechanisms, including IDA, notably by providing pooled grant financing for specific countries, targeted development issues, and provision of global public goods. They also serve to coordinate governmental and nongovernmental sources of aid, and support programs with new governance arrangements. But their potential added value, their aid effectiveness, and their coherence with other elements of the international aid architecture varies considerably across the many ways they are currently used. It would be useful, therefore, for the international aid Medium Medium community to reflect on the reasons for the gaps in the multilateral system that lead donors to use trust funds, and to assess the comparative advantages of the trust fund and other aid vehicles. Such reflection would help to identify opportunities for reforms in the multilateral aid architecture, including the World Bank, while allowing trust funds to specialize on situations where the multilateral institutions alone cannot be fully effective. The Bank should initiate such an assessment, as well as a discussion among shareholders to explore the comparative advantages of multilateral and trust fund aid modalities before the 2012 Annual Meetings. MANAGEMENT RESPONSE 2012 12.1 Original Response. Not Agreed. Management does not agree to this three-pillar structure. While Management is interested in better understanding IEG‘s perspective, it does not currently see the usefulness the proposed typology as presented. The three-pillar structure also does not take into account that trust funds are vehicles for channeling aid resources (i.e., they are a funding source) and not programs in and of themselves. 12.2 Original Response: Partially Agreed. Management agrees to prepare a strengthened framework for guiding the Bank‘s acceptance and management of FIFs and expects to present it to the Board in FY12. In preparing the framework, Management will review the Bank‘s experience to date in accepting, establishing and managing FIFs and it will revisit OP 14.40, including the definition of FIFs and the adequacy of OP14.40 for guiding the acceptance and management of FIFs. In conducting this review, Management will continue to treat FIFs as separate business line, where the Bank provides financial intermediation services to 125 shareholders and clients to support broader international development partnerships. In that context, FIFs for which the Bank‘s role is that of trustee only, cannot be expected to have full synergy with the Bank‘s own operations, as implied in the IEG recommendation. Management accepts the principle of Board approval for FIFs and will develop relevant criteria and procedures as part of the FIF framework referred to above. Management does not agree to report to the Board regularly on FIF results. It notes that assessing development effectiveness is the responsibility of the FIF governing bodies. As such, the Bank has no mandate for conducting reviews of the development effectiveness of FIFs to report to the Board on the overall delivery of FIF results. MAR 2012: Management is developing a new FIF framework that will establish criteria and procedures for Bank‘s engagement in FIFs, provide guidance for risk management, and improve Bank service delivery to FIFs. A concept note for the framework has been subject to Bank-wide review and consultations with external stakeholders, including donors and recipient countries. The FIF framework is at an advanced stage and Management expects to present it to the Board in late FY12. 12.3 Original Response. Partially Agreed. Management believes that it would not be appropriate for the Bank to conduct an assessment of the comparative advantage of multilateral and trust fund aid modalities. However, Management does agree that it is important to better understand the factors driving donors‘ decisions to use multilateral or trust fund modalities, including how they assess their respective comparative advantage. Through its work on IDA, trust funds and FIFs, the Bank regularly engages donors on these complex issues. In the context of IDA 16, the Bank has worked to make the case for multilateralism and in the case of trust funds and FIFs, it is working toward greater selectivity and clarity on the type of gaps these mechanisms are filling, in line with the principle of ―think twice‖ agreed in Accra. Furthermore, the Bank is actively involved in helping to shape the international agenda on aid effectiveness, including through active participation in the upcoming High Level Forum on Aid Effectiveness to be held in Busan this November. MAR 2012: As per the original response above, Management continues to believe it would not be appropriate for the Bank to conduct an assessment of the comparative advantage of multilateral and trust funded aid modalities. Instead, Management is engaging in discussions with donors on the role of IDA and of trust funds in various forums, i.e. in the context of joint donor trust fund portfolio reviews, IDA16‘s working group on financial sustainability. To inform these discussions, Management has deepened its analysis of the role of trust funds in the global aid architecture and in the World Bank. This work includes a recently completed CFP working paper on the evolving aid architecture which also served as a background document for a seminar on this topic held at the Busan HLF4 in December 2011. The paper identifies the growing trend toward earmarked ODA either through multi-bilateral aid (i.e. bilateral aid channeled through the multilateral system) or through new thematically-focused multilateral entities in the context of rising ODA in the past decade. The growth of the Bank trust fund portfolio reflects these trends and the preference of donors to use the Bank as an effective delivery platform. Further analysis reflected in recent Board documents, including the upcoming Medium Term Business and Finance Paper FY13-FY15, shows that for the most part trust funds complement the Bank‘s work in critical areas such as fragile states, disaster 126 response, global public goods, civil society outreach and innovation. Issues of fragmentation and selectivity are being addressed in the context of recently launched initiatives (i.e. umbrella facilities, FIF and partnerships frameworks). IEG RESPONSE 2012 Bank Management has not indicated their ―levels of adoption‖ in the accompanying table. 12.1 While Management did not agree to adopt a ―three-pillar‖ structure for trust funds, Management did agree with certain aspects of this recommendation, for example:  ―Management does think that country specific (and thematic) trust funds should be managed using the same processes as for the Bank budget and IDA/IBRD lending, in line with the Trust Fund Management Framework and as stated in OP/BP 14.40.  ―Management does think that clear governance arrangements are needed [for GRPPs] appropriate to the nature, scope, complexity and risks of the partnership program.  ―Management also thinks that Bank staff associated with GRPPs should operate under clear terms of reference and is working on guidance.‖  Umbrella facilities are ―in principle consistent with current thinking in some parts of the Bank . . . However, the practical application of this approach will still need to be determined.‖ Therefore, IEG would have expected a progress report on some of these aspects, particularly, the work that has been done on piloting umbrella facilities. 12.2 Management‘s actions in response to this recommendation are a work in progress. IEG reserves judgment on the outcome of this work until it is completed and starts to be implemented — hence the medium rating. IEG is pleased that the new FIF framework and the new Partnership Program Management Framework are being prepared in conjunction with each other, given that almost all FIFs are supporting GRPPs. But Management‘s timeframe for its presentation of the new FIF Framework to the Board seriously lags that in IEG‘s recommendation. 12.3 It is difficult for IEG to assess progress in this area in the absence of a specific, well- defined Bank initiative — hence the medium rating. IEG continues to recommend that an assessment be undertaken of the comparative advantages of multilateral and trust fund aid modalities, and that the Bank should play a leading role in initiating such an assessment given that the Bank is the largest multilateral aid organization administering the largest portfolio of trust funds. The proliferation of large thematically focused GRPPs supported by Bank-administered donor trust funds to help countries achieve specific MDGs drew much attention in Busan. Paragraph 23(b) of the Busan outcome document provides a particular opportunity for the Bank to initiate action in this area: 127 23 (b) ―We will improve the coherence of our policies on multilateral institutions, global funds and programmes. We will make effective use of existing multilateral channels, focusing on those that are performing well. We will work to reduce the proliferation of these channels and will, by the end of 2012, agree on principles and guidelines to guide our joint efforts. As they continue to implement their respective commitments on aid effectiveness, multilateral organisations, global funds and programmes will strengthen their participation in co�ordination and mutual accountability mechanisms at the country, regional and global levels.‖ 128 World Bank Implementation Report 2012 13. Public Sector Reform: What Works and Why? – An Evaluation of World Bank Support LIST OF RECOMMENDATIONS 13.1 Design PSR projects and allocate Bank resources to them with recognition that PSR has especially complex political and sequencing issues.  Be realistic about the time it takes to get significant results, understand the political context, identify prerequisites to achieve the objectives, and focus first on the basic reforms that a country needs in its initial situation.  Reconsider the balance between development policy and investment lending, because institutional change usually needs the sustained support of investment projects, although development policy lending can help secure the enabling policy changes. 13.2 Within country PSR strategies, set priorities for anticorruption efforts based on assessments of which types of corruption are most harmful to poverty reduction and growth.  Only when the country has both strong political will and an adequate judiciary system should the Bank put primary emphasis on support for anticorruption laws and commissions. Given that reducing corruption will be a long-term effort, the Bank should emphasize (i) building country systems that reduce the opportunities for corruption that is most costly to development and (ii) making information public in ways that stimulate popular demand for more efficient and less corrupt service delivery.  Provide operational clarification to the country team about how the Bank‘s anticorruption efforts fit within the overall country strategy. 13.3 Strengthen the CSA components of PSR, providing them with a better framework and indicator set, and give more attention to the budget execution phases of financial management.  This will require PEFA-like actionable indicators for civil service and administrative performance and more linkage between the implementation of reforms for civil service and for financial management. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 13.1 Design PSR projects and allocate Bank Incompl resources to them with recognition that PSR Substant Medium ete and has especially complex political and ial archived sequencing issues. 13.2 Within country PSR strategies, set priorities Incompl Substant for anticorruption efforts based on Medium ete and ial assessments of which types of corruption archived Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG are most harmful to poverty reduction and growth. 13.3 18.3 Strengthen the CSA components of PSR, providing them with a better Incompl framework and indicator set, and give more Medium Medium ete and attention to the budget execution phases of archived financial management. MANAGEMENT RESPONSE 2012 Original Response: 13.1 Ongoing/Agreed. Bank management agrees in principle with this recommendation, noting that it points to the importance of intensifying AAA upstream of PSR operations— which can have significant budget implications. How the recommendation can best be implemented will require learning by doing and will depend on country context. To implement the GAC strategy, the Bank‘s regional Vice Presidential units have identified 26 countries that currently are initiating country-specific country GAC strategy processes—including, in some of these countries, intensified governance assessments that aim to, among others, identify political obstacles to reform and feasible approaches to sequencing. At the conclusion of this learning process, Bank management is committed to reporting to the Board whether and how it intends to systematize and scale up its GAC work, including AAA. Reporting on the agreed actions will be done in the context of overall GAC reporting. 13.2 Mostly agreed. Management agrees with the recommendation that the most effective way in which PSR can support anticorruption efforts is by giving priority to work on country systems and on information flows to the public. As the recommendation suggests, the more complex challenge (which goes beyond the scope of PSR operational work) has to do with the relationship between country strategies and operations more broadly and anticorruption efforts. Management‘s response to this broader challenge has been laid out in the strategy, ―Strengthening World Bank Group Engagement on Governance and Anticorruption‖ (World Bank 2007c). Three ways in which GAC strategy implementation addresses this broader challenge are (i) by signaling that GAC is not only a PSR concern but ―is everybody‘s business‖; (ii) by intensifying efforts to manage fiduciary and other GAC risks in Bank operations; and (iii) by underscoring that approaches to addressing GAC are country specific and should be derived from poverty-reduction priorities. With regard to IEG‘s request for operational clarification, this last point implies that attention to GAC issues generally will be most intensive in those sectors that are given priority for poverty reduction in country strategies. The GAC implementation progress report to the Board, to be presented in 2008, will report on experience. 13.3 Ongoing/Agreed. Bank management agrees with the recommendation that a better framework is needed for the civil service and administrative components of PSR work. A strategic staffing exercise, being undertaken as part of GAC strategy implementation, will help implement this recommendation. The Poverty Reduction and Economic Management anchor already has begun recruiting to strengthen its staffing on civil service and administrative reform. Under the GAC strategy and implementation plan, intensified work is under way within that anchor to develop a new generation of ―actionable 131 indicators,‖ with indicators for civil service and administrative a top priority. However, as is evident from the seven-year experience of developing the PEFA indicators—cited as a success in the IEG evaluation—the development of new and better indicators is a challenging task that will take time. For the budget execution phases of financial management, Bank management notes that both the PEFA indicators and the CFAAs give them strong attention. An earlier, narrower focus on budget formulation has already has been incorporated in the Bank‘s operational work. Management will monitor and report on progress on these actions in reports to Executive Directors on the implementation of the GAC initiative. MAR 2012: Following the endorsement of the GAC Strategy in March 2007 management has taken a series of steps to mainstream the governance agenda in the Bank and ensure greater aid effectiveness. Public sector governance initiatives have been central in this effort. In particular, the Bank has focused on (i) leading by example and mainstreaming the GAC approach in Bank‘s operations; (ii) strengthening country system and building capacity at the local level, (iii) monitoring progress and results; and (iv) promoting global cooperation and knowledge sharing. There have been steady gains in both the development of GAC tools and their mainstreaming in the work of the WBG – from upstream CAS strategies to downstream design and implementation of lending operations and technical assistance initiatives. A central focus of the first two years of implementation of the WBG‘s Governance and Anticorruption Strategy implementation has been on enhancing development effectiveness and poverty reduction while managing risk (responding to the poverty focus of governance and anticorruption work recommended by IEG). From July 1, 2010, the Bank required all investment lending projects to incorporate the new Operational Risk Assessment Framework (ORAF) that systematically looks at stakeholder, implementation and overall risks and how these risks are directly linked to a project‘s development outcomes. Further, Management has undertaken a review of the entire PSG portfolio, including focused attention on 27 specific problem projects, implementation of (almost) quarterly reviews of the PSG portfolio, decentralization of some senior technical staff to country offices as ‗mini-hubs‘ to better implement PSG projects, and reviewed the skill mix of PSG staff. Management is currently preparing a Public Sector Management strategy as part of the overall GAC Phase II Strategy and as part of this work is completing analysis of PSR components of all Bank financed projects, both DPL and IL, to better assess the qualitative assessments of instrument mix to support PSG interventions. Finally, Management established a new Preventive Services Unit within the Department of Institutional Integrity that proactively works with Bank's teams to identify and address risks of corruption ex ante. At the country level, this approach has translated in promoting a deeper understanding of local challenges and incorporating into CASs and operations the governance, institutional and political drivers of decision making in specific country settings—recognizing the complex political sequencing and time frame issues raised by IEG . The Bank has increasingly focused on improving core country systems, including through strengthening country procurement systems and helping build country capacity and investing more systematically in public sector reform approaches tailored to different types of countries. Since the spring of 2009, the Governance Partnership Facility (GPF) has supported country-level governance programs in 18 countries. Innovative initiatives, such as building capacity to manage Ghana‘s oil resources by involving civil society, have been implemented through the GPF work. A GAC Portal was launched to all staff in mid-May that brings together GAC knowledge, learning and guidance from all the regions and networks and provides access to online GAC 132 Essentials and Ethics training courses that had deep participation levels. Further, the Bank has supported the integration of political economy analysis in Bank operations and programs to ensure greater impact and sustainability of the reform process. This has been especially relevant for the Bank‘s work in the areas of public financial management and anticorruption. In particular, with the support of the GPF, the Bank has made significant resources available to country, sector and selected anchor teams to undertake governance and political economy analyses. Currently, the Bank is maintaining an ongoing list of political economy analytic work that is being undertaken. The list includes about 50 pieces of such work across regions and sectors as of Q2 FY11. In the areas of Public Financial Management (PFM) activities and the reforms for civil service, the Bank is taking action to strengthen the empirical basis of its work, for instance, as reflected in the creation of Financial Management Information System (FMIS) Database.21 The FMIS database has supported a review of all completed and active FMIS projects since 1984. The report is expected to be publicly available in Q1 2011. The FMIS Report identifies project characteristics, scope and cost of systems, information and communication technology solutions, project preparation approaches, regional differences in the design and implementation of FMIS solutions, and importantly the operational status of the information systems, in addition to the duration, cost and performance ratings. Based on the findings of the report, 80% of the completed FMIS projects were extended and 82% were completed within budget and resulted in a sustainable and useful solution as a basis for further PFM reforms. Among 55 completed projects, 49 systems (89%) are fully or partially operational, which suggests that from the perspective of obtaining results and sustainability, many of these projects achieved their technical and operational targets. The Report also presents the trends in the design and implementation of FMIS solutions since 1984, and shares observed/reported achievements, challenges and lessons learned with interested parties to assist in improving the performance of related PFM reform programs, and where applicable, human resource management reforms. In fact, HRMIS solutions were visible as an additional component of FMISs in 17 out of 55 completed projects (14 of these HRMIS solutions were fully or partially operational). There seems to be a growing interest in the inclusion of HRMIS modules as a part of integrated FMIS solutions in new projects; this has implications for the ongoing advice and operational streams in the Bank. The Bank is also supporting the strengthening of institutions of accountability at the country level, focusing on access to information and transparency. As part of this work, the Bank has launched an initiative to build capacity and promote effectiveness of anti-corruption authorities (ACAs) in collaboration with UNODC, the US State Dept. and the European Commission. To this end, the Bank organized a Workshop with heads of ACAs in March 2010. The Workshop highlighted a set of factors that influence the effectiveness of ACAs. As a follow up the Bank is gathering detailed information from ACAs around the world on lessons learnt, and is creating a web portal to share the results of this effort. In addition, the Bank has launched in 2008 the Public Accountability Mechanisms (PAM) Initiative. This initiative is a part of the Actionable Governance Indicators (AGI) work described below and brings forward detailed and regularly updated data on countries‘ efforts to enhance transparency and the accountability of public officials (income & asset disclosure, conflict of interest, freedom of information/right to information, immunity protections, ethics training), and 21 The World Bank FMIS Database (1984-2010) includes 94 projects (55 completed + 32 active + 7 pipeline) from 51 countries as of August 2010. http://connectprem.worldbank.org/psg/pf/fmis 133 has been applied in a total of 87 countries. Measuring progress and impact of the governance and anti-corruption agenda on the ground has also been a critical component in the strategy moving forward. In October 2009, the Bank formally launched the Actionable Governance Indicators (AGI) Portal that includes data from 13 separate sources (including Global Integrity Index, Public Expenditure and Financial Accountability (PEFA), CPIA, Afrobarometer, Governance in Social Sectors, Open Budget Index, International Budget Practices and Procedures Survey, Polity IV Project, Press Freedom Index, Public Accountability Mechanisms, Statistical Capacity, Enterprise Surveys, and Doing Business). The data Portal, publicly accessible and regularly updated, provides data and tools on different areas of public sector governance reform. Over 200 PEFA Performance Measurement Framework assessments (led by a range of donor partners and countries) have been completed in over 110 countries and some 40 are completed annually. These assessments cover all phases of the budget process, from budget formulation, through execution and reporting. In addition, the Bank has been promoting the use of AGIs at the country level supporting the development of local initiatives, such as the ―Data Tracking Mechanism‖ in Uganda. Here, by using existing data, and at no direct cost to the government, the Bank in collaboration with other donors has supported the Government to identify a set of corruption performance data that could be used to systematically monitor progress. In this case, the Ugandan Government has taken ownership and is moving forward to monitor corruption performance on an ongoing basis. Thanks. In light of the IEG‘s highlighting the need for PEFA-like actionable indicators for Human Resource Management (HRM), the Bank has developed an instrument for capturing systematic evidence on the design, implementation and performance of HRM systems in client countries – the HRM AGI Instrument. The instrument has been applied in eleven pilot countries. Moreover, in an effort to establish an ―industry standard‖ for capturing such evidence, the Bank has met with and achieved agreement on the HRM system performance indicators from a number of donors active in supporting civil service reforms (OECD, European Commission, DFID). In addition, the Bank has developed or is now in the process of developing a number of sector-specific instruments for AGIs (Health, Education, and Forestry) which have been piloted in several countries. The challenge moving forward is to ensure widespread use of such instruments in our support for institutional and governance reforms addressing HRM issues in our client countries. In parallel to the AGI Data Portal development and launch, the Bank has continued to support the implementation of detailed Governance and Anti-corruption Diagnostic Surveys aimed at identifying priorities for anti-corruption efforts. Since 2007 the Bank has carried out country specific initiatives in several countries in Africa and MENA (Mauritania, Cameroon, Senegal, Cote d‘Ivoire, Yemen, Morocco, among the others). Finally, the Bank has focused on promoting global cooperation and knowledge sharing. The Bank has actively supported new initiatives that are emerging at the global level, including through partnerships among donor agencies, civil society groups, and governments for better development outcomes – such as the Extractive Industries Transparency Initiative (EITI), and the Construction Sector Transparency Initiative (CoST). A key global effort for the Bank has been the Stolen Assets Recovery (StAR) initiative, launched in partnership with the United Nations Office on Drugs and Crime. StAR which supports international efforts to deter illicit flows of the proceeds of corruption and facilitate asset recovery. In addition, the Bank has focused on promoting knowledge management 134 and sharing on these key issues. This has led to the creation of the new GAC web-portal, which includes good practice cases, a good practice framework, e-learning module and other useful information -- and the establishment of the Bank's Political Economy Community of Practice ('PE CoP'), which first convened in October 2009. IEG RESPONSE 2012 As it enters its fourth year of implementation, the Bank‘s Governance and Anticorruption strategy has leveraged considerable incremental budgetary and donor resources to support efforts to more systematically address GAC concerns in country programs and projects. IEG‘s 2011 GAC evaluation found that the Bank continues to support GAC initiatives in virtually all countries where it has operations, often through its support for public sector management, service delivery, and investment climate reforms. Over the 2008-10 period, the Bank expanded its commitments to support country institutions, increased its use of political economy analysis in projects, and aligned more with at least some country systems in Africa and fragile states. However, it is not yet systematic in following through with project-level institutional strengthening components, or in employing demand-side, enhanced fiduciary, and results measures in its projects across sectors. The Bank‘s risk management approach is also not yet consistent across countries or lending instruments, despite the launch of the ORAF, which applies to investment lending. In reviewing this particular MAR, the key issue is how and whether GAC and GAC-related efforts have enabled the Bank to address the mixed track-record of its public sector reform (PSR) portfolio, and in particular, respond to the specific IEG recommendations on PSR. Also, as noted last year, IEG has observed that the 2012 MAR has yet to distinguish between Bank-wide GAC efforts and those that have specific relevance to the PSR portfolio. Furthermore, the MAR is primarily concerned with inputs such as Network guidance, workshops, and other internal activities, rather than the Bank‘s main outputs – that is, well-designed and high performing PSR operations across countries. Yet, IEG observes that several operational and organizational issues continue to affect the Bank‘s PSR business in particular. For instance, The recent dissolution of the Africa public sector unit as part of a larger focus on GAC issues raises important questions for the Bank‘s PSR business overall. Will an emphasis on larger GAC issues be at the expense of a focused operational agenda around core public sector and public finance issues? These and other questions will need to be considered as the Bank prepares to launch its updated ―PSM approach.‖ The new approach aims to mainstream problem-driven and political economy-sensitive methodological approaches to public sector reform. That said, implementation of the new approach will require systematic application of PEA diagnostics, better instruments (including more extensive use of P4R), and the development of a pipeline that addresses concerns raised in the IEG PSR evaluation. Since it will launched in the context of GAC2, IEG should monitor the progress going forward with a view to assessing progress against relevant recommendations of the PSR and GAC evaluations. Comments on the specific IEG recommendations and management responses are provided below: 13.1 Compared with the 2011 MAR, Management has noted the considerable guidance and resources (primarily through the GPF) provided to support the use of political economy analysis in the design and implementation of public sector reform/governance operations. It also highlights efforts to 135 monitor PEA work. Absent from the 2012 MAR is any evidence of improvements in the use of PEA either in projects or in country dialogue. In this regard, Management could usefully cite IEG findings that use of PEA expanded systematically in projects – in particular those with Governance and Economic Policy (G/EPOL) themes -- approved during FY08-10 when compared with projects approved in FY04-07. IEG also rated operations with G/EPOL themes in FY08-10 more favorably in terms of their ―adaptation to informal/actual reality.‖ Progress in the use of PEA is less systematic in Bank economic and sector work, even reports relating to PSR issues. IEG has recommended that such analyses be mainstreamed and carefully – yet deliberately – disclosed to enrich public discourse in client countries. Notwithstanding these improvements, Management‘s emphasis on 18 countries that received GPF is too selective to make a Bank-wide impact (as envisaged in the GAC). Furthermore, the 2011 GAC evaluation found that CASs or projects in countries that received CGAC or GPF financing were not systematically more GAC-responsive. Also, IEG found that, in many instances, initiatives deemed to be ―innovative‖ by the GPF were similar to initiatives undertaken in earlier years. Finally, a 2010 IEG survey of operational staff indicates that the Bank‘s guidance and selective support efforts to roll-out a ―political economy approach‖ were not sufficiently tailored to the needs of Bank operational units. As in the MAR2011, Management‘s response does not address the second component of IEG‘s recommendation, i.e., striking the right balance between DPLs and investment lending in support for PSRs. IEG‘s GAC evaluation found that DPLs were associated with the achievement of core public sector outcomes – a relationship that is, in many cases, strengthened by the presence of complementary TA and investment lending support. By the same token, the evaluation found a negative correlation between the use of DPLs in a country portfolio and the achievement of various accountability and demand for good governance objectives. In conclusion, while some progress has been made on the use of PEA in projects, the Bank has yet to address the issue of the instrument mix to achieve PSR objectives more systematically. On recommendation 14.1, IEG rates the overall level of adoption as medium. 13.2 The bulk of the GAC efforts relating to anticorruption involve guidance on measures to reduce fraud and corruption risks in projects. While a considerable amount of guidance had been provided, the IEG GAC evaluation found that the use of enhanced GAC fiduciary measures – including preventive measures against fraud and corruption – had not expanded across projects and sectors in a systematic manner. Furthermore, the application of these fiduciary risk mitigation measures across country portfolios (for example, through anticorruption plans), were used in only about a fifth of country programs reviewed. As noted in IEG‘s comments on the 2011 MAR, the ACA initiative as well as the global efforts to prevent corruption (for instance, StAR, AML, EITI, COsT, etc.) bode well for the Bank‘s objectives if it can fit global initiatives to country needs. In its response, Management should provide more evidence of how these corporate initiatives are being adopted at the country level, and whether they are helping the Bank and its clients avoid the pitfalls of earlier approaches to strengthening ACAs (which was the spirit of IEG‘s recommendation). In fact, according to IEG‘s GAC evaluation, Bank country programs typically identified only a subset of accountability institutions in client countries (for instance, external audit and legislative oversight bodies in about 50 percent of countries reviewed). Efforts to strengthen judiciaries (40 per cent), the media (32 percent), right to information and open government (28 per cent), and offices of the ombudsman (0 per cent) were far less frequent before and after the GAC. Bank support in 136 these areas did not expand even in the period following the IEG PSR evaluation. An area that continues to be of concern is the lack of progress on the use of country systems (UCS). The MAR2011 seems to indicate that the Bank has systematically increased the use of core country systems. However, the Board‘s discussion of the Use of Country Systems pilot for procurement revealed a number of obstacles to progress. More important, IEG‘s portfolio-wide review indicated that the use of at least some country systems (PFM, procurement, and civil service systems) expanded only in countries with weaker institutions and those in Africa. In other words, systematic improvements in the alignment of Bank operations with country systems were not evident in other countries, particularly those with more capacity. The rating for level of adoption does not warrant change since MAR 2011 and should be medium. 13.3 In 2009, Management invested in strengthening the Bank‘s work on actionable governance indicators (AGIs), and in particular, indicators on HR systems. An attempt to apply the PEFA-system methodology to HR systems was made, along with some early country pilots. However, the Bank has not been able to sustain the same kind of institutional momentum that it did in the context of PEFA. The HD network is also developing sectoral AGIs, although systematic improvements in the use of AGIs in the portfolio were not evident in the recent IEG GAC review. In general, process indicators were more frequently used rather than AGIs or institutional quality or institutional outcome indicators. Progress on the second round of PEFA assessments is noteworthy as well as the FMIS data base, but there is no evidence of operational take-up of the specific budget execution or HRM-PFM linkages recommended in the Bank‘s evaluation IEG agrees with Management that recommendation 18.3 should be rated as medium for level of adoption. 137 14. Doing Business: An Independent Evaluation – Taking the Measure of the World Bank-IFC Doing Business Indicators LIST OF RECOMMENDATIONS 14.1 To improve the credibility and quality of the rankings, the DB team should: (a) Take a strategic approach to selecting and increasing the number of informants: – Establish and disclose selection criteria for informants. – Focus on the indicators with fewest informants and countries with the least reliable information. – Formalize the contributions of the supplemental informants by having them fill out the questionnaire. – Involve Bank Group staff more actively to help identify informants. (b) Be more transparent on the following issues of process: – Informant base: Disclose the number of informants for each indicator at the country level, differentiating between those who complete questionnaires and those who provide ―supplemental‖ information. – Changes in data: Disclose all data corrections and changes as they are made. Explain their effect on the rankings, and, to facilitate research, make available all previously published data sets. – Use of the indicators: Be clear about the limitations in the use of the indicators for a broader policy dialogue on a country‘s development priorities. (c) Revise the paying taxes indicator to include only measures of administrative burden. Since the tax rate is an important part of the business climate, DB should continue to collect and present simple information on corporate tax rates, but exclude it from the rankings (as it does for information on nonwage labor costs in the employing workers indicator). A wider range of informants should also be engaged for the paying taxes indicator. 14.2 To make its reform analysis more meaningful, the DB team should: (a) Make clear that DB measures improvements to regulatory costs and burdens, which is only one dimension of any overall reform of the investment climate. (b) Trace the impact of DB reforms at the country level. The DB team should work with country units to analyze the effects of implementing the reforms measured by the DB indicators (such as revised legislation or streamlined process) on: (i) firm performance, (ii) perceptions of businessmen on related regulatory burdens, and (iii) the efficiency of the regulatory environment in the country. 14.3 To plan future additions to or modifications of the indicators, the DB team should: (a) Use Bank analyses to drive the choice of DB indicators. Business Enterprise Surveys, Investment Climate Assessments, and other work can help determine stakeholders‘ priorities for domestic private sector growth. The DB team should use such analyses to determine the choice of new indicators, and periodically assess its current set of indicators. (b) Pilot and stabilize the methodology before including new indicators in rankings. Frequent changes in methodology make comparison across time less meaningful. New indicators should be piloted (that is, data collected and published for comment, but not factored into the rankings) until the methodology is validated and stabilized. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 14.1 To improve the credibility Completed Completed and quality of the rankings. High Substantial and and Archived Archived 14.2 To make its reform analysis Completed Incomplete more meaningful. Substantial Medium and and Archived archived 14.3 To plan future additions to Completed Completed or modifications of the High High and and indicators. Archived Archived MANAGEMENT RESPONSE 2012 14.1 To improve the credibility and quality of the rankings, the DB team should: Original Response: Mostly Agreed. Bank Group management mostly agrees with this recommendation, noting that it primarily points to the importance of intensifying the rigor of recruiting and maintaining a large pool of expert respondents. – To implement the first part (point a) of this recommendation, management has hired a respondents‘ manager on the DB team. The task of the respondents‘ manager is to select and increase the number of respondents, focusing in particular on the poorest countries and other economies with the fewest number of respondents. In addition, management commits to increase the involvement of Bank Group staff in recruiting respondents and to conduct annual data collection visits to the 50 economies with the fewest number of respondents. Thirdly, management will expand the piloted practice of giving out awards to the respondents who have contributed high-quality data over a sustained period of time. Such awards serve to express gratitude for the respondents‘ efforts and to maintain the pool of respondents. – To implement the second part (point b) of this recommendation, management commits to disclosing the number of respondents for each indicator at the country level, starting with the launch of Doing Business 2009. Management is also making available details on data corrections/changes and methodology changes that have been made in the year 139 following the launch of the previous report. Lastly, management commits to expanding the discussion on the limitations in the use of the DB indicators in the ―Methodology‖ section of the report and on the Web site. However, management disagrees with the recommendation to make available all previously published data sets, not corrected for errors and methodology changes. This practice is unorthodox and is not followed by other major primary data providers. The data used in the background research for DB are already published on the ―Research‖ page of the DB Web site. The full time series of DB data, corrected for errors and methodology changes, is also available at the ―Get Full Data‖ page of the DB Web site. These two data sources have been widely used by researchers, with more than 800 academic papers utilizing the DB data. – Management mostly disagrees with the last point (point c) of the recommendation. The tax rate is often identified as a major constraint to business activity in the World Bank Enterprise Surveys. Including a measure of overall tax burden Management Action Record Recommendation Management Response in the DB indicators provides a complete treatment for the topic of paying taxes. Focusing only on the administrative burden of paying taxes will take the DB methodology away from covering a broader spectrum of areas relevant to small domestic businesses. However, management commits to expand the range of respondents on the paying taxes‘ survey by recruiting a larger set of accounting and tax experts. 14.1 a) Take a strategic approach to selecting and increasing the number of contributors: Increase the overall number of contributors for DB2012: Management continued to increase the respondent pool with each new report. The number of contributors was increased from 5,540 in the DB08 report (published September 07), to 6,773 in the DB09 report (September 08), to 8,040 in the DB2010 report (September 09), and to 8201 in the DB2011 report (November ‗10). This year, the DB2012 report counted on the participation of 9,608 contributors (October ‗11). DB12* % DB12* % increase increase (excluding (including new new countries countries DB08 added*) added*) Starting a 1697 75% 1755 81% Business 968 Dealing with 828 68% 837 70% Construction Permits 492 Registering 1250 66% 1257 67% Property 749 Getting Credit 701 1251 78% 1277 82% Protecting 1111 128% 1139 134% Investors 486 Paying taxes 708 1236 74% 1276 80% 140 Trading across 854 - 6% 869 - 4% borders 913 Enforcing 1070 41% 1088 58% contracts 687 Resolving 1016 53% 1044 57% insolvency 661 Employing 1068 61% 1092 64% workers 663 *Note: From DB08 to DB12 the sample of countries increased from 178 to 183 economies. * Note 2: 782 private and public sector professionals contributed to the new Getting electricity indicator The Doing Business team appointed a staff member to focus on contributor management who, in collaboration with other units in the World Bank Group, has worked to increase the number of contributors. To ensure contributor selection and participation in particularly hard to reach countries, the team travels to a selected group each year. In 2011, the Doing Business team traveled to 40 countries to verify information, collect relevant laws and recruit new contributors. Contributors are selected based on their expertise in the topic. For Doing Business 2012, the contributor base was expanded and further diversified by including municipalities, company, securities commissions and collateral registries in the relevant areas. The Doing Business team also continues to collect relevant laws and regulations underlying the indicators. Doing Business 2012 also benefited from input received during the June 2011 update review in collaboration with WBG country offices and Executive Directors‘ offices regarding changes in business regulations or institutions that impact the indicators. The team received updates from 63 governments and additional input related to 80 economies through WBG colleagues. Given the finite pool of possible recruits, several actions have been taken to improve retention rates of contributors. A periodic newsletter is sent out regularly, keeping contributors up-to-date on the latest project news. New online surveys were introduced for the Doing Business 2011 cycle to facilitate the collaboration of the contributors. Over 13% of responses received were completed online. A Client Relationship Management system was introduced to accurately manage the contact database. Contributors are also invited to participate in project related events. Among others, contributors participated at events in Afghanistan, Brunei, Chile, Georgia, Kazakhstan, Malaysia and Vietnam. Moreover, the Doing Business has developed alliances with internationally recognized firms such as Allen & Overy, Baker McKenzie, Cleary Gottlieb Steen & Hamilton, Ernst & Young, and KPMG to expand the number of contributors. Establish and disclose selection criteria for contributors: To ensure accurate interpretation of regulations and reliable time estimates, Doing Business works with local experts who routinely administer or advise on legal and regulatory requirements. Given the nature of the DB methodology, the objective is to find the most knowledgeable and experienced contributors in each area of business regulation. DB targets professionals in various areas of expertise in line with the topics covered by DB and ensures that they are fully familiar with DB methodology and relevant case study assumptions. Accordingly, DB contributors are selected based on specific criteria such 141 as their area of expertise and practice specialty in the case of lawyers. DB encourages stakeholders including governments to nominate local contributors and invites local experts to contribute to the project through its website. Once nominations are received, selection is finalized once this criterion is ascertained to the best of the team‘s ability and available information. The selection criteria and profile of contributors are the following:  Commercial Lawyers  Notaries  Judges  Architects  Trade logistics specialists  Accountants  Electricians In addition, government agencies or institutions directly related to an area covered by Doing Business are also selected and invited as contributors (e.g. company registrars, securities commissions, public credit registries, private credit bureaus, land registry officials, tax authority officials, utility companies). The selection criteria and method to nominate contributors are included on the website: http://www.doingbusiness.org/contributors/doing-business/become-partner Focus on economies and indicators with fewest contributors: Special efforts and attention continue to be accorded to economies and indicators with the lowest numbers of contributors. Contributor recruitment and data collection missions were conducted in 33 countries for DB2011 and in 40 countries for DB2012. New contributors were recruited in all countries visited in 2010 and 2011, including in economies in which contributor recruitment was a challenge, such as Bhutan, Bosnia and Herzegovina, Chad, Congo (Rep), Dem. Rep. of Congo, Fiji, Guinea Bissau, Kiribati, Lao PDR, Lesotho, Mauritania, Papua New Guinea, Samoa, Swaziland, Vanuatu, Venezuela and Zimbabwe. The number of contributors per indicator in DB2012 ranged from 837 globally (for Dealing with construction permits) to 1,755 globally (for Starting a Business). For the Protecting Investors indicator, for example, the number of contributors increased from 874 in DB11 to 1,139 in DB2012, or about a 30 percent increase. The contributor base for the Paying Taxes indicator was further diversified. In DB2012 the number of economies with contributors other than PricewaterhouseCoopers for the Paying Taxes indicator increased to 175. On average, each indicator increased its contributor base by 40% compared to DB08 (as presented in the contributors table above). Formalize contributions of supplemental contributors: Management maintains a database of contributor responses in the form of self-filled surveys for the great majority of contributors. On relatively limited occasions, these surveys are elaborated or supplemented by team-filled surveys after conference calls and country visits, usually because of language barriers or technological constraints (such as lack of access to 142 computers or facsimile machines). Moreover, DB consults contributors who have knowledge or expertise in a particular area of a survey and thus offer partial survey responses, for example consulting utility companies to respond to the relevant section of the ―dealing with construction permits‖ survey. In these cases, DB maintains either the self-filled relevant portion of the survey or other formal documentation of the interaction. All people who provide a substantial contribution are included and recognized in the contributor list, both in the published report and on the DB website: http://www.doingbusiness.org/contributors/doing-business. Involve Bank Group staff to help identify contributors: For DB2011 and DB2012, the DB team and its Management have continued to consistently invite World Bank Group staff, as well as Executive Directors and their staff, to nominate contributors. Special effort has been made to encourage staff (both World Bank and IFC) based in country offices to nominate contributors. These invitations and efforts have continued to be made in meetings, country visits, and other communications, and they continue to be a source of new contributors. As a result, for the DB 2011 and DB2012 report, country staff in Algeria, Armenia, Belarus, Bhutan, Cameroon, Chad, Rep. of Congo, Japan, Kazakhstan, Lao PDR, Mali, Malawi, Montenegro, Mauritania, Morocco, Nepal, Nigeria, Saudi Arabia, Tajikistan, Trinidad and Tobago, and Uzbekistan have assisted the DB team in identifying new contributors. MAR 2012: 14.1 b) Be more transparent on the following issues of process: Informant base: Contributor lists are published in DB reports and on the website. (Contributors are only excluded from this list upon their own request.) Management makes available on the DB website the number of contributors by indicator at the country level. This information is available by selecting any country at: http://www.doingbusiness.org/contributors/doing-business. An electronic law library with links to relevant laws and legislations is also accessible at: http://www.doingbusiness.org/law-library, Changes in data: Data correction rates are published annually, for DB11, are found on page 111, and, for DB12, are found on page 42. The approach to methodology changes and data corrections are described in the ―About Doing Business‖ section of the DB 2011 and DB2012 report. In addition, a ―Data challenges and revisions‖ section is included in the ‗Data Notes‘, which complements that information. In 2010, Management implemented a modernization of the DB website, enhancing the overall access to information. Following IEG recommendations, particular emphasis was put on the availability of information on methodology and revisions, and making the pages more user-friendly. A full page on ‗Methodology‘ (http://www.doingbusiness.org/methodology) is now accessible from the main menu on the DB home page. In addition, a large ‗More Information‘ box is included on that page with direct links to the following: 143 1) Changes to methodology this year (http://www.doingbusiness.org/Methodology/Methodology-Note) 2) Changes to methodology of previous years http://www.doingbusiness.org/Methodology/Changes-to-the-Methodology) 3) Common misconceptions about Doing Business (http://www.doingbusiness.org/methodology/common-misconceptions) 4) Data Corrections to DB11 data (which includes both published and corrected data) http://www.doingbusiness.org/~/media/FPDKM/Doing%20Business/ Documents/Methodology/Others/DB12-corrections.xls 5) Data Corrections to DB10 data (which includes both published and corrected data) http://www.doingbusiness.org/~/media/FPDKM/Doing%20Business/ Documents/Methodology/Others/DB11-corrections.xlsx 6) Data Corrections to DB09 data (which includes both published and corrected data) http://www.doingbusiness.org/~/media/fpdkm/doing%20business/documents/meth odology/others/db10-corrections.pdf 7) Economy characteristics that were used by the DB report http://www.doingbusiness.org/data/exploreeconomies/economycharac teristics 8) A link to the Doing Business IEG report: http://go.worldbank.org/CYH45V15G0 In response to follow-up discussions with IEG during 2010 regarding the Data Corrections file referenced in point 4 above, Management has now published a user- friendly Excel file (to replace the previous PDF file) which contains the details of corrections made to the last three years of Doing Business data. For example, the DB2012 corrections sheet includes the DB11 corrected data, as well as the DB11 data as it was published (same for the DB2011 corrections sheet). The DB10 corrections sheet which illustrates changes to DB09 data is also available on the website. Moving forward, the data corrections in future years will be published in a similar fashion. The publication of all historical data corrections beyond DB09 data remains a technical challenge due to the data-structure that was maintained in the past, which makes such visualization rather difficult. Further to the recommendations of the IEG and discussions that ensued, $1.8 million was obtained in an allocation from the World Bank IT Council to build a modern IT system to handle the continuous enlargement of DB database and will have the capacity to record and report on various elements, including corrections and methodology changes. The project is currently ongoing over FY11 and FY12, with an anticipated final delivery in FY13. Use of the indicators: The IEG evaluation noted that, while effective in catalyzing reform debates and dialogue, the Doing Business indicators did not appear to have distorted policy priorities or encouraged policy makers to make superficial changes to improve rankings. DB reports have consistently noted that DB measures only a subset of the broader business environment. For DB2012, Management added a ‗Preface‘ to the report (page v). In the fourth paragraph, it is explicitly states that ―Doing Business is limited in scope. It does not attempt 144 to measure all costs and benefits of a particular law or regulation to society as a whole. Nor does it measure all aspects of the business environment that matter to firms and investors or affect the competitiveness of an economy. Its aim is simply to supply business leaders and policy makers with a fact base for informing policy making and to provide open data for research on how business regulations and institutions affect such economic outcomes as productivity, investment, informality, corruption, unemployment and poverty.‖ The ―About Doing Business‖ chapter (page 17) in the DB 2012 report describes the limitations of the DB indicators in the fourth paragraph. In addition, the ―About Doing Business‖ chapter introduced in DB09 was maintained and updated in DB10, in DB 2011, and in DB2012 (pp.16-25). The chapter transparently lays out for the user the scope and limitations of the project and clarifies the use of the indicators. This chapter also states that ―World Bank Group dialogue with governments on the investment climate is designed to encourage critical use of the data, sharpening judgment, avoiding a narrow focus on improving Doing Business rankings and encouraging broad-based reforms that enhance investment climate.‖ 14.1 c) Revise the paying taxes indicator to include only measures of administrative burden Management‘s difference of opinion on this recommendation was noted in our original Management Response. We note that the IEG recommendation to revise the paying taxes indicator to include only measures of administrative burden, and excluding taxes and other mandatory payments to government, is at odds with the separate recommendation (see 2.3.a) on the use of the other World Bank analyses to determine the priorities for regulatory reform. In the World Bank Enterprise Surveys (www.enterprisesurveys.org), for example, tax rates are considered a top obstacle in twice as many countries as tax administration. Doing Business records the time and cost related to paying taxes and mandatory contributions from the perspective of a local SME, for whom tax and mandatory payment burdens are an important consideration. In recognition of divergent perspectives on this point, and as part of its commitment to engage with key stakeholder groups, the DB team has been engaged in a consultation process with the International Tax Dialogue (ITD), which includes the OECD, IMF, IDB, EC and World Bank Group, for the last 3 years. As a result, several questions were added to the DB11 Paying Taxes surveys regarding tax administration to expand the data collected and inform the indicators. The new questions look into the time firms spend on post-filing interactions with the tax authorities as well as time spent on taxes other than consumption, labor and profit taxes (the current time sub-indicator only measures time spent on these 3 types of taxes). The survey also added more detailed questions on employment taxes and social security contributions paid by employees in addition to the employer to inform research on the labor tax wedge and see how many countries are affected by a greater burden placed on the employee versus the firm. For the Doing Business 2012 data collection cycle, the team expanded the consultative process beyond the ITD to include: 145  Internal consultations with colleagues working on taxation within the World Bank and IFC  External consultations with stakeholders on methodology with stakeholders including private sector representatives, tax experts from IMF and CIAT and employee representatives. Consultations focused on suggestions to improve the methodology of the Paying Taxes indicator, Upon completion of these consultations, management introduced a threshold for the total tax rate for the purpose of calculating the ranking on the ease of paying taxes. All economies with a total tax rate below the threshold (which will be calculated and adjusted on a yearly basis) now receive the same ranking on the total tax rate indicator. Since the total tax rate is 1 of 32 indicators included in the ranking on the overall ease of doing business, this change has minimal effects on the overall rankings. The correlation between rankings on the ease of paying taxes with and without this threshold is 99%. The threshold is not based on any underlying theory. Instead, it is meant to emphasize the purpose of the indicator: to highlight economies where the tax burden on business is high relative to the tax burden in other economies. Giving the same ranking to all economies whose total tax rate is below the threshold avoids awarding economies in the scoring for having an unusually low total tax rate, often for reasons unrelated to government policies toward enterprises. For example, economies that are very small or that are rich in natural resources do not need to levy broad-based taxes. For more details on the calculation of the threshold, 15.2. Original Response: Agreed. Bank Group management agrees with this recommendation and will strive to make it even clearer in future DB reports and presentations that DB covers only some dimensions of the overall reform of the investment climate. Management also commits to a measurement and evaluation agenda, in partnership with World Bank country units and IFC regional facilities, to document the effect of DB reforms on a set of economic and social indicators. The World Bank Enterprise Surveys in particular will be used for this work. MAR 2012: 14.2 To make its reform analysis more meaningful: 14.2 a) Make clear what DB measures and what it does not: DB reports have consistently noted that DB measures only a subset of the broader business environment. In DB12, Management included a ‗Preface‘ to the report (page v) providing background on the project. In the fourth paragraph, it is clearly stated that o ―Doing Business is limited in scope. It does not attempt to measure all costs and benefits of a particular law or regulation to society as a whole. Nor does it measure all aspects of the business environment that matter to firms and investors or affect the competitiveness of an economy..‖ 146 The ‖About Doing Business‖ chapter (page 17) in the DB 2012 report also describes the limitations of the DB indicators in the fourth paragraph. ―Doing Business does not measure all aspects of the business environment that matter to firms or investors—or all factors that affect competitiveness. It does not, for example, measure security, corruption, market size, macroeconomic stability, the state of the financial system, the labor skills of the population or all aspects of the quality of infrastructure. Nor does it focus on regulations specific to foreign investment. o In addition, the ―About Doing Business‖ chapter introduced in DB09 was maintained in DB10, in DB 2011 and again in DB2012 emphasizes that ―World Bank Group dialogue with governments on the investment climate is designed to encourage critical use of the data, sharpening judgment, avoiding a narrow focus on improving Doing Business rankings and encouraging broad-based reforms that enhance investment climate.‖ In response to follow-up discussions with IEG during 2010, in the DB2011 and DB2012 publications, additional changes were introduced to ensure precise use of language. For example, the previously named ―Top 10 Reformers‖ are now described as ―the 10 economies that improved the most in the ease of doing business‖. Precise language is also used to describe and refer to changes in business regulations or institutions that are recorded through the indicators and reported on in the report (e.g. ―introduction of a one-stop shop‖, ―passing of a new company law‖, or ―improvements in the area of starting a business‖). Appropriate phrasing was adopted in all relevant press materials supporting the launch of DB11 and DB12. 14.2 b) Trace the impact of DB reforms at the country level: A key aim of the DB project is to inform business regulatory policy decisions and policy advice where appropriate. Doing Business tracks and reports annually on regulatory changes implemented in the areas which it covers. A number of efforts have been undertaken to track the direct impact of such changes, and to trace the linkages to desired economic outcomes. They include statistical data obtained directly from governments, research from within the WBG, as well as external findings. For DB2011, the DB team established communication with governments that had improved their business regulations as measured by DB to impact data such as number of users of new registries. In particular, the governments of Burkina Faso, Colombia, the Arab Republic of Egypt, the Republic of Korea, the former Yugoslav Republic of Macedonia, Mexico, Portugal and Rwanda provided such information, which was presented in the DB11 report. The DB2012 report includes detailed economy case studies for the Republic of Korea, the former Yugoslav Republic of Macedonia, and Mexico. To show longitudinal change in business regulation over time and at the country level, DB2012 introduced a new measure to illustrate how the regulatory environment for local businesses in each economy has changed over time. The distance to frontier measure illustrates the distance of an economy to the frontier and shows the extent to 147 which the economy has closed this gap over time. The frontier is a score derived from the most efficient practice or highest score received for the strongest property rights protections on each of the component indicators in 9 Doing Business indicator sets by an economy since 2005. In the past 6 years policy makers in 163 economies made domestic regulations more business-friendly (see figure below). They lowered barriers to entry, operation and exit and strengthened protections of property and investor rights. : The dataset has enabled a growing body of empirical research. DB2012 reports on findings from recent research. Management also introduced a new ‗Research‘ section to the DB website in 2010: http://www.doingbusiness.org/research, which cites peer- reviewed articles published in the top 50 economic journals. This new section, which will be updated yearly, highlights relevant research on the linkages of business regulations with business activity and economic outcomes. The information is searchable by Doing Business topic, and abstracts are available for quick reference. To encourage new research and information diffusion, the website also provides a mechanism for submission of publications. A section summarizing some of the key findings of empirical research was included in the ―About Doing Business‖ chapter (page 20). This section provides a summary of some of the main findings of the research on Doing Business indicators related issues. Some of these findings are: i) Lower costs for business registration encourage entrepreneurship and enhance firm productivity; ii) Simpler business registration translates into greater employment opportunities in the formal sector; iii) A sound regulatory environment leads to stronger trade performance; and iv) Regulations and 148 institutions that form part of the financial market infrastructure—including courts, credit information systems, and collateral, creditor and insolvency laws—play a role in easing access to credit. 1.1 The DB team is collaborating with other WBG departments, including the Enterprise Analysis unit in GIA, DEC and CIC to better understand the impact of business regulation reforms, and to trace the linkages to desired economic outcomes. In DB2011, a new sub-section was included to each chapter: ―What are some results?‖ DB2011 presented some initial findings from such research supplemented by information on short-term results such as cost savings for firms or number of beneficiaries. New initial results are presented on the relationship of regulatory changes to simplify business start-up, dealing with construction permits, registering property and trading across borders in East Europe and Central Asia, the region most active in implementing policy changes in those areas, and labor productivity. FPD is also currently mobilizing funding for further research using those datasets as well as the subnational Doing Business project datasets and census data where available, to expand the analysis of the impact of regulations on a wider range of firm performance measures. In addition, a research competition was initiated in 2011 to stimulate external research as part of preparations for a planned research conference in 2012 which will focus on the impact of DB reforms. This research conference will mark the 10th anniversary of the Doing Business project. A mix of researchers and policymakers will present their research and experience regarding Doing Business (http://www.doingbusiness.org/special-features/Conference). 15.3. Original Response: Agreed. Bank Group management agrees with this recommendation and will direct the DB team toward using other Bank Group analyses, and in particular the Enterprise Surveys and Investment Climate Assessments, for both determining the choice of new indicators and periodically assessing the existing set of DB indicators. Management also commits to publishing new sets of indicators in future DB reports for comment, while not factoring those in the rankings until their methodology is validated by academic research. MAR 2012: 14.3 To plan future additions to or modifications of the indicators, the DB team should: 14.3 a) Use Bank analyses to drive the choice of DB indicators: Management agrees with the recommendation to use other World Bank analyses, most importantly the Enterprise Surveys and Investment Climate Assessments, to inform the choice of topics in Doing Business and enrich the analysis in future reports. The current 149 work on the pilot indicator on getting electricity was derived from the results of Enterprise Surveys. In DB2011, the sample of economies for this indicator was expanded from 140 economies to 176. In DB2012, data was collected for the full set of 183 economies and the indicator was incorporated into the global ranking. Prior to the publication of DB2011, an extensive process of consultation within the World Bank Group was carried out by the Doing Business team on all of its indicators. This consultation process included the six regions of the World Bank Group and other VPU's. These consultations continued during 2011, including after the DB2012 publication, in particular with the Legal department. The result of these consultations and peer reviews fed into the questionnaires for the DB2013 data collection process. 14.3 b) Pilot and stabilize the methodology before including new indicators in rankings: For new indicators, Management‘s approach is to first publish new indicators for comment, not factoring them into rankings until their methodology is validated by academic research. The work on a new indicator on ―getting electricity‖ illustrates this approach: 1) A pilot for 40 countries was launched in 2006/2007. 2) Between 2008 and 2009, the team collected data for 140 economies. To ensure accuracy of the data the team consulted with more than 600 experts from both private and public sectors. 3) In 2009, the team produced a discussion paper on the results of the research: ―Getting Electricity – a pilot indicator data set‖ 4) In 2010, the team drafted an academic background paper on the Getting electricity indicator, which was presented at the International Conference on Infrastructure Economics and Development. 5) Throughout 2009 and 2010, the team presented the pilot results and sought input from relevant audiences (mainly utility regulators) at global and regional forums, such as the World Economic Forum for Energy Regulation in Athens in September 2009, followed by several other regional regulatory conferences. These included: the 7th Annual Organization of Caribbean Utility Regulators Conference (OCCUR), Trinidad & Tobago, November 4-6, 2009, the East Asia and Pacific Infrastructure Regulatory Forum in Hanoi, Vietnam - November 19 - 27, 2009; the USAID – Energy Reform Group – January 6, 2010; and the PURC/World Bank International Training Program on Utility Regulation and Strategy in Gainesville, Florida – January 21, 2010. The feedback provided by regulators and PURC was unanimously positive. PURC included the data and report in its standard training material for regulators. 6) Internally, World Bank staff (MENA, SAR, LAC, Infrastructure and Energy teams) provided substantive feedback on the methodology and orientation of the indicator during the research process. 7) The data of the Getting electricity indicator was included in both Doing Business 2010 and 2011 reports as an annex. 8) An indicator advisory group meeting was held on January 27, 2011 at which the methodology of the indicator was presented. Participants included members of 150 GIA, World Bank experts, and renowned academics. 9) The FPD Chief Economist Office of FPD reviewed and approved the methodology of the indicator. 10) Finally, the dataset of 183 economies was completed in June 2011 and the indicator was included in the Doing Business 2012 report and ranking. Management has been applying this approach to the revision of one of DB‘s existing indicators: the Employing Workers Index (EWI). The EWI was undergoing a review to improve the indicator to allow it to better measure labor regulation within internationally accepted worker protection standards. The World Bank Group has been working with a consultative group - including experts from the ILO, employers‘ and employees‘ representatives, OECD, civil society, labor lawyers, and private sector - to discuss the employing workers methodology, as well as suggest future areas of research (The report of the consultative group is available at http://www.doingbusiness.org/data/exploretopics/~/media/FPDKM/Doing%20Busi ness/Documents/Methodology/EWI/Final-EWICG-April-2011.doc). In addition, Doing Business in collaboration with the HDN network is collecting data and conducting further research on worker protection. In the meantime, the EWI has been removed from the Doing Business rankings, while the report continues to publish extensive data on labor regulation in 183 economies. Since last year, the mandate of the recently appointed FPD Chief Economist was expanded to include quality assurance of the indicators produced within FDP. Consequently the Doing Business team has engaged with the FPD Chief Economist Office to consult on the existing DB indicators and potential future indicators. DB2012 underwent extensive review by the FPD Chief Economist Office. IEG RESPONSE 2012 14.1 To improve the credibility and quality of the ratings As noted in management‘s submission, the 2012 Doing Business report makes further incremental improvements in the transparency and clarity of presentation in the directions recommended by IEG. It now acknowledges, as urged by IEG, that ―[DB] does not attempt to measure all costs and benefits of a particular law or regulation to society as a whole.‖ There remain three gaps between the evaluation‘s recommendations and DB‘s practice: 14.1 (a). DB data are supplied by selected informants in each country. The number of informants per indicator ranges from 1 to 24, with a median of about 6. Doing Business states that it obtains different results from enterprise surveys in part because it ― gathers the considered views of experts who examine the laws and rules underlying the business regulatory framework‖ (2012 Report, p. 24, footnote 10). It is thus important to know how the informants are selected so as to ensure that they accurately grasp the regulatory constraints on small businesses. The evaluation recommended disclosing the selection criteria for informants but DB continues to list only the informants‘ occupational categories, and not 151 criteria related to experience or expertise. 14.1 (b). DB continues to retroactively change already-published data and the country rankings they generate. The 2012 report acknowledges this practice more prominently than in previous years and provides–as suggested by IEG – tables listing each original and revised data point. The report does not, as recommended by the evaluation, break down the reasons for these changes. The 2012 report acknowledges —as recommended by the evaluation–that the 2011 country rankings have been retroactively modified22 but does not make clear that countries‘ rankings have changed by as much as 26 rungs, as illustrated in the table below. The 2011 rankings for Vietnam and Tajikistan, originally classed as among the ‗top ten‘ DB reformers for 2011, were retroactively downgraded by 12 and 13 rungs respectively. Egypt, for several years a featured top performer, was also retroactively downgraded. This instability of the rankings should be more clearly acknowledged because it undermines their usefulness as research and monitoring tools. 14.1 (c). IEG recommended that the ‗paying taxes‘ indicator be revised to measure only administrative burden (like DB‘s other indicators) and to exclude the ‗total tax rate‘, because lower total taxes on business – while good for firms – is not necessarily the best fiscal policy for a country. In 2012, DB modified the indicator to avoid rewarding egregious tax havens. But outside of these countries with extremely low taxes, lower total taxes on business will still generate a higher rating on this indicator. DB should make clearer that this indicator rewards what is beneficial to business, not necessarily to the economy as a whole. 14.2 (a). DB reports have improved their clarity about the limitations on what DB measures. In some places, however, they still refer to ―improvements in business regulation‖ and ―tracking business regulation‖ rather than ―improvements in the particular indicators measured by DB‖. Doing so implies that DB‘s 10 indicators can serve as proxies for the Largest Upward revisions of 2011 Rankings Largest Upward revisions of 2011 Rankings +DIFFERENCE (UPWARD -DIFFERENCE REVISION) (DOWNWARD REVISION) Brunei 26 El Salvador 26 Grenada 24 Belarus 23 Dominica 24 Kyrgyz 23 Trinidad and Tobago 21 Mexico 19 Sudan 19 Kiribati 18 Uruguay 17 Bosnia and H. 17 Philippines 14 Egypt 14 Jordan 16 Dominican Republic 14 Solomon Islands 15 Azerbaijan 14 Tunisia 15 Uzbekistan 14 Afghanistan 13 Tajikistan 13 Antigua and Barbuda 13 Armenia 13 Qatar 12 Cyprus 12 Poland 11 Vietnam 12 Yemen 11 Zimbabwe 11 Bangladesh 11 Iran 152 regulatory environment as a whole, which has not so far been demonstrated. 14.2 (b). There exists a literature that links the impact of regulations in general on economic outcomes, and some of this is cited on DB‘s website and in ―About Doing Business‖. There remains, however, little published research that tests the extent to which DB‘s 10 particular indicators, the specific actions they measure, and their exclusive attention to de jure conditions, are salient to economic outcomes. The research conference planned for December 2012 may help to fill this gap. 14.3 This recommendation is rated ‗complete‘ and may be archived. IEG will retire these recommendations but plans to conduct a case study in FY13 as a follow up to the MAR. 153 15. The World Bank Group Guarantee Instruments 1990–2007 LIST OF RECOMMENDATIONS 15.1 Take a strategic approach and make a decision whether to maintain the existing organizational structure while addressing some of the important problems, or develop and propose an alternative organizational structure to the Board. 15.2 Under any scenario, take action to introduce greater flexibility in the use of guarantee instruments in response to dynamic country and client needs and market developments by  Revising existing policies and regulations on guarantees to minimize supply-driven product restrictions where most needed and to allow product differentiation on the basis of value added.  Ensuring that adequate incentives exist for staff to offer the full array of WBG guarantees and PRM products to private sector clients within a single menu of options.  Establishing more systematic links between advisory services and the deployment of WBG PRM instruments and other products, particularly in infrastructure.  Following a consistent approach to pricing of PRM across its guarantee instruments to avoid potential distortions.  Strengthening internal awareness of the guarantee instruments and the incentives and skills for their use and reducing transaction costs where possible, keeping in mind the importance of maintaining adequate processes and regulations for risk management. 15.3 If a new organizational structure is developed and proposed, consider at least three alternative perspectives for organizational realignment: the client perspective, the country perspective, and the product perspective.  Under the client approach, all products for private sector clients, including guarantees and PRM instruments, would be offered through a single window.  Under the country approach, the deployment of WBG guarantee and PRM products would be made according to country needs, under a management arrangement common for all the three institutions.  Under the product approach, the bulk of guarantee/insurance products would be managed under one institutional roof. 15.4 If the current organizational structure is maintained, direct management of each individual WBG institution to improve the delivery of its own guarantee/insurance products by MIGA management  Proposing to MIGA‘s shareholders amendments to its Convention to remain relevant and meet its market potential. Considering, in the meantime, alleviating several constraints derived from its operational regulations and policies.  Increasing its responsiveness to market demand by addressing internal weaknesses that reduce efficiency and slow responsiveness without lowering MIGA‘s financial, social, or environmental standards.  Improving its client relationship management, including aftercare, to enhance the value MIGA adds and increase its client retention. Bank management  Maintaining and promoting the partial credit guarantee instrument as a potential effective countercyclical tool to leverage government access to commercial funds and extending such access to IDA countries.  Creating awareness among Bank staff of the potential use and benefits of partial risk guarantees and building necessary skills.  Developing a marketing strategy that encompasses both governments and the private sector to better identify situations in which the role of a partial risk guarantee can make a difference.  Streamlining processing steps to reduce both internal disincentives to working on partial risk guarantees and transaction costs for private sector clients while ensuring that crucial measures for social and environmental safeguards and risk management are maintained. IFC management  Mainstreaming its guarantee products through its operations departments in the same manner that its equity and loan products are deployed.  Assessing the extent to which it can bring its guarantee products closer to meeting Basel II and regulatory requirements in general, so that the guarantee beneficiaries can use IFC products more effectively for capital, provisioning, and exposure relief.  Revisiting its approach to RSFs to increase flexibility and improve the attractiveness of the product.  Scaling up successful models in energy efficiency, education, and capital market development based on the use of guarantee structures. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 15.1 Take a strategic approach and make a Differen Differen decision whether to maintain the existing ce of ce of organizational structure while addressing Opinion Opinion some of the important problems, or develop / Not / Not and propose an alternative organizational tracked tracked structure to the Board. 15.2 Under any scenario, take action to introduce Incompl greater flexibility in the use of guarantee Medium Medium ete and instruments in response to dynamic country archived 155 Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG and client needs and market developments by 15.3 If a new organizational structure is Differen Differen developed and proposed, consider at least ce of ce of three alternative perspectives for Opinion Opinion organizational realignment: the client / Not / Not perspective, the country perspective, and the tracked tracked product perspective. 15.4 If the current organizational structure is maintained, direct management of each Incompl individual WBG institution to improve the Medium Medium ete and delivery of its own guarantee/insurance archived products by MANAGEMENT RESPONSE 2012 As noted in the original Management Response, Management has opted to maintain the existing organizational structure while addressing some of the issues of coordination and marketing within the WBG. Hence, there are no agreed actions under sections 15.1 and 15.3 and the remaining parts of this Management Response focus on sections 15.2 and 15.4. 15.2 Original Response: As noted in the Management Response, the WBG institutions have complementary but differing mandates that are defined under their respective charters. Under these mandates, each member of the group has developed different products serving the different needs of its clients. Management therefore believes that current institutional arrangements are adequate, and issues of coordination and marketing can be addressed without a change in institutional structure. 15.2 a) The specific characteristics of each entity‘s products are governed by the entity‘s respective charters and policies, based on the clients it serves. Each institution continues to work to eliminate unnecessary restrictions, if any, in the use of its products. Management does not agree that an across-the-board revision covering very different guarantee products offered by the WBG members is needed. More effective coordination would better address these concerns (see below). Management would also like to point out that IFC has no specific policies restricting the offering of partial risk guarantees within its institutional boundaries and calls on MIGA to provide political risk insurance as needed. In accordance with its internal guidelines, IFC does not offer guarantee products that replicate the offerings of MIGA. 15.2 b) Management will assess the feasibility of increased staff incentives in the context of potential benefits. 156 IFC is prepared to work with MIGA and the Bank with regard to marketing the various PRM products through IFC‘s channels with clients. For example, there could be scope to leverage IFC‘s industry departments‘ relationships with key global private sector players to offer PRM instruments along with other financing options as appropriate. As the report notes, the newly established Client Relationship Management System could also be a vehicle for coordinated marketing efforts. The incentives for MIGA or Bank staff to utilize these channels could be considered. 15.2 c) The World Bank and MIGA will explore ways to establish more systematic links between advisory services and their PRM instruments. For MIGA, as solely a provider of political risk insurance and with no commercial interest, close collaboration with Bank policy advice is possible and indeed encouraged, as there is full alignment of interests between sound policy advice and such guarantees. The infrastructure area is in fact a good example of WBG coordination on advisory and financing services. IFC‘s investment team, which is separate from the advisory team, can offer a financing package to the winning bidder subject to satisfactory due diligence. Such a package could include WBG guarantee products as appropriate. Advisory teams working with government clients will, as a matter of course, need to advise a government on the best options for ensuring a successful and competitive bid, for a concession, build-operate transfer, or other structure. Those options may lead to recommendations that either indications of interest from potential financiers (IFC or others) or of availability of political risk reduction mechanisms (WBG or others) be included in bidding information packages to increase the prospects of the government achieving its objectives. Governments are of course always free to reject such recommendations. Given the possible appearance of conflicts of interest, potential conflicts arising from such recommendations are fully disclosed to clients and mitigating measures as per WBG Conflict of Interest policies are put into effect if the governments choose to follow such recommendations. WBG units may not be able to offer financing or guarantee products if the winning bidder does not turn out to be acceptable to them. 15.2 d) Each of the Bank group institutions uses pricing methodology that reflects the unique characteristics of it charter and its clients. Management will provide guidance to staff to ensure the consistency of Bank advice to governments with regard to the ―hierarchy of instruments‖ principle and the fee charged by governments to the private sector to offset the costs associated with issuing a counter guarantee. Such a fee will be considered the default option and application of the guidance will be monitored through the Finance, Economics, and Urban Development Department (the Sustainable Development Network department that supports the Regions‘ guarantee work). 15.2 e) Efforts to increase awareness and ensure adequate skills in different specializations are ongoing. MIGA and the Bank will review their procedures to address any specific issues identified in the report that lead to higher transaction costs. IFC is prepared to provide training to staff on MIGA and/or Bank guarantee products it may be asked to promote. Within IFC guarantee products, IFC‘s Investment Guidelines and Practices provide a detailed description of each product with a note to inform IFC Treasury as soon as is practical of plans to offer a guarantee product. Within the Finance/Treasury Vice Presidential Unit, the Structured and Securitized Products Department is the center of knowledge and practice on guarantee products in IFC, providing structuring guidance to investment staff as needed. IFC 157 will continue to cover guarantee products in relevant training modules for staff. IFC is also mainstreaming a range of innovative financing techniques to investment departments to the degree possible. MAR 2012 Management had agreed to take actions to improve policies and guidelines as necessary and strengthen coordination within the WBG. The previous MARs already described the mainstreaming of IDA partial risk guarantees, thus ending its pilot status, and the expansion of the eligibility criteria for IBRD enclave guarantees, thereby making their use more flexible. The previous MARs also described progress made in comprehensively revising and updating the Bank‘s operational policy for guarantees. Since the January 2011 Bank-wide concept review described in the previous MARs, Management has prepared and conducted a Bank- wide review of an Approach Paper on Modernizing the World Bank‘s Operational Policy on Guarantees, which outlines numerous proposals for reforms of the operational policy as well as the Bank‘s internal procedures for guarantees. The overall aim of the proposed policy reforms is to streamline and consolidate policy requirements; remove outdated and unnecessary restrictions; expand the applicability of Bank guarantees; and align policy requirements for guarantees with their corresponding lending instruments. One of the major proposals is to remove the current limitation of partial credit and policy based guarantees to IBRD countries, thus making them available to IDA-only countries. Many of the reforms proposed in the Approach Paper reflect recommendations in the IEG report. The Paper was discussed with CODE members in October 2011 and currently Management is conducting internal and external consultations in order to elicit inputs and feedback from a wide range of stakeholders in as broad and inclusive manner as possible. Management will prepare a policy paper based on the results of the consultations and plans to seek the Board‘s endorsement of the policy paper by end-FY12. Subsequent to the Board‘s approval, a new OP/BP on guarantees will be issued. In the interim, Management supports Regions in seeking policy waivers that are consistent with the proposals, e.g., a partial credit guarantee component for an operation in Cameroon and additional financing for an operation in Nigeria. Modernizing the Bank‘s operational policy on guarantees is part of a broader agenda to realize the potential of Bank guarantees. Management has taken or is exploring a number of complementary actions to address obstacles to the greater use of guarantees. This broader agenda will be developed as part of the upcoming WBG Infrastructure Strategy FY12-15. Possible complementary actions include the following: (a) counting guarantee exposures only partially against the Bank‘s total possible lending to a country, in order to incentivize client demand; (b) internal corporate review processes of guarantees aligned with those for lending operations; (c) options for scaling up preparation of public-private partnership (PPP) projects which benefit from guarantees, including by providing greater resources for project preparation; (d) improved development and deployment of staffs with the required specialized skills; and (e) strengthened WBG coordination and collaboration, including on outreach to clients on the financial solutions the WBG can provide. Note that Management has already initiated many of the complementary actions for realizing the potential of Bank guarantees. In order to strengthen WBG coordination and also to 158 establish more systematic links between advisory services and guarantee operations, the Bank and MIGA have signed a cooperation and marketing agreement, and a similar agreement has been signed between the Bank and the IFC advisory unit. The Bank‘s policy dialogue and technical advisory services on sector reforms continue to be critical components of guarantee operations. Training workshops, seminars and presentations on guarantees have been carried out within the Bank, in order to increase awareness and ensure adequate skills for guarantees. The Bank already supports PPP development through TA lending and TA grants from the Public-Private Infrastructure Advisory Facility (PPIAF), and in addition Management is exploring possibilities for establishing and funding a dedicated PPP project preparation facility. With regards to guarantee pricing, Management maintains its original response that each of the Bank group institutions uses pricing methodology that reflects the unique characteristics of it charter and its clients. Management considers reviewing the extent to which partial risk guarantees expose IBRD to country risk, which is a key component in its pricing methodology. Management continues to provide guidance to staff to ensure the consistency of Bank advice to governments with regard to the use of their own guarantees to projects and how to offset the costs associated with issuing a counter guarantee, in the broader context of mobilizing private sector financing in difficult market conditions. 15.4 Original Response: 15.4 a) Partially agreed at this time. Extending access of the Partial Credit Guarantee instrument to IDA countries would be an option that will be discussed under the IDA Guarantees Review to be presented to the Board by December 2008. 15.4 b) Agreed. Banking and Debt Management currently provides training on the political risk guarantee and other IBRD/IDA guarantee instruments for operational staff, and also includes a discussion of the guarantees in its general training for task team leaders. The Finance, Economic, and Urban Department (the Sustainable Development Network department that supports the Regions‘ guarantee work) will continue to maintain adequate capacity to respond to demand from task teams for specialized guarantee expertise. 15.4 c) Partially agreed. The potential use of guarantees is most usefully discussed as a part of Country Assistance Strategy preparation, thus making the governments fully aware of the availability of guarantees. Management will work to ensure that the potential use of guarantees is discussed as part of the preparatory Country Assistance Strategy discussions for all countries. In addition, as part of outreach programs for IBRD financial products, Banking and Debt Management routinely includes material on the availability and potential for IBRD guarantees. Management plans to undertake a similar outreach program for IDA guarantees and is exploring various institutional options. 15.4 d) Partially agreed. Management is exploring ways to streamline the processing of guarantees but will not commit before identifying specific measures. Compliance with Bank policies pertaining to environment and social safeguards would not be affected by any changes in policies related to guarantees. 159 MAR 2012 Management continues to take actions to improve the delivery of guarantee products. In order to enhance the use of partial credit (and policy based) guarantees, the Approach Paper on reforms of the Bank‘s guarantee policy includes the proposal to extend them to IDA-only countries. The Approach Paper also includes measures to streamline the processing steps for guarantees and align them with their lending instrument counterparts. The corporate review process for guarantees has been aligned with those for loans and credits, by specifying that the required corporate review, notably depending on the level of risk, is not automatically conducted by the Operations Committee and instead can be carried out by the Regional Operations Committee. Management‘s response above for section 16.2 already describes actions taken to create awareness of guarantees; build the necessary skills among Bank staff; and improve the marketing of guarantees and better identify opportunities to use these instruments. Management has encouraged country teams to seek assistance from FEU/SDN and BDM/TRE when engaging with clients to identify suitable projects. These units continue to participate in CAS discussions on a case-by-case basis in order to help identify appropriate opportunities for guarantees. In addition, Management has maintained incentives to use guarantees by counting only 25 percent of guarantee exposures to the IBRD country exposure limit and IDA country allocations. IEG RESPONSE 2012 15.1 The Bank has continued to make important steps toward enhancing its guarantee products, streamlining processing, and strengthening staff skills. IEG notes that an approach paper on modernizing guarantees has been discussed with the Board and that the Bank is presently undertaking broad consultations on the agenda. Following the consultations, the implementation phase can be expected to contribute to better realizing the potential of the Bank‘s guarantee instruments. These steps and directions are fully in line with the recommendations of the Guarantees evaluation. 15.2 With respect to PRG guarantee pricing, the IEG reiterates that the pricing models for similar products offered by WBG institutions are different. It is not clear what in the Bank‘s charter requires that it price an insurance product for a private company in the same way that it prices a loan to a member country government. 160 16. Decentralization in Client Countries: An Evaluation of World Bank Support 1990-2007 LIST OF RECOMMENDATIONS 16.1 Ensure that Bank support—particularly lending—is underpinned by genuine client commitment to decentralized service delivery, given its importance to the success of Bank interventions. Occasionally, a role for the Bank may be justified in the absence of client commitment (for example, to forestall potentially adverse measures), although the evaluation finds that Bank interventions under such circumstances are not usually effective. 16.2 Encourage the adoption of a more results-based approach to decentralization by helping to develop in-country and Bank capacity for monitoring and evaluation (M&E) that focuses on local outcomes (such as enhanced accountability, greater citizen participation, and improved service delivery) rather than on just the process of decentralization. 16.3 Ensure that Bank support at the country level is (among other things):  Founded on a clear analytical framework based on an integrative understanding of economic, political, and institutional factors at different levels of government and across sectors affected by decentralization  Accompanied by support (from the Bank or others) to develop and maintain local government capacity to the extent feasible. 16.4 Strengthen institutional arrangements within the Bank to ensure that an integrative view underpins Bank interventions, particularly those based on sector-specific entry points. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 16.1 Ensure that Bank support—particularly lending—is underpinned by genuine client commitment to decentralized service delivery, given its importance to the success of Bank interventions. Incomple Occasionally, a role for the Bank may be Medium Medium te and justified in the absence of client archived commitment (for example, to forestall potentially adverse measures), although the evaluation finds that Bank interventions under such circumstances are not usually Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG effective. 16.2 Encourage the adoption of a more results- based approach to decentralization by helping to develop in-country and Bank capacity for monitoring and evaluation Complet Substan Substan (M&E) that focuses on local outcomes (such e and tial tial as enhanced accountability, greater citizen archived participation, and improved service delivery) rather than on just the process of decentralization. Incomple 16.3 Ensure that Bank support at the country Medium Medium te and level is (among other things): archived 16.4 Strengthen institutional arrangements within the Bank to ensure that an Incomple integrative view underpins Bank Medium Medium te and interventions, particularly those based on archived sector-specific entry points. MANAGEMENT RESPONSE 2012 16.1 Ensure client commitment: Original Response: Ongoing/Agreed in Principle. Management agrees in principle with this recommendation; however, it notes that the degree of country consensus around appropriate decentralization modalities may differ by sectors and levels of government. Consequently, the Bank can often play a critical role helping countries identify and implement entry points intended to promote quality in decentralized service delivery. However, a number of cases suggest that political drivers for decentralization are often an important country dynamic, and Bank country strategy design and implementation must be responsive to these realities. Management agrees to continue to emphasize cross-cutting client commitment for decentralized service delivery as a critical factor in defining entry points for successful engagement, and to calibrate assistance to support sustainable service delivery and poverty reduction as good practice in relevant Country Assistance Strategy (CAS) design and implementation. To implement the Governance and Anti-Corruption (GAC) Strategy, the Bank‘s Regional vice presidential units have identified 26 countries that are initiating country specific GAC processes. Where country conditions involve significant issues of decentralized service delivery (for example, the Democratic Republic of Congo, Indonesia, and the Philippines), country teams are systematically deepening the analysis around political- economy circumstances in shaping development effectiveness. At the conclusion of this learning process, Bank management is committed to reporting to the Board whether and how it intends to systematize and scale up its GAC work, including analytic and advisory activities. Reporting on the above agreed actions will be done in the context of overall GAC reporting. 162 MAR 2012: A recent evaluation (January 2012) conducted by PREM underscores that investment lending operations with a subnational focus (defined either by ―sector‖ or by ―theme‖ in the business warehouse) are increasingly managed by different networks. The updated Public Sector Management Approach (2011-2020) has underscored actions for a shared understanding of the public sector, through professional communities, common training and analytical frameworks. With this background, PREM has taken initial steps to re-launch its Thematic Group related to decentralization and it will seek systematic outreach to staff in other networks. This strategy is consistent with the updated GAC principles and recommendations.  In the meantime, and consistent with the MAR 2011 response, risks of poor ownership continue to be systematically captured by ORAF. All projects approved in FY11 (with a PCN after July 1, 2010) have 100% compliance with regards to the application of the ORAF framework. In July 2011, OPCS has also issued Guidelines for the application of ORAF and section 2.2 specifically refers to the risk of poorly defined responsibilities among levels of government. The clarity of roles can highlight issues of low client commitment. However, the guiding questions are not of mandatory application on behalf of task teams.  The accuracy of this profile is improving as ORAFs are completed for all new operations and extended progressively to operations under implementation. At the end of FY11, only about 28% of the active portfolio have an ORAF in place, but by end-FY12 it is planned that there will be full ORAF coverage for all new IL operations, and for the IL portfolio under implementation. The new Quarterly Business and Risk Review (QBRR) will allow continuous monitoring against outcome targets. Recent decisions by the Board also allow for greater ownership to materialize for DPL operations. Development policy loans, credits and IDA grants can now be extended to a wider range of political subdivisions, according to a March 11, 2011 decision by the Bank‘s Executive Directors. Such operations ―may now be provided not only to states and provinces but also to such entities as federal districts, national territories, cities and municipalities—as long as the entity has appropriate budget authority and legislative competence.‖ 16.2 Adoption of more result-based approach: Original Response: Ongoing/Agreed. The Bank will continue to place particular emphasis in its dialogue with client countries on supporting the existence of a credible data-collection and reporting system for relevant services, one that is consistent with the structure of decentralization both in terms of its level of aggregation and management responsibilities and that can be used to make midcourse corrections as needed. At the same time, the design of M&E systems needs to highlight the presumed results chains between intermediate outcomes and ultimate outcomes (service delivery). For operations supporting a broad-based engagement in decentralized outcomes, countries will be encouraged to adopt a system that reflects the M&E system in place at the national level. At the national level, the Public Expenditure and Financial Accountability (PEFA) 8 indicators (concerning intergovernmental fiscal relations) and the PEFA 23 indicators (concerning front-line school and health facility financing) provide for systematic assessments of intermediate outcomes. The PEFA 163 Secretariat also recently issued guidance on subnational government applications of this public financial management diagnostic tool, useful to countries to consider for M&E at this level. As part of the GAC Implementation Plan, management has committed to strengthen the application of Actionable Governance Indicators, including those in the area of decentralization. Management will continue its efforts to provide country and individual operations teams with guidance on helping countries strengthen M&E around decentralization. MAR 2012:  Regarding subnational government and decentralization indicators, PREM is currently discussing with other partners (including IMF and OECD) the possibility of a multi- stakeholder effort to improve these indicators. The widely used GFS data produced by the IMF do not discern by levels of autonomy of subnational governments both on the revenue and expenditure side. This undermines efforts not only to assess the degree of decentralization but also its outcomes.  Under the Updated GAC Strategy and Implementation Plan, renewed efforts are currently underway to identify additional indicators of the strength of country systems (ISCS), that will complement the existing Actionable Governance Indicators (AGI) and provide Bank practitioners with a more systematic and comprehensive coverage of countries and topics for monitoring purposes. At the country level, the Bank will explore—with other partners—the feasibility of developing indicators of the strength of country systems (ISCS) by following an approach similar to that used for developing the PEFA indicators. OPCS is currently drafting a guidance note on how M&E country systems can be strengthened.  In line with these policies and strategies, the Program for Results instrument seeks a focus on results. It has been successfully applied in a few subnational governments in several regions (Minas Gerais and Ceara in Brazil, Punjab in Pakistan); in these cases, the new approach provided enhanced focus on strengthening associated provincial systems (e.g., fiduciary, safeguards, and monitoring and evaluation, or M&E) through technical capacity building and implementation support. PEFA has increased its coverage and application on subnational levels of government. While previous year 55 SN assessments have been prepared, and 10 were in the pipeline, by September 2011 there were 69 SNG PEFA Reports completed up to the draft stage; four reports have commenced; and 17 reports were in the pipeline. . 16.3 Work to an integrative analytical framework and strengthening local capacity for country level development effectiveness: Original Response: Ongoing/Agreed in Part. Given the cross-cutting nature of decentralization, management agrees that an integrated approach is important as part of general and specialized economic and sector work, including attention to the political- economy of decentralization (for example, Public Expenditure Reviews and sectoral 164 diagnostics, including around GAC issues). Management agrees with the first part of this recommendation and will implement it and monitor progress in the context of the GAC work noted in its response to the first recommendation. Management agrees that support for subnational capacity can be a vital ingredient to strengthening service delivery outcomes. However, management notes that strengthening government capacity must be linked to ultimate service delivery outcomes and based on appropriate engagement models and Bank comparative advantage, especially in the presence of a large number of diverse subnational jurisdictions. Local capacity building cannot be limited to, for example, the supply of training, but depends on appropriate incentives. Experience shows that it must be demand driven to be effective. Although strengthening local capacity often represents an important element for effective decentralization, management does not commit to always supplying local capacity building as an element in the activities it supports. MAR 2012:  The Bank, through DeLoG (Decentralization and Local Governance—Donor Coordination Group), has actively contributed to the preparatory work for the aid effectiveness agenda in Busan 2011, regarding subnational governments.23 The outcome document ―Busan Partnership for Effective Development Co-operation‖24 underscores ―the important role that local and regional governments and their associations are playing in the delivery of aid and technical cooperation through decentralized development cooperation including the promotion of peer to peer learning and other cooperation methods‖. DeLoG and its members continue to analyzing and refining institutional strengthening modalities, for instance ―municipal contracting‖, as a means of combining financial incentives for institutional strengthening. The subnational government capacity and decentralization agenda entails case studies in different country settings, among them fragile states, post-conflict and middle income in order to draw relevant conclusions; these will be further discussed at DeLoG‘s 7th annual meeting in Berlin May 2012, where the Bank is actively engaged for the preparation of the event. 16.4 Strengthening WBG Institutional Coordination, especially around sectoral inputs: Original Response: Ongoing/Agreed. Management underscores the role of country management units and Regional vice presidential units (with support from multisectoral The Bank has also actively contributed to DeLoG‘s publication ―Busan and Beyond: 23 Localising Paris Principles for more Effective Support to Decentralisation and Local Governance Reforms.‖ 24The document can be found under: http://www.aideffectiveness.org/busanhlf4/images/stories/hlf4/OUTCOME_DOCU MENT_-_FINAL_EN.pdf. 165 communities of practice in the Regions) in ensuring that consistent approaches are implemented as part of the CASs and operational review processes. Management will ensure that the issue is raised early in relevant CAS discussions. At a Bank-wide level, the Decentralization and Sub-National Regional Economics Thematic Group will continue to serve as a platform to promote integrative approaches to strengthen results based decentralization engagements, working together with other related cross-cutting thematic groups, notably the Urban Economics, Finance, and Management Thematic Group. Within the Bank-wide initiative to revitalize and support communities of practice, management sees this as a cross-network area to prioritize. This work will facilitate coordination across networks/ sector teams at the Regional level. To support coherent approaches, the thematic group structure will offer senior facilitation and advisory services on a demand basis to country teams engaged in upstream CAS or project design issues. Specific steps to strengthen Bank engagement on issues of decentralization and local governance are under preparation under the guidance of senior operational management. MAR 2012:  At a Bank-wide level, the Decentralization and Sub-National Regional Economics Thematic Group (PREM) continues to serve as a platform for integrative approaches for decentralization engagements. As it is currently being re-branded into ―intergovernmental relations‖ it seeks further coordination and integration with other cross-cutting thematic groups, notably the Urban Economics, Finance, and Management Thematic Group; as well as the Demand for Good Governance (DFGG) cluster. These coordinated approaches will be reflected in the programmatic concept note that PREM is currently preparing and scheduling for February 2012. The Matrix Leadership Team is currently working across all of the networks and sectors to revitalize existing sector boards. A survey to all staff is being applied in early 2012 and its results will be taken into account for further action. A review of the Anchor responsibilities with regards to subnational government and decentralization undertaken in December 2011 showed full compatibility and complementarity.. IEG RESPONSE 2012 16.1, 16.2, and 16.3: No comments 16.4 IEG notes the programmatic concept note that PREM is currently preparing (scheduled for February 2012) as well as the efforts of the Matrix Leadership Team. In view of this, IEG accepts Management‘s rating of ‗Medium‘. 166 17. Environmental Sustainability – An Evaluation of World Bank Group Support LIST OF RECOMMENDATIONS 17.1 Increase the attention to environmental sustainability in the World Bank Group by ensuring that environmental issues enter fully into discussions of its strategic directions and Regional and country assistance programs.  Jointly reformulate and update the 2001 Environment Strategy to reflect new realities— including the increasingly important role of the private sector, technology transfer to developing countries, global public goods, and transnational environmental footprints—and emerging Bank Group corporate priorities.  Consider both medium- term (5–10 year) and longer- term (10–20 year) approaches to strengthening environmental sustainability at the Regional and national levels and incorporate short- term (3–5 year) environmental programs into country assistance and partnership strategies.  Identify opportunities for intra- Bank Group cooperation in helping clients address key national and global environmental challenges, including pollution reduction and long- term goals (up to 50 years) for greenhouse gas abatement and adaptation to climate change.  Work with development partners to help countries address environmental problems. Use Country Environmental Analyses (CEAs) and Strategic Environmental Assessments (SEAs) for this purpose at the national, policy, sectoral, and subnational levels. Treat institutional capacity building as a means rather than an end and link it to attainment of observable environmental outcomes. Give greater attention to improving the performance of projects that focus primarily on environmental policy and institutions.  Encourage the adoption and use of the Equator Principles as global environmental standards in private sector investments in the developing world and IFC Policy and Performance Standards on Environmental and Social Sustainability by multilateral development banks.  Continue to develop IFC‘s systems to improve accountability and transparency among Equator Principles signatories. Focus IFC Advisory Services and capacity building on Regions and sectors with low environmental performance, especially on Sub- Saharan Africa and the textile, food and beverage, tourism, and agriculture sectors, and continue supporting market transformation toward sustainability, emphasizing technology transfer and development in clean production, energy efficiency, and sustainable supply chain management.  Expand MIGA‘s environment- related technical assistance to clients. 17.2 Move to more cross-sectoral and spatially oriented approaches to environmental support and strengthen staff skills.  Be more proactive on environmental concerns, including adaptation to, as well as mitigation of, climate change, but not neglecting other local and global environmental priorities. Better integrate environmental, health, and labor issues under the Bank Group‘s sustainability agenda in the short and longer terms.  Give greater analytical and operational attention to addressing problems that cross national and regional boundaries as well as to increasingly serious environmental and carbon footprint concerns. In analytic work, increase emphasis on linkages between poverty and the environment. Strengthen collaboration on environmental health issues among those responsible for health, water supply and sanitation, energy, transport, urban development, and environment.  Strengthen staff skills in such areas as adaptation to climate change, carbon finance, and the ability to deliver environment-related investment and policy reform projects.  Improve IFC- Bank coordination on policy dialogue with governments to enhance structural reforms aimed at public- private partnerships in water, wastewater, and waste management, reuse, and recycling sectors, and ensure that industry views are present in the national and sectoral policy dialogues.  Stress the need for IFC and MIGA clients, especially financial intermediaries, to develop and implement solid environmental and social management systems, ensure that engineering and pollution control system design and community engagement is integrated in the early project stage, and use more independent environmental audits as part of project completion tests. In IFC‘s project selection and marketing, emphasize the potential for environmental benefits. In MIGA‘s engagement with projects, provide advice on environmental (and social) issues to help bring clients closer to industry best practices. 17.3 Improve the Bank Group’s ability to assess its support for the environment and to monitor and evaluate the results of its environment- related interventions.  Improve World Bank monitoring, evaluation, and reporting of environmental performance and results of lending operations. Give greater attention to improving baseline environmental assessments in IFC and MIGA— and measure more fully the aggregate effects of projects with large environmental impacts—for example, in energy and agribusiness. Work with partners such as UNDP and UNEP to help quantify progress toward the achievement of Millennium Development Goal 7 for environmental sustainability.  Improve the way the World Bank determines how much of its total financing has supported environmental improvement and revise preparation guidelines for Implementation Completion Reports (ICRs) to require a more systematic review of environmental dimensions and results. A mechanism to track the influence of Bank nonlending services on environment- related policies and institutions in client countries would also be desirable.  For environmentally sensitive IFC agriculture and forestry projects, especially in areas of high biodiversity, undertake carefully designed baseline studies to identify indirect, induced, and cumulative (as well as direct) environmental and social impacts. Design, implement, and monitor adequate plans to mitigate any negative effects. Enhance sustainability of supply chains with certification schemes and third- party monitoring. Measure specific emissions and mass flows in advance of relevant projects and assess them afterward to gauge project impact on the abatement of effluent discharges and dust and greenhouse gas emissions.  Improve the performance of projects on MIGA‘s environmental and social policies on a timely basis, as appropriate in a project cycle. Require investor-clients to establish environmental and social project management systems at a sufficiently early stage to 168 effectively monitor impacts. Consistently incorporate provisions for regular reporting of safeguard performance during project implementation in MIGA‘s Contracts of Guarantee. 17.4 Improve coordination among the Bank, IFC, and MIGA and between the World Bank Group and external partners (public and private) in relation to the Bank Group’s environmental mission and ensure consistent and effective implementation at the corporate and country levels.  Establish mechanisms to promote and monitor coordination across the Bank, IFC, and MIGA with respect to environment-related policies, strategies, and instruments. In particular: – Actively involve IFC and MIGA in updating the 2001 Environment Strategy and in monitoring and evaluating its implementation. – Jointly identify environmental aspects of World Bank Group country assistance and partnership strategies and jointly plan, monitor, evaluate, and report on mitigation of adverse impacts. – Increase efforts to share experience with assessment, monitoring, evaluation, and reporting on environmental aspects, results, and impacts of activities. – Systematically monitor and evaluate the application and results of environmental due diligence policies and procedures (safeguards and performance standards).  Make strengthening external partnerships a central theme in an updated World Bank Group environmental strategy.  Improve MIGA‘s coordination with global programs, such as the Global Environment Facility and the Bank‘s Carbon Financing Group, and identify potential partners whose clients might benefit from MIGA guarantee support. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 17.1 Increase the attention to environmental sustainability in the World Bank Group Compl Comple by ensuring that environmental issues ete and te and High High enter fully into discussions of its strategic archive archive directions and Regional and country d d assistance programs. Compl Comple 17.2 Move to more cross-sectoral and spatially Substan Substan ete and te and oriented approaches to environmental tial tial archive archive support and strengthen staff skills. d d 169 Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG Incomp 17.3 Improve the Bank Group’s ability to Incompl lete assess its support for the environment and Mediu Mediu ete and and to monitor and evaluate the results of its m m archive archive environment- related interventions. d d 17.4 Improve coordination among the Bank, IFC, and MIGA and between the World Bank Group and external partners (public Compl Comple and private) in relation to the Bank Substan Substan ete and te and Group’s environmental mission and tial tial archive archive ensure consistent and effective d d implementation at the corporate and country levels. MANAGEMENT RESPONSE 2012 With respect to the specific IEG recommendations and agreed World Bank actions, progress in several areas has been made in the past year, as outlined below. The following responses refer to World Bank actions. IFC and MIGA responses are covered in their respective Management Action Record processes with IEG. 17.1 Attention to Environmental Sustainability. As noted in the 2010 Management Response, the Bank has committed to promote environmental sustainability through a new WBG Environment Strategy. The draft Strategy has undergone various levels of corporate review within the WBG and was endorsed by CODE in January 2012. The Strategy lays out a vision for a ―Green, Clean and Resilient World for All‖. The Strategy recognizes the centrality of growth but also that income-producing opportunities are pursued in ways that do not limit or close off opportunities for future generations. The Strategy responds to the call by WBG stakeholders, for a different kind of development pathway, one that enables growth while retaining a focus on sustainability and ensuring that the environment is a key enabler for growth.  A total of 13 background papers that addressed various topics related to environmental sustainability were completed to underpin the new Strategy. The topics covered included biodiversity, environmental institutions and governance, pollution management, ecosystems valuation, financing, gender, and country systems. Three additional papers that explicitly examine environmental sustainability were also completed on Lessons from Environmental Mainstreaming: Towards Sustainability; Monitoring Environmental Sustainability: Trends, Challenges and the Way Forward; and the Role of Environmental DPLs in Supporting Environmental Sustainability. Another background paper Assessing the Environment Co-benefits of Climate Actions helps further the environmental sustainability agenda from a cross-sectoral climate change lens. Most of these papers are available on Environment Strategy website www.worldbank.org/environmentstrategy. 170  The World Bank has launched a global partnership, Wealth Accounting and Valuation of Ecosystem Services (WAVES), to promote the implementation of comprehensive wealth accounting that focuses on the value of natural capital and integration of ―green accounting‖ in more conventional development planning analysis. WAVES will enable more informed decision making - targeting Ministries of Finance and Planning and Central Banks - to support sustainable development and green growth trajectories.  In the 2010 Management Response, Management did not commit to formally establish new medium-term and long-term approaches, citing the need for flexibility, given the importance of customization and country ownership. Instead, to strengthen intra-Bank collaboration and knowledge sharing, a Climate Change Knowledge Forum has been established to strengthen the exchange of information and coordination of activities. This Forum includes representatives from regions and sectors and IFC that are working on climate change. Recent analysis also reveals that climate change has been integrated into approximately 90 percent of all new Country Assistance/Partnership Strategies in FY10 . Furthermore, the INFRA (SDN specifics for supporting financial crisis investments) platform places an emphasis on sustaining funding of operations and maintenance of current infrastructure, which is of environmental and social benefit, placing sustainability squarely at the core of financial crisis related financing.  The World Bank continues to work closely with development partners to help countries address environmental problems. Country Environmental Analysis (CEAs) has been an important instrument in linking environmental priorities to growth and development priorities. By the end of FY 11, 34 CEAs have been completed to inform the Bank and client-country dialogue in environment.  As noted in the 2011 Management Response, with respect to Strategic Environmental Assessments (SEAs), their use, particularly that of policy SEA, has continued increasing notably in Africa and in forestry, mining and transport sectors. A policy SEA approach consistent with O.P. 4.01 is being used by the Bank for the Forest Carbon Partnership Facility (FCPF). This approach has evolved into a common approach designed to provide the World Bank and other FCPF delivery partners with a common platform for risk management and quality assurance in the REDD+ Readiness Preparation process, using the safeguard policies of the World Bank as a minimum acceptable standard. In addition, a SEA community of practice with a defined learning agenda for FY 12 has been established to enhance the Regions SEA capacity.  The Global Partnership for Oceans, launched by World Bank President Zoellick in February 2012, proposes as one of its building blocks, the development of operational guidelines. The first of these guidelines will address how ocean issues can be integrated in CEA and SEA to inform country assistance strategies, policy and sector reform.  The World Bank, through the Development Grant Facility, is working with partners to design a formal mechanism on an international scale to deal with toxic legacy pollution and its health effects in low and middle income countries and assist local communities to protect and improve their health and livelihood. 17.2 Cross-sectoral and Spatial Issues. As noted in the 2010 Management Response, Management agreed with parts of this recommendation but did not commit to new actions, noting that work was already ongoing on these issues. 171  Knowledge, strategies, and lending activities have increasingly addressed environment and natural resources issues in some sectors. Sectors most actively integrating environment into programs and projects include water supply and sanitation; energy mining and transport; and agriculture and rural development. With respect to environmental health, the Bank is more actively helping countries address environment- related health issues, including water supply, sanitation and hygiene, indoor air pollution, and urban air quality – although the current coding system precludes ready determination of the number of such projects. Furthermore, as noted in the 2010 Management Response, measurement of health impacts, for example in the water sector, is a challenge as many water and sanitation projects that integrate improved health and environmental practices are still economically justified and labeled as traditional ―increased access to water‖ projects. An important aspect of the upcoming WBG Environment Strategy will be to ensure that sectors across the WBG keep to their environmental commitments, which are compiled and summarized in the draft Environment Strategy. Consistent with this objective, the Results Framework for the new Strategy proposes project level core indicators for measuring and tracking environmental sustainability in sector operations.  In the area of climate change, a number of initiatives have been undertaken to foster cross-sector collaboration and spatial integration. The on-going analytical work to develop methodologies for GHG analysis of World Bank lending operations in the forestry, transport and energy sectors has involved collaboration between ENV and these groups as well as external IFIs engaged in this area. A ESW report by the Environment Department on Cookstoves, Environment, Health and Climate Change involved collaboration with staff working on energy, agriculture / forestry, and environmental health. The climate change knowledge portal housed within the Environment Department is a one-stop shop for information on climate impacts and has been a joint initiative with DEC, SDN, GFDRR and the regions (http://www.worldbank.org/climateportal/); and the Bank‘s operational regions (http://www.worldbank.org/climateportal/). More recently, the Climate Finance Options Platform – a joint UNDP-WBG knowledge platform (http://www.climatefinanceoptions.org) – is another example of a knowledge brokering service that targets a wide range of stakeholders across the global development community and provides access to information on more than 60 sources of climate finance.  Trust funds managed by sectors, such as ESMAP, BNWPP and co-financing by GEF, carbon facilities and CIFs have contributed to the observed progress on incorporating environment in sectors. The new WBG Environment Strategy will seek to build on the progress achieved, by continued cross-sectoral collaboration to improve environmental sustainability of WBG portfolio. As part of the Strategy preparation process, internal consultations were held with Anchor and regional environment staff, staff of the Bank‘s Legal Department, SDN Sector Boards, and Sector and some Regional Management teams.  Trust funds (including BNPP, TFESSD and DGF) have continued to support ‖traditional‖ environmental priorities, incorporating vulnerability to climatic and environmental 172 shocks and crises as new concerns. In addition, trust funds such as the GAPTF have addressed gender-environment linkages. The GEF program remains an important driver and complement to Bank Finance for regional programs and has helped attract regional IDA funds for global environmental issues. GEF also remains an important tool for testing and innovation to tackle environmental challenges and leverage subsequent scale- up of successful programs with other funds. The WBG GEF Program also integrates anchor, regional, and IFC coordination functions across a shared WBG program.  As noted in the 2011 Management Response, spatial approaches have also been taken in analytical work such as the Economics of Adaptation to Climate Change study, which estimated the cost of adaptation to climate change for developing countries. In addition to focusing on specific sectors, the study examined risks and macro-economic impacts of climate change across sectors and in spaces such as coastal zones. Also, spatial and cross sectoral analyses are some of the main approaches used by SEAs e.g., the West Africa Minerals Sector Strategic Assessment and the Mozambique Strategic Environmental and Social Assessments for the Spatial Development Planning Technical Assistance Project.  Global Partnership for Oceans will also pursue objectives of integrated coastal zone management which incorporates marine spatial planning.  IFC, World Bank and MIGA continued their cooperation through the preparation of the Pollution Management Sourcebook, now available online for comment. The PoMa Thematic Group (PoMa TG) first established as part of this effort, had an important role in shaping the guidance, and evolved into a community of practice on pollution management. The PoMa TG helps bring pollution management professionals together and is used as a medium to share information and solicit advice / expert recommendations.  Lastly, the Sustainable Development Leadership Program (SDLP) and the Climate Change for Development Professionals (CCDP) learning programs are under implementation. To date, over 350 senior staff members have participated in the SDLP. The CCDP program has been rapidly expanding over the past three years, and now has a set of e-learning modules that include Fundamentals of Climate Change, Economics of Climate Change and Social Dimensions of climate change. Recognizing that these can serve as a resource independent of face-to-face events, they are now available in the Learning Management System (LMS). By the end of FY12, we will add Climate Finance and Economics of Adaptation to Climate Change. All of these will also be linked to the Climate Change Knowledge Portal. ―Emerging Global Issues – Climate Change and Green Growth‖ has been launched and is bringing distinguished global scholars to the World Bank to present their views on timely and controversial issues related to climate change, green growth and environmental sustainability. 17.3 Assessment. As noted in the 2011 Management Response, work is underway in the area of improving the monitoring, evaluation, and reporting of environmental performance and results of lending operations. Core biodiversity indicators have been issued by OPCS. The Environment Strategy includes project and country-level result indicators of environmental sustainability which will be complemented during implementation of the strategy in collaboration with sectors. The project-level indicators will help to significantly improve 173 monitoring and measurement of results of environment-related interventions.  Work is underway to strengthen portfolio monitoring and tracking systems to allow the Bank to track financial flows (IBRD and IDA) with climate co-benefits. This is a joint initiative between ENV, SDN sector and regional colleagues, and OPCS and will be rolled out in the Bank in FY13.  The proposed Global Partnership for Oceans has a focus on results which will be tracked Bank-wide. In support of this, a proposed knowledge portal which would work with key ocean monitoring organizations such aas UNESCO/IOC, NOAA, GRIDA-Norway, and FAO would further track the state of oceans and help determine the Bank contribution and impact towards ocean sustainability. 17.4 Coordination. As noted in the original Management Response, Management agreed with most of the recommendations but not all. Management committed to actively involve IFC and MIGA in the Environment Strategy preparation and continues to meet that commitment. The preparation of the new Environment Strategy has been a collaborative effort, with active involvement of the Bank, IFC and MIGA. The consultation process on the Strategy has been extensive, reaching out to different external partners - including government, private sector, development partners, donors, civil society, academia, NGOs - to better define the Bank‘s role, as well as to ensure harmonization and coordination on specific themes. A total of 66 external consultation sessions were conducted with more than 2300 stakeholders from over 126 countries. Stakeholders also had the opportunity to provide their views and follow the Strategy development process through a website (www.worldbank.org/environmentstrategy). In addition, as part of the preparation of the WBG Environment Strategy, a number of analytical papers, including on pollution management, environmental mainstreaming and environmental policies, were prepared in collaboration with IFC. Furthermore, preparation of several background papers that underpin the Strategy involved consultation with external stakeholders. IFC and the Bank‘s regional environmental units contributed to drafting significant parts of the draft Strategy document and MIGA provided valuable inputs. Key sectors with environmental footprints were engaged in discussions aimed at improving the environmental sustainability of the WBG portfolio, and provided inputs, summarized in the Strategy, on their commitments to integrate environmental sustainability in sector operations (e.g. ICT, Infrastructure, Agriculture and Rural Development, Water, Energy, Transport, Social Development, Manufacturing, Mining, and Oil and Gas Policy)  The Bank has continued harmonizing SEA approaches with donors through its active participation in the OECD DAC SEA Task Team and widely disseminating its work on policy SEA through different fora ranging from presentations in international conferences to submissions of papers to academic journals. The Bank has also contributed to the OECD DAC Guidance on ―Enhancing Capacity for Environmental Management and Governance‖ and continued coordination and sharing of experiences in the context of the Multilateral Financial Institutions Working Group on Environment.  The Bank has also established a partnership with UNDP in the form of a joint Climate Finance Options knowledge platform, launched in 2010 under the theme "Acting on 174 Climate Change: the UN Delivering as One". Through the Issue Management Group on Green Economy, established under the UN Environment Management Group in 2010, the Bank took the lead in two chapters and contributed substantively to several other ones for the report "Working towards a Balanced and Inclusive Green Economy: A United Nations System-wide Perspective", published in December 2011.  ―Environment and Social Policy and Procedural Guidelines for projects financed jointly by Bank, IFC, and/or MIGA‖ were finalized in 2009. The 2010 Management Response noted that other issues were dependent on the country context and that monitoring and evaluation can only take place within the project context, rather than at a broader level, so that the sovereignty principle is respected. ENV has continued supporting GEF Focal Areas programmatic-level results reporting. Often these programmatic results frameworks assist the funding agency more than the individual projects, as the results reporting demands aggregating projects-to-program rather than projects-to-country. However, the programmatic exercises are expanding aggregating methodologies in multi- sectoral areas (like nutrient pollution reduction), which can then lead to improved indicators for country-level aggregation. See examples in Georgia, Romania, and Turkey. Programmatic interventions have also been associated with demand for subsequent scale- up loans at the national levels.  The Wealth Accounting and Valuation of Ecosystem Services (WAVES) initiative has been set up as a global partnership to foster coordination with external partners. WAVES provides a broad platform including the United Nations Environment Programme (UNEP), United Nations Development Programme (UNDP), other UN agencies, developed and developing nations, international organizations, NGOs and academics. Partner countries currently include: (i) Developing country partners: Botswana, Colombia, Costa Rica, Madagascar, and the Philippines; (ii) Developed country partners: Australia, Canada, Japan, Norway, and the United Kingdom; and (iii) the other countries doing environmental accounting have been invited to join as well.  The Global Partnership for Oceans is being designed by a WBG-wide ―Blue Team‖ including staff from the six regions, as well as ENV, ARD, IFC, TWITR, and SEGOM. The Partnership aims to bring small island states and coastal developing nations together with some of the world‘s premier ocean-focused organizations and bi-lateral country partners interested in pairing cutting-edge knowledge with finance to support workable and scalable solutions that sustainably enhance the natural capital of oceans. To this end, Development Grant Facility funds were approved by the DGF Council in March 2012. The first meeting of partners will take place in April with next public announcement of financial commitments, strategy, and targets to be announced at the Rio+20 UNCSD meeting.  On the Natural Resources Management side, ENV is leading the development of a community of practice to leverage in-house expertise to combat environmental crime and foster natural resource integrity (environmental crime here is defined broadly to include both resources leaving unlawfully and resources entering without permission). This community of practice includes members from the anti-money laundering, institutional integrity as well as ENV, ARD and regional staff.  With respect to safeguards, the Bank is making efforts to systematically monitor 175 environmental due diligence efforts. To this end, East Asia and the Pacific and Latin America and The Caribbean Regions have put in place Safeguard Focal Points that monitor due diligence efforts. Secondly, the Bank is taking IFC‘s Performance Standards into account in the ongoing update and consolidation of safeguards policies. It is important to note that, while performance of environmental due diligence is the responsibility of sectors, responsibility for the assessment of safeguards compliance rests with OPCS. IEG RESPONSE 2012 17.1 With the endorsement of the new Environment Strategy, IEG rates compliance with this recommendation as "High". The new Strategy makes a strong case for action across the range of environmental concerns and proposes an ambitious program centered on knowledge and analysis. the . The Bank has meanwhile continued to support Country Environmental Assessments. 17.2 Cross-sectoral and spatial approaches are increasingly used in analytic work, including in the Bank‘s urban work. The continued use of CEAs and SEAs have a role for cross sectoral and spatial analysis, and there is increased use of spatial hydrological modeling for water resources. The Mozambique Spatial Development Planning Technical assistance project is an example of mainstreaming of spatial development. IEG notes the increasing availability of spatial data on the Climate Portal, the mapping of active projects by Mapping for Results, and increased emphasis on the geocoding of project locations. However, cross-sectoral work on investments still remains difficult to undertake in the Bank's siloed structure, as IEG‘s Matrix Evaluation notes. The new Environment Strategy advocates cross-sectoral work, but does not spell out an approach to facilitating it. The fact that the Bank's project coding system is unable to identify environmental health investments is illustrative of the barriers. IEG acknowledges the continued commitment to the Sustainable Development Leadership Program, in which a significant proportion of senior staff have participated. 17.3 The new environment strategy acknowledges the shortcomings of current environmental monitoring and evaluation efforts and pledges to update them. (more on value of M&E) IEG applauds this intention but concurs with Management that the current state of adoption is Medium. 17.4 The new Environment Strategy is a Group-wide strategy prepared with IFC and MIGA collaboration, though it does not spell out in detail mechanisms to take advantage of cross- organization synergies. However, an IEG review of recently completed Country Assistance Strategies shows an increase in cross-organization collaboration. Among 23 CASes that begin in FY06 or before, 23 had some kind of environment activity, but only two had activities by both IFC and the World Bank in the context of a joint CAS. For post FY06 CASes, the 176 proportion jumps to 7 out of 15 CASes with environmental content. The Strategy places heavy emphasis on external partnerships, some of which are yet to be realized. The Pollution Management Sourcebook represents an achievement in collaboration among the WBG members. However, one important part of the recommendation lags. As noted in IEG's Safeguards Evaluation, the Bank (unlike IFC) does not systematically monitor project compliance with safeguard plans. 177 18. Climate Change and the World Bank Group Phase I—An Evaluation of World Bank Win-Win Energy Policy Reforms LIST OF RECOMMENDATIONS 18.1 Systematically promote the removal of energy subsidies, easing social and political economy concerns by providing technical assistance and policy advice to help reforming client countries find effective solutions, and analytical work demonstrating the cost and distributional impact of removal of such subsidies and of building effective, broad- based safety nets.  Energy price reform, never easy or painless, can pose social and political economy risks in client countries. But the Bank can help provoke and promote reforms by providing clients with assistance in charting and financing adjustment paths that are politically, socially, and environmentally sustainable.  One way to do this is for the Bank to continue to develop and share knowledge on the use of cash transfer systems or other social protection programs as potentially superior alternatives to fuel subsidies in assisting the poor. This would include systematic analyses of the distributional impact of energy subsidies. Timely monitoring and analysis of energy use and expenditure, at the household and firm levels, will also be important in policy design, in securing public support, and in detecting and repairing holes in the safety net. 18.2 Emphasize policies that induce improvement in energy efficiency as a way of reducing the burden of transition to market- based energy prices.  Cost- reflective prices for energy boost the returns to efficiency, but the Bank should support country policies that allow households and firms to exploit efficiency opportunities. Conversely, the deployment of energy- efficient equipment such as compact fluorescent lights can be used as a device for cushioning the impact of price increases. The Bank should explore innovative ways to finance efficiency (and renewable energy) investments in the face of fuel price volatility.  In order to strengthen internal incentives toward promotion of energy efficiency, the Bank should develop appropriate metrics, such as indicators that more directly reflect energy savings, instead of dollar growth targets in lending for energy efficiency (which may distort effort away from the high- leverage, low- cost interventions). These indicators, in turn, need to be harnessed to country strategies and project decisions. All of these efforts are likely to call for increased funding for preparation, policy dialogue, analysis, and technical assistance rather than lending. 18.3 Promote a systems approach by providing incentives to address climate change issues through cross- sectoral approaches, teams at the country level, and structured interaction between the Energy and Environment Sector Boards.  Helping clients reform will require a systems view, such as looking at the power system as a whole; looking at energy subsidies as just one, undesirable, part of a social protection system; and looking at the connections between water and power management.  To be effective the Bank needs to break down sectoral silos and encourage cross- sector approaches and teams. This will require championship by country directors and vice presidents, to promote incentives such as supporting capacity building for power system regulators in integrated resource planning, and using the Clean Technology Fund to support public systems that will catalyze widespread investments.  Structured interaction of the Energy and Environment Sector Boards, initiated with ad hoc groups to address specific crosssectoral challenges, could move the Bank closer toward mainstreaming sustainable development. 18.4 Invest more in improving metrics and monitoring for motivation and learning at the global, country, and project levels.  Good information can motivate and guide action. One particularly useful global initiative for the World Bank would be to collaborate with the International Energy Agency or other partners to set up an Energy Scorecard that would compile up- to- date and regular standardized information on efficiency indicators, energy prices, policies, and subsidies at the national and sectoral levels. Indicators could be used by borrowers for benchmarking; in the design and implementation of country strategies, including sectoral and cross- sectoral policies; and in assessing Bank performance in assisting countries.  At the national level, the Bank should support integration of household and firm surveys with energy consumption and access information to lay the foundation for assessing impacts of price rises and mitigatory measures, as well as planning for improved access.  At the project level, the Bank should invest in rapid- feedback monitoring and impact evaluation of efficiency projects and policies. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 18.1 Systematically promote the removal of energy subsidies, easing social and political economy concerns by providing technical assistance and Comple Comple policy advice to help reforming client Substanti Substanti te and te and countries find effective solutions, and al al archive archive analytical work demonstrating the cost d d and distributional impact of removal of such subsidies and of building effective, broad- based safety nets. 179 Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 18.2 Emphasize policies that induce Comple Comple improvement in energy efficiency as a Substanti Substanti te and te and way of reducing the burden of al al archive archive transition to market- based energy d d prices. 18.3 Promote a systems approach by providing incentives to address climate Comple Comple change issues through cross-sectoral Substanti Substanti te and te and approaches, teams at the country level, al al archive archive and structured interaction between the d d Energy and Environment Sector Boards. 18.4 Invest more in improving metrics and Incompl Incompl monitoring for motivation and learning Substanti ete and ete and Medium at the global, country, and project al archive archive levels. d d MANAGEMENT RESPONSE 2012 With respect to the specific IEG recommendations and agreed Bank actions, progress in several areas has been made in the past year, as outlined below. 18.1 Promote the Removal of Energy Subsidies. Original response Agreed; work is already ongoing. The Bank continues to work with client countries to address the issue of energy subsidies. Technical assistance and policy advice are provided, as requested by our client countries. The Bank focuses on the legal and regulatory mechanisms needed to support sustainable energy pricing reforms. Energy staff will continue to work with Poverty Reduction and Economic Management Network and Human Development Network staff (for example, Guidance for Responses from the Human Development Sectors to Rising Food and Fuel Prices, World Bank HDN 2008) to develop and apply social safety nets, including cash transfers, designed to protect the poor from the impact of energy price adjustments. A regulatory thematic group has been established in the Bank to foster dissemination of lessons learned. These lessons will be applied, taking into account the unique circumstances in client countries. When requested, the Bank provides support to enable countries to monitor and analyze energy use so that findings can be applied to their energy policies. MAR 2012 The Bank is playing a major role in promoting the removal of energy subsidies, both in the context of high level initiatives such as the G20, as well as by providing technical assistance and policy advice through its lending and non-lending activities. In the G20 180 context, the mandate came from the Pittsburgh Communiqué that, building on the efforts of many countries to ―reduc[e] fossil fuel subsidies while preventing adverse impact on the poorest‖, committed to ―rationalize and phase out over the medium term inefficient fossil fuel subsidies that encourage wasteful consumption‖. G20 Leaders requested ―IEA, OPEC, OECD, and World Bank to provide an analysis of the scope of energy subsidies and suggestions for the implementation of this initiative and report back at the next summit in June 2010. Three Joint Reports were produced by the four organizations. The Bank played a catalytic role in initiating and coordinating the process. The first Joint Report was featured at the very high level and presented by the President in the context of the Spring Meetings and the G20 Leader Summit which was held in Toronto in June 2010. Our joint reports were ―noted with appreciation‖ and ―welcomed‖ rather than "discussed" as other reports which were submitted and the priority actions to phase out fossil fuel subsidies by some important G20 countries, including India and Indonesia. Through the Joint Report and the Background Papers the Bank adopted a systematic approach for analysis and monitoring of aggregate and disaggregated data of producer and consumer subsidies and of relevant case studies. The Bank‘s Background Papers have been coordinated with PREM, HDN, and DEC. The Bank presentations in the context of several G20 Energy Expert Workshops on Fossil Fuel Subsidies (the most recent ones taking place in April 2011 in Washington DC and in September 2011 in Istanbul) and in APEC High Level meetings were very well received by the countries who also committed to use such analytical tools in their efforts to phase out subsidies. The Bank continues to work with client countries to address the issue of energy subsidies. Technical assistance and policy advice are provided, as requested by our client countries. The Bank‘s support is focused on the design of legal and regulatory mechanisms needed to support sustainable energy pricing and subsidy reforms, as well as on the effective implementation of those reforms following politically and socially acceptable paths and a detailed analysis of the distributional impact of energy subsidies. The Bank is currently providing technical assistance to several countries in the Middle East, including Egypt and West Bank and Gaza, based on a sound analysis of the distributional impact of subsidies and their removal. The Bank has provided technical assistance to Vietnam in the design and implementation of a new pricing system for electricity in the country, based on an average tariff level reflecting total costs of efficient service provision and subsidy-free rates for all categories of consumers. The pricing system is complemented by a social safety net aimed at protecting exclusively low-income consumers. Another important case refers to the Bank‘s support to the Dominican Republic in the elimination of a regressive geographic subsidization scheme for electricity tariffs, and its effective replacement by the inclusion of electricity in an already existing social safety net (also designed with Bank‘s support) involving investing in welfare enhancing activities (such as health and education). The change was fully implemented in less than 18 months. In the case of Iran, the Bank documented the largest cash transfer scheme following subsidies reforms, covering 19 million families (90% of the population) The government then set up a fund to administer receipts from the higher-priced goods, demarcating 50% to go towards families, 30% to help businesses affected by price rises and 20% to meet the state‘s own added costs. 18.2 Promote Improvements in Energy Efficiency. 181 Original Response. Partially agreed; work is already ongoing. The Bank has established an Energy Efficiency for Sustainable Development program to help guide and scale up energy efficiency activities. It is implementing the first step of this program, to increase the staffing with energy-efficiency experience, in ESMAP, the Energy Anchor Unit, and the Regions. This effort is complemented by a learning program developed by the Bank‘s energy efficiency thematic group, under the oversight of the Energy and Mining Sector Board. Another step is the development of programs and projects at the country/policy level, the industry level, and the equipment level to ensure that a broad- based implementation program evolves. To foster World Bank Group support for energy efficiency, the draft ―Development and Climate Change: A Strategic Framework for the World Bank‖ (World Bank 2008) has proposed an initiative to screen the project pipeline for energy-efficiency potential early in the project design phase. The Bank is working with the donor community to: (i) increase the financial support needed to intensify energy-efficiency efforts; (ii) increase low- cost funding to support energy- efficiency and renewable energy programs; and (iii) broaden the support from partners in implementing a renewable energy and energy-efficiency program. In terms of internal incentives, the discussion on developing appropriate metrics has been ongoing with the International Energy Agency and with UN Energy, but to date it has been inconclusive. Given the inconclusive nature of the discussion to date, management is not prepared to agree with establishing new metrics that focus solely on energy efficiency. The World Bank Group has committed to accelerate lending for new renewable energy and energy efficiency to 30 percent per annum over the next three years, a 50 percent increase over the 2004 Bonn commitment (which it has consistently met since that time). MAR 2012 The Bank continues to expand its non-lending (policy, institutional, and advisory) and lending portfolio in the areas of supply-side and demand-side energy efficiency, supported by efforts to improve knowledge management, cross-sectoral collaboration and sharing of best practices across the energy and other practices. The WBG's renewable energy portfolio more than doubled from $2.9 billion in FY06-08 (18 percent of total WBG energy financing in the period) to $6.6 billion in FY09-11 (22 percent). Low carbon projects likewise more than doubled from $6.8 billion to $14.9 billion over the same period, and the share of total WBG energy financing for low carbon projects rose from 42 percent to 51 percent over this period. FY11 marked an all-time record in WBG low carbon financing, with $5.9 billion in new commitments. Compared to a target average annual increase of 30 percent in financing for new renewable energy (new RE) and energy efficiency (EE) to FY11, amounting to total financing of about $6 billion over FY09-11, the WBG's financing for new RE and EE actually increased by about 48 percent per annum, and totaled some $9.2 billion. Going forward, the WBG will no longer categorize hydropower projects based on the threshold size of 10 megawatts, a distinction that is not supported by technical evidence, and will consider hydropower generation of all sizes to be renewable. The Bank is closely engaged with the global initiative of ―Sustainable Energy for All‖ which was launched in 2011 under the auspices of the UN and which aims to ensure universal access to modern energy services, double the rate of improvement of energy efficiency and 182 double the renewable energy share in global energy mix by 2030. The Bank is represented in the High Level and Technical Groups as well as in the Task Forces. Recognizing the importance of the energy challenges, the UN General Assembly has declared 2012 the International Year of Sustainable Energy for All. Energy has been considered a central element of the ―green economy in the context of sustainable development and poverty eradication‖ also in the context of Rio + 20. The ramp up in energy efficiency lending over the last several fiscal years has been accompanied by a gradual increase in staffing. Endorsed by the Energy and Mining Sector Board, a new Energy Efficiency Community of Practice has been initiated to further enhance more effective collaboration and knowledge brokering within the Bank Group and with external knowledge and practitioner networks. As a part of this collaboration, a knowledge portal of energy efficiency and a communications platform to link both internal and external practitioners in energy efficiency are planned for 2012. In addition, partnerships with Urban, Water, ICT and Transport sectors, and with global programs like the GEF, Carbon Finance, Clean Investment Funds, WBI knowledge platforms, have helped strengthen the operational synergies and leveraged several innovative projects, programs and activities in the area of energy efficiency. Many AAA initiatives are underway within the Energy Practice that focus directly on mainstreaming the policy development and implementation experiences, which has also helped in expanding the development policy lending (DPL) initiatives focusing on energy efficiency (e.g., Poland DPL approved in FY11, Vietnam DPL under preparation, etc.). 18.3 A Systems Approach. Original Response: Partially agreed; work is already ongoing. The Bank will continue to use a system- wide approach in reviewing projects and programs. Most Regions and many country teams have already created climate change teams of staff from several sectors to promote synergies, and are developing cross- sectoral business strategies to integrate climate change considerations. The World Bank Group established a Climate Change Management Group as a focal point to discuss cross- sectoral issues and promote synergies. The Bank supports regulatory capacity building, drawing on lessons learned from successful cases accomplished to date. On the basis of previous experience, management disagrees with the proposed use of integrated resource planning, as it is unconvinced of the effectiveness of the use of integrated resource planning by either supply- side entities or their regulators. However, the Bank supports the use of broad- based planning tools by policy makers to support the implementation of policies in the legal and regulatory framework. The Bank is currently considering large- scale responses to demand- side issues using new funding for low- carbon technologies when the funds become available. The merging of infrastructure and environment into a common vice presidency has facilitated interaction at the sector boards and thematic working groups MAR 2012: The Bank is continuing to enhance collaboration across different departments and 183 organizations, in its effort to assist client countries in adopting a broader cross-sectoral approach to climate change. Strategic collaboration between the Energy and Water sectors, particularly on hydropower, is being pursued through the newly established Hydro Community of Practice. At the operational level, the Energy practice has been working with other practices, sectors and networks, and particularly the Environment practice, through specific platforms like the Climate Change Management Group (now recalled Climate Change Knowledge Forum), Climate Change Finance Working Group, Clean Technology Fund (CTF), Scaling Up Renewable Energy Program (SREP), Low carbon studies, Sustainable Cities Initiative, Energy Efficient Cities Initiative (EECI), etc. Through the support of CTF for instance, 14 country-level cross-sectoral investment plans and a regional concentrated solar power plan for selected countries in the MENA region (Algeria, Egypt, Jordan, Morocco, and Tunisia), were developed. Following completion of the 7 low carbon studies conducted in large emitting countries over 2008-2010 (China, India, Brazil, Indonesia, Mexico, Poland, South Africa), FY11 focused on expanding the low carbon development studies into additional countries (Nigeria, Macedonia, Morocco, Tunisia, etc). In 2011, the low carbon development planning work was further enhanced to enable consensus on policies and investments across a wider set of stakeholders through the deployment of tools developed by ESMAP such as EFFECT (Energy Forecasting Framework and Emissions Consensus Tool) model, MAC (Marginal Abatement Curve) Tool, and the TRACE tool (Tool for Rapid Assessment of City Energy) – these tools have been deployed in several client countries already, and their usage in other regions and countries is expanding with time. EECI has sought specifically to work cross-sectorally with urban, water and transport anchors and infrastructure operational units in the regions – examples include the joint work with the Water Anchor and the Water and Sanitation Program on A Primer on Energy Efficiency for Water and Wastewater Utilities, and the Shanghai Green Energy for Low Carbon City Project which adopts a multi-sector approach to integrate energy efficiency, clean energy technologies, and sustainable transport. Further, over US$ 11.4 billion of country-level DPL programs during FY 09-11 supported climate change policy and operational development in 15 countries including loans to Brazil, Mexico, Morocco, Indonesia and Turkey. Many of them are being developed and implemented jointly by Energy and Environment teams, and technical assistance with regard to low carbon development is being applied along with DPL and investment lending programs in countries like Tunisia, Morocco, Vietnam and Nigeria. Also supported by a new working group on ICT on advanced metering infrastructure (AMI) and ‗smart‘ grid development, the Bank is providing technical and financial support to electricity distribution companies in several countries in the design and implementation of projects aimed at improvement of commercial performance. The three main objectives of those projects are better customer service, reduction of non-technical losses (amounts of energy consumed by users but not billed by the service company) and increase in collection rates. The projects are based on the incorporation by the involved utilities of state-of-art information technology (IT) applications as tools to improve management and enhance transparency and corporate governance. A management information system (MIS) to support efficient execution of commercial processes and activities is one of these key applications. A second one is the use of advanced metering infrastructure (AMI) to remotely read and monitor consumption of selected large and medium electricity users, assuring they are billed according to their actual consumption on a permanent basis (revenue protection). 184 Several projects involving ICT are either under implementation or under preparation in electricity utilities in Brazil, Dominican Republic, Ghana, Haiti, Honduras, India, Malawi, Mozambique, Uzbekistan and Vietnam. In the Dominican Republic, an AMI-based revenue protection project designed and implemented by the three distribution companies in the country with technical assistance provided by the Bank resulted in the reduction of total losses by 4 percentage points in less than one year. Project investments (meters, communication devices and software package to remotely collect and analyze metering data) were paid back in 7 months. More importantly, the design of the project assures sustainability of the improvements achieved through its implementation. 18.4 Invest in Improving Metrics. Original Response: Partially agreed; work is already ongoing. The Bank has been working with the International Energy Agency on collecting energy-efficiency–related information in pilot countries for two years, with limited success. Management does not commit to the idea of establishing a centrally maintained Energy Scorecard. Rather, the focus of our efforts is now on helping client countries establish their capacity to undertake the data collection exercise in a manner that targets both effective implementation and related policy- making guidance. Without this capacity and country willingness to participate in and lead this initiative, it will not be sustained. The Bank is also looking into possible new, innovative knowledge- sharing mechanisms to facilitate sharing lessons learned. The Bank lacks the resources to maintain a comprehensive and reliable database on energy policies, prices, subsidies, and energy efficiency at the national level. Regional organizations provide part of this information, which the Bank selectively draws upon, depending on the information‘s reliability. The Bank, with ESMAP support, has led in improving Living Standards Measurement Survey (LSMS) instruments for increased collection of energy data as part of LSMS surveys. The Bank will include rapid- feedback and monitoring and impact evaluation of efficiency projects when requested by our borrowers. MAR 2012: An effort to collect standardized information on energy prices policy and subsidies for a representative sample of 20 countries and for a relatively long time series was launched by the Bank as part of the work underpinning the G20 Energy Agenda. To capture the key incentives and drivers for energy subsidy reforms, evidence was collected on the fiscal burden of energy subsidies, both directly in the budget and implicitly. The trend in fossil fuel (gasoline and diesel) prices and electricity prices was also collected complemented by the gasoline and diesel fuel consumption, as well as power consumption. Evidence from household surveys on the distributional impact of subsidies by income category, as well as the materiality of energy subsidies, reporting for different forms of energy how much income is spent by each income group and the welfare impact of removing energy subsidies (what proportion of poor households as a whole receive the subsidy) have been documented in several studies. Global studies on the political economy of energy subsidies are ongoing. The Bank has continued to draw from the work and analyses of the IEA, IAEA, APEC, World Energy Council, ADEME and others, for energy sector benchmarks and data, as required. 185 These sources are also regularly used in establishing targets under the clean investment plans for the CTF, low carbon country studies, and the analytical projections for potential improvements in energy efficiency associated with the country and regional level assessments (for instance, in Turkey and other ECA countries in 2011). An ESMAP international roundtable in 2010 for reviewing different practices and inter-linkages for evaluating energy efficiency performance at national, sectoral, activity, end-use and policy levels, showed the complexity of tracking indicators for energy efficiency and the significant resource requirements associated with it. The draft energy sector strategy sets as its two goals the expansion of access to reliable modern energy services and a shift to more sustainable energy sector development. Within this context, the Bank has also recently initiated several initiatives. A new activity is currently developing methodological approaches to understand the impact on GHG emission of our different interventions in the energy sector. The aim is to fill the existing knowledge gaps, pilot new approaches with clients, and engage with other IFIs to pursue harmonization. An ESMAP-funded activity on developing and implementing an approach for defining and measuring access to energy has also recently been initiated. This is a multi-stakeholder effort (along with UNDP, GIZ, NL Agency, DFID, Global Alliance for Clean Cookstoves (GACC), WHO, IEA, UNIDO etc.) aimed at developing a framework for measuring energy access, in terms of availability of energy for various applications across household, community and productive use requirements. The study will also include aspects such as quality, affordability and actual use of energy. IEG RESPONSE 2012 18.1 and 18.2 Promote the removal of energy subsidies; use energy efficiency to ease the transition to cost-reflective pricing of energy The World Bank has devoted increasingly high-profile attention to this sensitive issue, including analytic work assessing country experience on removing energy subsidies. The sequence of investment and policy loans, and technical assistance, in Vietnam, exemplifies the approach advocated by the Evaluation – namely, the use of demand side energy efficiency measures to ease the burden of electricity tariff reform. There remains much to do to realize the potential of this line of action, building on the analytic and political base that has been established. 18.3 IEG acknowledges increasing cross-sectoral analytic work related to climate change, and development policy lending with multisectoral content. However ‗stovepiping‘ still inhibits cross-sectoral investment work on energy efficiency. IEG‘s Matrix Evaluation describes the structural barriers to implementing cross-sectoral work. 18.4 The Bank‘s study on ―Implementing Energy Subsidy Reforms‖ shows the feasibility and utility of gathering information on energy prices and subsidies. Rather than a one-off study, this kind of effort should be made routine, given its importance for energy, climate, and fiscal policy, and in light of the upsurge of activity in development policy lending for energy efficiency. The initiative to assess the GHG impacts of World Bank energy projects is a valuable one, 186 and should be complemented by continued, rigorous monitoring of the GHG, economic, and other impacts of energy projects. 187 19. Using Knowledge to Improve Development Effectiveness – An Evaluation of World Bank Economic and Sector Work and Technical Assistance, 2000- 2006 LIST OF RECOMMENDATIONS 19.1 Reinvigorate the mandate—which underpinned the fiscal 1999 ESW reforms—for country teams to maintain a strong knowledge base on countries and sectors where the Bank is providing or planning to provide funds.  Bank country strategies and lending activities need to continue to be supported by requisite analysis, although a return to strictly defined ―core diagnostic‖ ESW is unnecessary. 19.2 Ensure ESW tasks in IDA countries are adequately resourced, even if it means fewer ESW in some countries.  This will help to address the lower level of resources for individual ESW tasks in IDA countries than those in International Bank for Reconstruction and Development countries and is supported by the finding that cost matters for quality and quality matters for effectiveness. Greater selectivity will also help to reduce the burden on limited government capacity in some IDA countries and will free resources for more collaboration and follow-up needed to enhance the effectiveness of ESW. Selectivity could be enhanced by giving priority to ESW that informs Bank lending and strategy or that is clearly desired and needed by the client. 19.3 Enhance the institutional arrangements for undertaking ESW and TA.  To the extent allowed by budget, ensure substantive task team presence in country offices, particularly in countries with low institutional capacity. This will facilitate closer collaboration with clients from task initiation through follow-up. In addition, formulate a dissemination and implementation strategy for ESW and TA at the concept paper stage. Such a strategy should identify the target audience, the mode of dissemination, and the follow-up arrangements after dissemination, all of which should be explicitly budgeted for as integral to the task. 19.4 Recognize and build on client preferences, whether for nonlending versus lending services or for TA versus ESW.  Institute a mechanism to obtain client feedback on a periodic basis on delivered ESW and TA products. Such feedback should include clients‘ views on collaboration, follow-up, and usefulness of the tasks (including specifics of how the tasks were used). The client feedback should be requested at a set period (for instance, around 1 year) after the delivery of the task to the client to allow time for follow-up, and it should be the last milestone for ESW and TA. Obtaining client feedback would encourage a stronger results focus for ESW and TA and would help counterbalance current Bank incentives for lending over nonlending and for ESW over TA. 19.5 Take the results tracking framework seriously, including by incorporating systematic client feedback, as noted in the above recommendation. STATUS OF IMPLEMENTATION Level of Adoption Status Rating Recommendations Mgmt IEG Mgmt IEG 19.1 Reinvigorate the mandate—which underpinned the fiscal 1999 ESW Comple Comple reforms—for country teams to maintain a Substa Substan ted and ted and strong knowledge base on countries and ntial tial archive archive sectors where the Bank is providing or d d planning to provide funds. Comple Comple 19.2 Ensure ESW tasks in IDA countries are Substa Substan ted and ted and adequately resourced, even if it means ntial tial archive archive fewer ESW in some countries. d d Comple Comple 19.3 Enhance the institutional arrangements for Substa Substan ted and ted and undertaking ESW and TA. ntial tial archive archive d d Incompl Incompl 19.4 Recognize and build on client preferences, Mediu Mediu ete and ete and whether for nonlending versus lending m m archive archive services or for TA versus ESW. d d Comple Comple 19.5 Take the results tracking framework Substa Substan ted and ted and seriously, including by incorporating ntial tial archive archive systematic client feedback, as noted in the d d above recommendation. MANAGEMENT RESPONSE 2012 Original Response 19.1 Agreed/Ongoing. Management agrees with the recommendation of maintaining a strong knowledge base in countries where the Bank is providing or planning to provide funds. This recommendation, which is supported by the empirical association between the existence of upstream ESW and the quality of loan design, is particularly relevant as the Bank moves into thematic lending areas such as governance and climate change mitigation. In the context of the review of AAA starting this fiscal year, and particularly for International Development Association (IDA) countries, management will take stock of the existing body of ESW and outline steps to address identified issues. 189 19.2 Agreed/Ongoing. Management agrees with the importance of selectivity in allocating ESW resources. In the context of ESW and TA, greater selectivity may lead to reducing the number of tasks and to increasing the budget assigned to each of the tasks. But the issue should not be seen just in terms of numbers of tasks, but also in terms of consolidating tasks within a programmatic, multiyear, results-based approach so as to enhance their overall impact. The planning of the AAA program, including ESW and TA activities, takes place in the context of preparation of the CAS, and selectivity is a major consideration in preparing a CAS program that is tailored to country needs. As part of the consultation process, as well as ongoing country policy dialogue, specific knowledge gaps are identified. How and when to address them is planned, taking into account country needs, the work programs of other partners, and country circumstances. For IDA countries, issues with data quality and availability of analytic work prepared by the government or other sources may entail a larger effort to fill gaps over time. In countries where what is needed is knowledge on implementation (the engineering of reform) rather than traditional ESW, it is possible that, given weaker country capacity, smaller and more narrowly focused ESW or TA is more effective than larger efforts. In its AAA review, management will analyze resource allocation issues surrounding ESW and TA tasks and will propose steps to strengthen collaboration with in-country institutions and other stakeholders to ensure an effective ESW/TA program. 19.3 Agreed/Ongoing. Management agrees that task team presence in field offices, particularly in countries with low institutional capacity, will facilitate close collaboration with clients. The delegation of task management responsibilities to the field has been an ongoing process—for example, in the Sub-Saharan Africa Region and fragile states—and further steps are being considered. There are associated trade-offs that transcend budget constraints. Ensuring that ESW/TA is of top quality often requires that task management be carried out by thematic experts who cannot be decentralized, given their global role. In many countries, the size of the ESW program and budgetary constraints limit decentralization of specialized staff. In any country, the balance depends on the country context and the specific issues. Management is reviewing these considerations in the context of the ongoing review of decentralization. Management fully agrees with the recommendation to clearly formulate and adequately fund the dissemination and follow-up strategy for ESW and TA at the concept stage of the task. The AAA review will provide further guidance in this area. 19.4 Agreed. Management agrees that client feedback is a key input to assess how clients view the different dimensions of ESW/TA in particular and the design of the Bank‘s program in general. Feedback is already sought and used in the context of client surveys, CAS preparation, policy dialogue, lending preparation, and project implementation. Feedback focused on ESW and TA should provide further insights on client views as well as lessons for the design and implementation of the ESW and TA program. Overall, the focus should be more on the development results that might follow from changes in client approach and policies, rather than short-term client satisfaction with the production process. In the context of the AAA review, management will review options regarding the periodicity of requests for feedback to minimize the burden on country counterparts. Management will also analyze whether the feedback should focus on independent pieces of work rather than on a body of work with a common thematic focus. 190 19.5 Agreed. Management agrees with the recommendation to ensure greater attention to results, for example, by incorporating client feedback. Although use of feedback is an important component of a results framework, other elements could also be quite relevant. Attribution is certainly an issue: it is difficult to ascribe results to a single piece of ESW. In some cases a credible results framework can be established around a thematic program of activities that will help monitor results within an adequate time frame. In the context of the AAA review that is now starting, management will examine the results framework for ESW and TA and outline steps to strengthen it. Management will consider all these agreed actions completed with the conclusion of the AAA review and management decision based on the results of the review. Management will report to the Board on the outcome of that process in fiscal 2010. MAR 2012. The findings of the AAA Review (focusing on ESW/TA) have been incorporated into processes, procedures and systems in Q1 FY12, addressing key recommendation of the IEG Evaluation: strengthening the results framework for ESW/TA by incorporating systematic consideration of the success factors underlying high impact ESW/TA (e.g. relevance, ownership, timeliness, appropriate dissemination and follow-up, in addition to technical quality) as identified by IEG. This ESW/TA work is serving as a backbone to the broader Knowledge Agenda. The following was completed by end-FY11: (i) improved results framework, monitoring and measurement; (ii) better systems & processes, differentiated by products: just-in-time, programmatic, etc.; and (iii) input (eg. monitorable results indicators) to feed into the broader Knowledge work; Results Report and Corporate Scorecard. 19.1 Reinvigorate the mandate for country teams to maintain strong knowledge base on countries where the Bank is providing or planning to provide funds. The importance attached to the Bank‘s knowledge assets, especially at the country level, has been reinvigorated by the knowledge strategy, and discussed by the Knowledge and Learning Council, and the Matrix Leadership Team. The revised processing of ESW/TA rolled out in Q1 FY12 includes a standardized Concept Note which specifically focuses on the strategic alignment of knowledge services in the Bank‘s overall engagement with clients. 19.2 Ensure ESW tasks in IDA are adequately resourced, even if it means fewer ESW in some countries. There has been continued consolidation of ESW tasks, with numbers of tasks declining without a drop in overall expenditure levels. The new product line –Programmatic AAA— under development with roll-out expected in second half of FY12 enables the clustering of ESW and TA around a common development objective within a multi-year framework, thus reducing fragmentation and increasing selectivity based on country demand/need. This product line will respond to requests from Regions in order to not only reduce fragmentation, but also to facilitate increased management attention to a fewer number of larger tasks. 191 19.3 Enhance institutional arrangements for undertaking ESW and TA. Fundamental to increasing the impact of ESW and TA is the upfront recognition of client needs and hence ownership. The standardized Concept Note focuses discussion and approval of the concept on a clear statement of objectives and expected intermediate outcomes of the activity. Integral to this is the clear identification of the intended audience and appropriate ―dissemination‖ (including need for translation). The sub-recommendation to ―ensure substantive task team presence in country offices, particularly in countries with low institutional capacity‖ continues to be considered in the context of decentralization, balancing the need for the Bank to maintain and share global expertise. 19.4 Recognize and build on client preferences whether for non-lending versus lending or for TA versus ESW. Revealed client preferences continue to show a trend of increased demand for technical assistance. Programmatic AAA will also enable a more strategic fit with client needs and preferences. Piloting client survey instruments to seek their feedback on ESW/TA was undertaken as part of the first Knowledge Report. Follow-up is now underway to determine the most effective way of seeking that feedback. For larger programs of ESW/TA (i.e. Programmatic-AAA) a client feedback module is being developed. 19.5 Take the results tracking framework seriously. The findings of the AAA Review have been implemented and a strengthened results framework built into online processes and procedures rolled out in Q1 FY12. Such a framework runs through the lifecycle of the activity (i.e. intermediate outcomes are be linked to clear development objectives identified at concept stage). This recognizes that ―results‖ for knowledge products are difficult to determine especially when an ESW or TA has just been completed, hence the focus on intermediate outcomes and their indicators. A Completion Summary (CS) requires TTLs to self-assess the performance of the activity in meeting its development objective as well as a Bank performance rating (reflecting quality, timeliness, relevance and client engagement). Self-assessments will be validated by Management and aggregate indicators reflected in the Corporate Scorecard, Results Report and Knowledge Report. The results framework is also consistent with IEG‘s own evaluation methodology which should facilitate their evaluation of ESW/TA at the post- completion stage. IEG RESPONSE 2012 IEG acknowledges the progress the Bank has made in strengthening its management of knowledge during the past year, such as conducting a comprehensive review of its knowledge work and issuing first Knowledge for Development Report, issuing new guidelines for Discrete ESW/TA and Just-in-Time ESW/TA, and incorporating AAA into 192 the assessment of Bank development outcomes (i.e. ―AAA objectives accomplished‖ has been included together with ―Satisfactory CAS/CPS operations‖ and ―Satisfactory operations outcomes‖ as a criteria for rating Development Outcomes under the Corporate Scorecard). IEG CAS Reviews (FY11 to date) indicate that the Bank continues to deliver its AAA program, adjusting to changes in country circumstances. While IEG cites three country cases where the Bank‘s AAA program served as useful inputs into lending, it pointed out poor dissemination limited AAA impact in two countries. So outreach continues to be a concern. IEG agrees with Management‘s rating of the recommendations. Specifically, IEG upgrades the adoption of 19.1 (―Reinvigorate the mandate for country teams to maintain a strong knowledge base‖) from ―Medium‖ to ―Substantial‖ based on the steps being taken to improve ESW/TA management. IEG reinforce its agreement with Management‘s ―Medium‖ rating for 19.4 (―Recognize and build on client preferences whether for non- lending versus lending or for TA versus ESW‖) given the importance of client buy-in for effectiveness and the fact piloting of the survey instrument is pending. Process improvements taken by Management are expected to bear fruit in future. Therefore the recommendations are considered ―Active‖ but should be archived with subsequent IEG evaluations, such as the annual Results and Performance Report, addressing issues of Bank AAA effectiveness. 193 Management Action Record IFC 2012 1. PEP-ECA 2007 Record STD032REC04 Study Short PEP-ECA - 2007 Study Date 06/29/2007 Number Name (verified) Headline Improve the quality of M&E indicators, data collection methods, and cost accounting IEG Rec Improving the quality of M&E indicators, data collection methods, and cost accounting. Project-specific M&E targets should be tailored to country situations and conditions as part of project preparation, verified as part of the project approval system, and monitored during project implementation. More rigorous survey techniques and data collection methods should be developed to establish baselines and enable comparisons across time and across countries where possible. The proper utilization of the indicators and techniques should be closely monitored to derive lessons and ensure data quality. A more comprehensive expenditure accounting and tracking mechanism should be introduced, which would not only enhance M&E, but provide a useful project management tool for benchmarking costs of different activities and developing Advisory Services pricing and client contribution strategies. This would also provide data for analysis of benefits and efficiency of the programs and their components. Mgmt Response IFC has made major advances in M&E since the review period. IFC has identified and implemented sets of standard performance indicators of output, outcome, and impact for each product under its five business lines and is developing incentives for the use of these indicators. IFC has undertaken 20 experimental trials to understand the impact of selected projects better, and the findings are being fed back to advisory staff. IFC is also undertaking project reviews, conducted using external experts, to improve program design and implementation where 11 such reviews have been completed. Furthermore, IFC is currently developing a tailored approach to advisory cost-benefit analysis as a means of better linking M&E to project cost accounting. Indicator A tailored approach to advisory cost-benefit analysis is developed. Comment Consistent with Management Response Mgmt  IFC has developed the standard performance indicators of output, outcome, and impact for each product Comment 2011 under each business line.  IFC is piloting a set of Development Goals (IDG) which will guide IFC to reach a double bottom line of financial sustainability and development impact.  IFC is developing an evaluation framework aimed at identifying evaluation priorities; integrating the impact assessment in the project life cycle and feeding the evaluation results into the decision making process. 196  IFC is developing a new financial management system (AS FM System) for IFC‘s Advisory Services aimed at better tracking the project cost for each component. This AS FM System will help the project managers carry out the cost-benefit and cost-efficiency analysis. Mgt LOA 2011 Medium Mgt Status 2011 Active (Rated) IEG Comment On the first recommendation STD032REC04, indeed major improvements have been made in terms of 2011 developing and rolling out standard core indicators (SCI) for output, outcome and impact for each product. However, there are problems that still need to be addressed with the SCI: a. these SCI are often blindly adopted by teams because they are largely required and as such they are not always appropriate in terms of relevance to the individual project objectives and frequently other, non SCI, are needed to actually assess a project's achievement but which are not included. The outcome of this situation is that sometimes projects are claiming credit for SCI results which cannot be attributed to the project (because they did not undertake activities aimed at influencing such SCI) and the SCIs cannot help shed light on what the project actually set out to achieve. In such cases a more customized approach in terms of indicator selection. b. definitions of these indicators are still unclear in some areas and even problematic in some instances because they are interpreted and measured inconsistently across staff and teams. More detailed definitions and examples of usage are still needed. c. data collection, baseline and reporting is still problematic. Presumably the new idesk system for AS to be rolled out will offer a chance to correct a lot of these issues, but the proof will be in the pudding. IEG LOA 2011 Medium IEG Status 2011 Active Mgmt In the past FY, IFC has continued to make significant progress on issues identified by IEG through Comment 2012 improvements in M&E network and frameworks, stronger provisions in the Advisory Services governance rules, changes in project templates, and new features in our IT systems (the Advisory Services Operational Portal). 1. Several business lines have developed or are developing new frameworks which will streamline the standard indicators. The Investment Climate business line is re-designing the framework for all of its products, which will be finalized by end of FY12; Access to Finance has developed new indicators in several of its products, including microfinance; Sustainable Business Advisory has consolidated its product offerings and is revamping its frameworks. 2. Advisory Services senior management has issued, along with the Development Impact Department, a new set 197 of policies and procedures which governs AS projects and processes. Per the new governance policies, staff are given up to one year to collect baseline and other data prior to the operational start of an AS project. Staff are required to define their pre-implementation budgets, setting aside necessary funds for scoping and baseline data collection. The new guidelines also now require that baselines are collected and finalized within one calendar year. 3. To help staff identify and include relevant indicators which help measure progress towards achievement of project objectives, the Advisory Services project templates (specifically for the Project Supervision Report and the Project Completion Report) have been modified to identify what are known as ―key indicators.‖ This will help staff think more carefully and better articulate the indicators they select. 4. The Advisory Services Operational Portal has been designed to assist staff in identifying key indicators by suggesting standard indicators based on product mix. To mitigate the chance that teams will misunderstand or misinterpret the indicators, the ASOP offers staff the standard definitions of all indicators through the system. Also, staff have the ability to search for non-standard indicators used by other projects and to nominate their non-standard indicators for inclusion into the standard list of indicators. Indicators selected for a given project are reviewed and discussed between the operational and M&E staff person assigned to the project in order to ensure that all indicators are appropriate and measure progress towards project objectives. Mgt LOA 2012 High Mgt Status 2012 Active (to be made inactive) IEG Comment IFC continues to make progress along the various fronts proposed in the recommendation STD032REC04. 2012 Initiatives1-4 mentioned above in the Management Comment 2012 have been recently launched or are still work in progress. I IEG‖s PCR Evaluation work program under way focuses on the 2011 batch of PCRs which largely precedes the launching of initiatives described above. IEG looks forward to assessing the fore-mentioned improvements during the course of its 2012 PCR reviews. IEG welcomes the M&E (indicators and analysis) progress made in Microfinance product line, which is currently being rolled out as a pilot. In this effort IEG worked closely with IFC A2F staff to make the refinements, and IEG is keen to continue its collaboration with other A2F products in a similar manner. While the plan is to revise and fine tune the M&E approach (indicators, analytical framework) for other A2F products, this is still in very early stages. IEG would also be happy to collaborate with colleagues in Investment Climate BL as part of those efforts underway to refine frameworks as this process is finalized in the next few months. IEG LOA 2012 Substantial IEG Status 2012 Active (to be made inactive) 198 Record STD032REC08 Study Short PEP-ECA - 2007 Study Date 06/29/2007 Number Name (verified) Headline Account for the full cost of Advisory Services interventions IEG Rec Accounting for the full cost of designing, implementing, and supervising the Advisory Services intervention, including IFC‘s overhead and administration costs. This will derive the entire cost of the Advisory Services intervention and enable more thorough assessment of the effectiveness of the Advisory Services intervention, and adequate comparison among various projects and programs. Mgmt Response IFC already has a major program under way to upgrade its advisory project budgeting further. This program, once complete, will see IFC apply a standardized approach to all of IFC‘s advisory project budgeting. This approach will also ensure that every IFC advisory budget accounts for all program elements, including the costs of design, implementation, and supervision. Indicator The advisory budgeting process has been reviewed to include all project elements, including the cost of design, implementation, and supervision. Comment Consistent with the Mgt Response Mgmt In FY11, IFC AS streamlined its cost allocation methodology and reduced the 9 standard AS activities to 3 Comment 2011 common categories of Project Expenses, Program Management & Support and General & Administrative Expenses. A detailed methodology on allocating staff time and other direct expenses was developed and training was rolled out to AS staff in all units and regions. A methodology on allocating overheads will be implemented in FY12. Once completed, these two improvements will allow a full understanding of project costs on a consistent basis. Mgt LOA 2011 Substantial Mgt Status 2011 Active (Rated) IEG Comment The new system to be launched sounds promising. However IEG cannot validate the improvement until the 2011 system is fully rolled out and the quality of such information in PCRs is demonstrated and validated via IEG reviews. IEG LOA 2011 Substantial IEG Status 2011 Active Mgmt This recommendation originates from a review of a program in ECA, but addresses global rather than regional Comment 2012 issues. IFC's approach to budgeting AS costs has been subject to substantial reform in recent years, and those reforms continue. All relevant costs of AS have been captured for some time, and are reflected in AS project documents. This represents full compliance with the 2007 IEG recommendation. The additional measures mentioned in the 2011 Management Response go beyond this -- they relate to ongoing efforts to harmonize some aspects of our cost-allocation approach across regions and facilities, rather whether relevant costs are captured and reported per se. Management has been reporting on progress in strengthening its budgeting system and processes for AS as part of its annual budget papers to the Board for several years, and plans to continue to do 199 so. Mgt LOA 2012 High Mgt Status 2012 Active (to be made inactive) IEG Comment IEG has observed improvements in the cost accounting of AS projects, and views the level of adoption as High. 2012 IEG LOA 2012 High IEG Status 2012 Active (to be made inactive) 200 2. INDONESIA 2007 Record Number STD033REC02 Study Short Indonesia - 2007 Study Date 01/07/2008 Name (verified) Headline Help develop a long-term local currency debt market IEG Rec IFC can focus on helping Indonesia develop a long-term local currency debt market; but to do so, needs to work with the World Bank and other development partners to encourage further reforms in the bond market and the life insurance and pension systems Mgmt Response In the financial sector, IFC has been supporting the consolidation and strengthening of the banking sector, and strengthening the capacity of rural banks and microfinance institutions. IFC has disbursed nearly $150 million in long-term Rupiah loans, most of which will be used for local credit to SMEs. With respect to the local currency bond market, the World Bank, the Indonesian Ministry of Finance, and the Capital Market Supervisory Authority (BAPEPAM) have held a series of conferences and a policy dialogue on non-bank financial markets, including Rupiah bonds in 2007. The dialogue has revealed a number of policy and capacity issues for which IFC can play a role in filling the gaps. These include tax policy, the lack of an active secondary market, the lack of liquid repurchase and derivative markets, and a need to strengthen mechanisms for registration, trading and settlement, Addressing these issues will help build the confidence of institutional investors in the local currency bond market, thereby increasing its liquidity particularly for corporate issuers. IFC will consider how it can effectively play a role in building these markets through demonstration transactions as well as advisory support. Indicator Issuance of long term Rupiah bonds by IFC to catalyze the development of the long-term local currency bond market. Comment Consistent with Management response Mgmt IFC has been leading discussions with the Government to allow supranationals to issue long term local bonds to Comment 2011 broaden and deepen the bond market. In so doing, IFC is working with Government agencies to address specific technical constraints. IFC is leading a combined approach with IBRD and ADB. The response from the Government had been delayed at the Tax Authorities. However, repeated enquiries and meetings have led to progress and now the Ministry of Finance is finalizing its position regarding an approval. Mgt LOA 2011 Medium Mgt Status 2011 Active IEG Comment While efforts have been done to develop a long-term local currency debt market, results are yet to be seen in these 2011 areas. IEG LOA 2011 Medium IEG Status 2011 Active Mgmt IFC has been in discussions with the Ministry of Finance, the Central Bank and the Tax Authorities in Indonesia to 201 Comment 2012 allow Supranationals to issue local and offshore IDR denominated bonds to broaden and deepen the bond market. In so doing, IFC has teamed up with ADB and IBRD in a joint effort to address specific technical constraints and concerns that the specific government entities may have. The issue of Withholding Tax and the IFC's policy of not being able to collect tax on the behalf of an individual government is a subject of concern raised by the Indonesian Tax Authority. When it comes to onshore and offshore issuance the IFC is leading a combined approach with IBRD and ADB in order to allay their fears of crowding out the Government Bond market and the effects of excessive offshore IDR issuance on the FX. The response from the Government had been delayed at the Tax Authorities (under the tutelage of the Ministry of Finance). However, repeated enquiries and meetings have led to some progress however the Ministry of Finance is currently focusing on a domestic bond settlement programme. It is our understanding that when this matter is completed, the Ministry of Finance may be able to take a closer look at the approval process." Mgt LOA 2012 Medium Mgt Status 2012 Active (Rated) IEG Comment Long-term local currency debt market is still underdeveloped. While IFC efforts can be seen, progress has been 2012 delayed due to technical issues and diverse government priorities. As a result ratings for level of adoption and status of implementation remain the same. IEG LOA 2012 Medium IEG Status 2012 Active (Rated) 202 3. IEDR 2008 Record STD034REC03 Study Short IEDR - 2008 Study Date 02/27/2008 Number Name (verified) Headline Measuring Advisory Services Performance IEG Rec IFC should continue, with input from IEG, to strengthen the steps it is taking to improve the data on the performance of advisory services operations, including efforts to improve understanding among staff about results measurement, quality assurance by managers, as well as performance monitoring beyond project close. Mgmt Response Management agrees with the recommendation to strengthen measures to improve the data on performance of advisory services operations, in consultation with IEG-IFC. IFC is currently taking a number of steps to improve the quality of data in the project supervision and completion reports, through further streamlining of core indicators by product line, promoting baseline data collection, increasing oversight by evaluation staff, and increasing training and knowledge sharing. Together with IEG-IFC, the IFC results measurement team is organizing a two-day course on monitoring and evaluation of private sector advisory programs at the International Program for Development Evaluation Training in June 2008. As part of knowledge sharing, two annual conferences are organized each year, in which lessons learned from evaluations and evaluation methodologies are shared among the broader World Bank Group, donors, academics, and other multilateral development banks. Partnerships in evaluation have been established with leading experts from institutions such as the MIT Poverty Action Lab, and with foundations such as the German Marshall Fund. Forums on results measurement are being held for donors and multilaterals as well as for foundations in May 2008. Competencies for monitoring and evaluation staff are being developed to strengthen the cadre.IFC will continue to consult with IEG-IFC on developing guidance for identifying, at approval, projects that would require post- completion monitoring so that appropriate frameworks and plans can be established at the outset. IFC would welcome IEG-IFC‘s efforts to institute a mechanism to provide feedback to staff on Project Completion Report (PCR) ratings as soon as possible, as is being done in the case of the Expanded Project Supervision Report (XPSR) system on the investment side. (PCRs and XPSRs are IFC‘s project-level self-evaluation reports for advisory services and investment operations respectively). Indicator Progress in further refining approach to results measurement for advisory services. Comment Consistent with Management response Mgmt  To strengthened IFC focus on measuring and evaluating development results, a new department - Comment 2011 Development Impact Department (CDI) was created in Sept 2010. The new department consolidates two teams currently dedicated to results measurement for the Investment Services and Advisory Services, 203 aligning their measurement and evaluation systems.  Quality review of project Approval, and Supervision and Completion reports is regularly conducted by CDI. The results are shared with AS management and staff via quarterly operations reports. These results also constitute an important part of biannual portfolio reviews.  PCR review meeting has been introduced by to ensure and improve the project quality completion report.  Agreement with IEG has been reached regarding projects that should be excluded from a DE rating.  A working group is set up in developing the post completion monitoring framework and guidelines.  All existing practices and guidelines regarding the AS operations governance are in the process of formalization.  CDI is developing a corporate guideline on data aggregation. Mgt LOA 2011 Substantial Mgt Status 2011 Active (Rated) IEG Comment IEG recognizes that much has been done to improve M&E system shortcomings, but that implementation has 2011 yet to show fruits of efforts. IEG LOA 2011 Substantial IEG Status 2011 Active (Rated) Mgmt During the past FY, IFC has taken further steps, in addition to the ones commended by IEG last year. IFC has Comment 2012 continued to develop its Results Measurement Network (RMN). IFC has continued to develop its Results Measurement Network (RMN). After a working group composed of staff from Investment and Advisory Services proposed the expansion of the RMN to Investment Services, as well as enhance the professionalization of the Investment Services results measurement staff, the IFC Management Team has agreed to implement this proposal in FY12. As part of this proposal, a competency framework and professional stream for results measurement staff are being developed by the Development Impact Department (CDI) and IFC Human Resources Department. Also, as part of the newly developed (May 2011) Advisory Services Project Governance Policies and Procedures document, AS has now identified Regional Business Line Leader managers as having primary accountability for the quality of their results data and the performance ratings. As part of the semi-annual Supervision review of active projects in the portfolio, managers now seek guidance from Monitoring and Evaluation staff and confirmation that results measurement issues have been addressed via the iCollaborate IT platform. These reviews are combined with the mandatory PCR review meetings mentioned in last year‘s comments from Management to improve the quality of reporting in AS. The quality of IFC AS documentation has improved significantly in recent years throughout the life cycle. CDI‘s 204 review of quality at entry (through a review of all AS Implementation Plan documents) shows that a vast majority of newly approved projects in FY11 had well-specified objectives and provided sufficient contextual information to inform managers about the rationale for project objectives and design. For the active portfolio, in the FY11Q2 set of Project Supervision Reports, a vast majority of reviewed active projects had provided baseline and target values for outcome indicators, and were found by CDI to have no significant data quality issues. CDI‘s review included checking for all appropriate indicators, consistent targets for linked output and outcome indicators, consistent reported results between linked indicators, and other checks. CDI can share the underlying data with IEG, if requested. In terms of AS Development Effectiveness, which is determined using the Project Completion Reports, IEG‘s Annual Report 2011 (Results and Performance of the World Bank Group) states: ―Among IFC‘s Advisory Services evaluated in FY08–10, 64 percent had successful development effectiveness ratings.‖ CDI‘s own analysis for that same period found our Development Effectiveness rating to be 63%. Given that the steps taken by IFC, the general improvement in the quality of the documentation, the extremely close (within 1 percentage point) ratings on Development Effectiveness, it would appear that the steps which IEG acknowledged last year have now borne fruit. Mgt LOA 2012 High Mgt Status 2012 Active (to be made inactive) IEG Comment IEG agrees that overall quality of PCRs has improved. However, justification of Outcome and Impact ratings 2012 could be further improved by conducting needed contextual analysis in the text (e.g. trend analysis, discussion of externalities, discussion of results versus baseline) to support the quantitative data shown in the Development Results table. Such analysis would provide more credible attribution-oriented understanding of results achievements claimed in tables. IFC teams should also conduct a more consistent filing of AS project supporting documents in IFC Docs including i) offline approval comments for project‘s approval and completion ii) minutes of PCR review meetings (similar to Investment Review Meetings Minutes on the Investment side) iii) any documents that constitute a project output and iv) any documents that provide additional evidence for ratings. Another area of priority related to recommendation STD034REC03 is to launch the system to monitor performance beyond project closure: it is critical to make progress in finalizing and rolling out the Post- Completion follow-up initiative to better capture project results, as many projects are not adequately mature by 205 PCR and require additional time in order to have a more complete and accurate documentation and perspective on the overall development impacts. IEG will retire this recommendation but keep addressing this issue in the context of its 2012 BROE. IEG LOA 2012 Substantial IEG Status 2012 Active (to be made inactive) 206 4. NIGERIA 2008 Record STD037REC01 Study Short Nigeria - 2008 Study Date 09/15/2008 Number Name (verified) Headline Infrastructure and non-oil growth IEG Rec IFC needs to diversify areas of intervention in Nigeria to: (i) help address development challenges related to poor infrastructure (in particular power and roads) and excessive dependence on the oil sector; (ii) contribute to trickle-down effects of oil-driven growth; and (iii) expand viable private sector activities beyond the present narrow confines of operations in terms of sectors. This would involve: (i) more strategic and effective deploymentof advisory services, particularly in infrastructure and related areas; and (ii) close cooperation with the World Bank to help improve the business environment. Mgmt Response Agrees with the thrust of the recommendation - IFC is actively engaged in seeking effective means of engaging in the real sectors and expect a greater progress in the future as reforms continue to take hold. IFC agrees to focus thier efforts on infrastructure on the power sector and key services sectors. IFC has deployed its advisory services in a more strategic manner with the advent of PEP - Africa from 2005. Indicator 1. Annual commitments and number of projects in non-financial sectors. 2. Percentage of annual commitments in non-financial sectors Comment Direct and easy to track. Mgmt IFC began making progress on this objective, increasing real sector shares in FY 10, both in terms of volumes and Comment 2011 projects: 2011 Commitments $ 2009 2010 YTD Nigeria all 547 689.4 638.5 Real Sectors 80 119.3 64 Real Sectors Share 14.6% 17.3% 10.0% 2011 Commitments # 2009 2010 YTD Nigeria all 11 12 12 Nigeria FM 3 4.5 1.5 Real Sectors Share 27.3% 37.5% 12.5% 207 2011 YTD at 2/28/2011; full year results will show additional real sector projects. However, beginning in FY 10 IFC purposely departed from this strategy and refocused on the financial sector to assist the country‘s emergence from a banking crisis. This resulted in less progress than planned, and will result in a decline in real sector shares in FY 11 in particular. Beginning FY 12, we will resume making progress, having completed the crisis response strategy. Mgt LOA 2011 Medium Mgt Status 2011 Active (Rated) IEG Comment IFC‘s progress towards meeting the objective remains marginal and the modest trend seems to be even reverting 2011 in FY11. However, given the impending financial crisis and the size of the financial markets portfolio in Nigeria, it was appropriate for IFC to make determined efforts to protect its investment portfolio, particularly knowing the adverse effects that past crises had on IFC‘s portfolio. On the other hand, a crisis can present good opportunities for impact through new investments. So while the re-focused strategy has been adequate, a continue effort to a more diversified portfolio would have been appropriate as well. While the Mgmt response indicates that IFC has deployed AS in a more strategic manner with PEP-Africa, except for a few initiatives in Health, not much has been done to diversify the portfolio. IEG LOA 2011 Negligible IEG Status 2011 Active Mgmt After making initial progress on the objective, IFC‘s crisis response strategy for Nigeria has led to a reversal and Comment 2012 subsequent increase in financial sector shares in FY 11, which will continue through FY 12. The region presented a revised strategy to COC in March 2011 emphasizing the diversification strategy, noting however that the crisis response which included a temporary increase in banking investments to provide capital and liquidity to strategic banks would result in a resurgence of FM volumes in 2011. This will delay implementation of the diversification objective, as FY 12 results are expected to be similar to FY 11. In September, the Nigeria country team prepared an action plan to respond to specific requests from the new Minister for Economy and Finance, which included a focus on power sector project development, advisory assistance with PPP structuring for the 2nd Niger River bridge, and several initiatives related to youth/women employment generation. While these do not (yet) reflect in investment results, they support the focus on addressing infrastructure constraints and real sector investment. In the past month, continued violence from the Boko Haram group has grown increasingly disruptive, and the instability, although likely only temporary, of the government‘s attempt to end the fuel subsidy program, have 208 deferred much investment and raised increased risk concerns for Nigeria. Commitments $ 2009 2010 2011 2012 H1 2012 C+HP Nigeria all 547 689.4 927.3 311.7 540.76 Real Sectors 80 119.3 99.4 15.5 137.5 Real Sectors Share 14.6% 17.3% 10.7% 5.0% 25.4% Commitments # 2009 2010 2011 2012 H1 2012 C+HP Nigeria all 11 12 13 8 13 Real Sectors 3 4.5 3.5 2 3.5 Real Sectors Share 27.3% 37.5% 26.9% 25.0% 26.9% Mgt LOA 2012 Medium Mgt Status 2012 Active (Rated) IEG Comment As noted in previous 2011 IEG comment, modest trend reverted in 2011 due to IFC‘s response to crisis. 2012 However, changes in strategic focus and new advisory service initiatives are important steps towards portfolio diversification. As a result, a medium rate is appropriate and implementation rating remains active. IEG LOA 2012 Medium IEG Status 2012 Active(Rated) 209 5. BROE 2008 Record STD038REC02 Study Short BROE - 2008 Study Date 10/01/2008 Number Name (verified) Headline Achieving better coverage of IFC‘s portfolio in reporting on results IEG Rec IFC should aim to expand the coverage of IFC‘s portfolio in reporting on results. It should improve the portfolio coverage of reach indicators and report the development results of its entire mature portfolio Mgmt Reponse DOTS cover all the active companies in IFC's investment portfolio (over 1200), and internal reporting to IFC management covers all active companies. In the Annual Report, IFC focuses on the 6 - year time slice of operations, similar to IEG's approach for its Independent Evaluation of Development Results. The Annual Reports clearly articulates what is covered and why. Since DOTS covers projects until closure, IFC will eventually report on both the active and closed projects, but in its current early stage, coverage of closed projects is too limited to be meaningful. IFC is committed to further increase the coverage of our portfolio with reach indictors, particularly in Sub Saharan Africa. The introduction of (DOTS-II) is expected to enhance the technological platform to enable staff and management to report, track and analyze reach information real time. Indicator 1. Coverage for mandatory reach indicators for the active portfolio (with some exclusions) to reach 80% coverage. Comment With the implementation of DOTS 2, IFC aims to achieve 80% coverage for mandatory reach indicators of active portfolio in the subsequent years. Exclusions such as projects in special operations will limit the level of coverage IFC can achieve. Although reach indicators for Africa will be monitored closely, information on MSMEs, which is an area of focus for Africa, is generally more difficult to obtain. Mgmt With the implementation of DOTS 2, IFC reported at least 80% coverage for 10 out of 13 mandatory reach Comment 2011 indicators for its active portfolio. All mandatory MSME reach indicators achieved at least 80% coverage. The coverage ratios of the rest of the mandatory reach indicators were nevertheless very close to 80%. IFC is well on its way to achieve 80% coverage for all mandatory indicators (with some exclusion). Mgt LOA 2011 Substantial Mgt Status 2011 Active (Rated) IEG Comment Progress made in DOTS 2 for wider coverage. Continue to improve coverage. 2011 IEG LOA 2011 Substantial IEG Status 2011 Active (Rated) Mgmt After DOTS2 has been in operation for about two years, IFC reported more than 80% coverage for all 13 210 Comment 2012 mandatory reach indicators for its active portfolio. Mgt LOA 2012 High Mgt Status 2012 Active (To Be Made Inactive) IEG Comment Progress made in DOTS 2 for better coverage. The DOTS 2 reports all 13 mandatory reach indicators had more 2012 than 80 percent coverage.(gas distribution 80% to power distribution 100%) IEG LOA 2012 High IEG Status 2012 Active (to be made inactive) NOTE: Source of validation : data received from CDI. All above 80%. SME indicators (SME Finance coverage = 87% and Housing finance coverage = 86% as below. *Portfolio reach figures represent SME and microfinance outstanding loan portfolio of IFC clients as of end of CY09 and CY10, for MSME-oriented financial institutions/projects. 221 and 222 clients were required to report their end-of-year SME and microfinance portfolios in CY09 and CY10, respectively. 183 and 195 clients did so for CY09 and CY10, respectively. The missing data were extrapolated. ** Portfolio reach figures represent housing finance outstanding loan portfolio of IFC clients as of end of CY09 and CY10, for housing finance-oriented financial institutions/projects. 38 and 30 clients were required to report their end-of-year housing finance portfolios in CY09 and CY10, respectively. 30 and 26 clients did so for CY09 and CY10, respectively. The missing data were extrapolated. One client in South Asia that had to be extrapolated based on figures obtained from the client's Annual Report, represents 57.6% of the total volume and 77% of the total number of housing loan figures listed above. Other indicators 211 # of Clients Requested Indicator Name Completion Rate Total # of Flags to Report Employment 747 92% 0 Power generation 41 95% 3 Power distribution 18 100% 3 Water distribution 11 82% 3 Gas distribution 5 80% 1 Phone connections 33 88% 2 Patients reached 30 87% 4 Students reached 17 94% 1 Farmers reached 72 90% 1 Purchase from National Suppliers 280 91% 1 Taxes and Other Payments 704 94% 2 212 6. IEDR 2009 Record STD040REC04 Study Short IEDR - 2009 Study Date 03/06/2009 Number Name (verified) Headline Improve execution of the Advisory Services (AS) pricing policy. IEG Rec Over the longer term, it would be important to seek client contributions that reflect value and impact, i.e. not just cost, to create a true test of client demand, incentives for better AS delivery, and ensure IFC is being additional. Mgmt Response Management agrees that our pricing policy is an important tool to strengthen the impact of our AS interventions, but differs with IEG on parts of their analysis and recommendations. IFC has been charging clients for some of its advisory products for many years. Since 2007 this policy was broadened to embrace the full range of our advisory services. The policy was intended to strengthen the clients‘ commitment to implementing the advice. Moreover, since our AS is focused on addressing market failures, pricing approaches based on the value or impact of our AS will often not be relevant pr practicable. IFC intends to keep the operation of our AS pricing policy under regular review, and will continue to refine the implementation of the policy based on experience. Indicator Progress in implementing 2007 Pricing Policy, as determined through regular reviews of that progress and emerging lessons of experience. Comment Consistent with Management response Mgmt IFC AS continues to elaborate and refine its pricing and client contribution policy including drafting more Comment 2011 specific procedures and guidelines for both operational and finance staff. Early in FY11, additional guidance was issued to staff for (i) Clarification of nature, form and valuation of client contributions; (ii) Requirements for recognition of in-kind and indirect contributions; and (iii) Use of legal agreements. A training program for all AS staff on client contributions and pricing will be rolled out in late FY11 / early FY12. Mgt LOA 2011 Substantial Mgt Status 2011 Active (Rated) IEG Comment IEG recognizes that progress has been made with regard to AS pricing. The institution-wide implementation 2011 of a pricing policy will be enhanced once the AS staff training can be effectively designed and implemented. This should provide more clarity to staff on how to categorize and assess appropriate and meaningful client contribution to AS projects. IEG should consider adjusting the status of this recommendation after the training has been conducted. IEG LOA 2011 Medium IEG Status 2011 Active Mgmt Compliance with IFC's pioneering pricing policy for AS has improved significantly in recent years: 213 Comment 2012  Ninety-five percent of projects approved in the first half of FY12 expected client contributions, up from 87% of projects approved in FY10.  While IFC's pricing policy treats cash, parallel and in-kind contributions as equal in status -- since each has its strengths and weaknesses in particular types of client engagement -- the share of projects expecting contributions in cash had increased to 77% in the first half of FY12, up from 51% in FY10, and the share involving in-kind contributions only had fallen from 16% to 11%.  The share of client contributions in total project costs expected at time of project approval has also risen to 40% for projects approved in the first half of FY12 (up from 39% in FY10), while the share of contributions expected in cash had risen from 26% to 33%, and in-kind and parallel contributions had fallen from 13% to 7%.  The value of client contributions realized by time of project completion has also increased strongly, and now exceeds expectations at time of project approval. For example, for projects completed in the first half of FY12, contributions amounting to 46% of total project costs had been realized, with 36% in cash, whereas at time of approval those projects expected only a 40% contribution, with 26% in cash. This compares with projects completed in FY10, where contributions valued at 24% of total project costs were realized (19% in cash), and at time of project approval those projects expected an average of only 20% contributions (10% in cash).  The quality of project documentation on pricing issues has also been improving significantly -- although the fact that the average AS projects lasts around 3 years means that this and other improvements will only be fully apparent for completed projects with a lagged effect. These improvements reflect growing experience in implementing the pricing policy, as well as a series of initiatives by IFC Management. The latter include:  Management has provided additional guidance to staff on aspects of implementing the pricing policy, including on such matters as valuing and recording in-kind contributions.  Management has incorporated training on AS pricing into broader training on policies and procedures for AS staff. This includes a mandatory E-test which has been completed by all AS staff, and is required to be completed by all new AS staff as part of their on-boarding, as well as twice-annual face-to-face training events ("Doing AS") held in regions. Formal training of this kind is complemented by discussion of recent developments and lessons of experience in application of the pricing policy at regular events organized by our four global AS business lines, as well as in regional meetings.  Since FY11, Management has incorporated a rating of progress in collecting client contributions in the twice- annual Project Supervision Reports required for all active AS projects. This strengthens staff incentives, provides Management with an early flag if performance is out of line with expectations at time of project 214 approval, and provides a useful additional source for ongoing learning. Based on the results of FY11 Q4 Supervision, 87% of projects were rated as at least Partially Satisfactory in collecting expected cash contributions, and 95% had similar ratings for realizing expected contributions in other forms.  Ongoing improvements to IFC's business systems for AS, including the recently launched AS Operational Portal, and planned major upgrades to the AS Financial Management system, are strengthening our ability to report on and analyze progress in realizing client contributions to AS projects. Taken together, these measures are resulting in much stronger compliance with the AS pricing policy, and are providing staff with the tools, incentives, and systems to support continuous improvement and learning. Indeed, we are also seeing progress on one of the practical impediments to IFC's pursuit of client contributions -- the fact that most other DFIs provide such services gratis. For example, following a series of exchanges with IFC, EBRD recently introduced a pricing policy for its advisory services, and similar initiatives are under discussion in other DFIs. Mgt LOA 2012 High Mgt Status 2012 Active (to be made inactive) IEG Comment IEG recognizes improvements have been made in terms of application of IFC‘s pricing policy over the course of 2012 PCR reviewed by IEG. However, IEG's preliminary analysis of IFC's pricing policy has identified some implementation issues which may require attention. Early findings indicate that only 32% of projects successfully achieved some form of client contribution, compared to the expected 58% at approval for projects approved after January 2007 and closed by FY10. Failure to realize expected client contributions was largely driven by shortfalls in actual in-kind and parallel contributions. In contrast, cash actual contributions were close to expectations at approval. What is not clear, however, is whether the large shortfall in in-kind and parallel contributions was due to poor recording of actual contributions, or simply under achievement. The quality of client contribution information coming from project documents and IFC‘s systems varies substantially from project to project. Common problems have been data inconsistencies (e.g. actual vs what‘s recorded in PCR or contributions reported in the qualitative section of the PCRs versus client contributions reflected in PCR financial tables), lack of information for unrealized contributions and insufficient justification for in-kind values assigned. Current data sources such as IBIS and the IFC AS package report on actual revenues of the project (that mainly refer to cash fees) but do not report actual in kind and parallel contributions. The introduction of IFC‘s Procedures for Recording and Reporting AS Client Contributions in July 2011 should further enhance the consistency and quality of client contribution data but the effects of this new procedure and 215 related training efforts training can be only assessed in the course of implementation. IEG plans to conduct a review of pricing policy compliance as part of an AS cluster review before the end of FY12 to more fully assess compliance. Once this is assessed, IEG will reconsider the recommendation‘s rating. IEG LOA 2012 Substantial IEG Status 2012 Active Record STD040REC05 Study Short IEDR - 2009 Study Date 03/06/2009 Number Name (verified) Headline Strengthen Advisory Services (AS) performance measurement and internal knowledge management IEG Rec In the short term, it would be important to have more hands–on M&E support in the field, post–project completion follow up, capturing of lessons from dropped or terminated projects, and more arms– length facility, product and project reviews. In the medium term, it would pay to introduce an XPCR system (akin to the XPSR system for investment operations, and carried out later than the PCR to better capture impacts), more programmatic impact evaluation and impact research, setting results–based targets for AS in its corporate scorecard, and regular benchmarking of IFC AS activities and systems with other providers of knowledge services, including other multilateral development banks and commercial providers. In the longer term, the aim could be to establish a specialized research unit focusing on generating and bringing together private sector development knowledge work. Mgmt Response IFC agrees on the importance of effective performance management and internal knowledge management, and is committed to improving its performance in both areas. IFC introduced its M&E system for advisory in 2005. The M&E function in IFC is decentralized, with every region staffed with one or more M &E officers. Going forward. Management would be very supportive of the development of an XPSR instrument. Management has recently launched a major KM initiative for IFC as a whole. In addition, the joint World Bank. IFC Vice Presidency for Finance and Private Sector Development engages in a substantial research with internal and external partners. Against this background, management does not believe that a specialized IFC unit focusing on private sector development knowledge work is necessary over and above the current initiatives. Indicator 1. Further refined approach to results measurement for advisory services. 2. Monitoring and evaluation capacity in the field. 3. Strengthening knowledge systems and processes, as measured by: Use of KM incentives in the performance management system; Leveraging of current KM infrastructure for corporate use; Increased link between business process efficiency & knowledge initiatives. Comment KM indicators focus on the behavioral aspects of adopting a knowledge management approach. Since behavioral 216 change is hard to accomplish, progress on these indicators will indicate meaningful change. Frameworks are continuously revised and updated and methodologies for impact evaluation are being improved. Mgmt  A working group is forming to develop the post project completion framework for tracking the long term Comment 2011 development impact of the AS operations.  CDI is undertaking a comprehensive analysis of KM operations with an objective to develop an M&E framework to better track the results of KM operations.  At the Corporate level, a comprehensive knowledge strategy was formulated and presented to Management in 2010. Based on this, the Global Knowledge Office has been implementing a multi-pronged KM work program that is comprised of benchmarking and analytics, tools, technology, behavioral change, incentives, knowledge products (including lessons learned and flagship products), KM competencies, communications and outreach. The Knowledge Strategy Committee was elevated to a corporate committee with delegated authority in February 2011. An IFC Knowledge Strategy is expected to be presented to CODE and to the Board in FY12. Mgt LOA 2011 Medium Mgt Status 2011 Active (Rated) IEG Comment  This is a process which has begun and improvements are being made over time. Despite the introduction of 2011 standard core indicators, implementation has been difficult due to (i) lack of common definitions across regions of SCI, (ii) use of SCIs even if not relevant to a given project, which ultimately overstates results which are not attributable, (iii) general problems of attribution of results being claimed—for example, an advisory to FM project which claims ‗credit‘ for all increased lending, instead of a realistic increment of lending, and (iv) too much focus on SCI at the expense of using indicators that more appropriately capture expected outcomes and impacts. Management is currently working on the attribution aspect with the introduction of the IDGs which is a step in the right direction, yet more can be done to strengthen definition and selection of indicators, as well as idesk data reporting.  IFC has increased M&E presence in the field in all regions, yet the quality of M&E capacity, and the support they have from regional management, differs across regions. Hence effective improvements are more tangible in some regions, while others continue to lag. IEG LOA 2011 Medium IEG Status 2011 Active (Rated) Mgmt To help staff identify and include relevant indicators which help measure progress towards achievement of Comment 2012 project objectives, the Advisory Services project templates (specifically for the Project Supervision Report and the Project Completion Report) have been modified to identify what are known as ―key indicators.‖ This will help staff think more carefully and better articulate the indicators they select. The Advisory Services Operational Portal (ASOP) has been designed to assist staff in identifying key indicators 217 by suggesting standard indicators based on product mix. To mitigate the chance that teams will misunderstand or misinterpret the indicators, the ASOP offers staff the standard definitions of all indicators through the system. Also, staff have the ability to search for non-standard indicators used by other projects and to nominate their non-standard indicators for inclusion into the standard list of indicators. Indicators selected for a given project are reviewed and discussed between the operational and M&E staff person assigned to the project in order to ensure that all indicators are appropriate and measure progress towards project objectives. In addition to the AS Operational Portal, other IT systems are coming online which will help with data quality checking and data aggregation. The new AS Information Delivery system (ASID) will aggregate results data and will help staff to identify data points which appear to be erroneous or anomalous. As IEG noted last year, IFC continues to improve on how to attribute results being claimed. The IDGs have clearly defined attribution rules; tools have been developed to advise staff on how to apply the rules, and staff in the Development Impact Department (CDI) are providing training across regions this FY. CDI is also checking to ensure that these attribution rules are being applied uniformly across all regions, Business Lines, and Industry Departments. In terms of the Results Measurement Network (RMN), IFC continues to invest in and develop the capacity of its RMN staff. After a working group composed of staff from Investment and Advisory Services proposed the expansion of the RMN to Investment Services, as well as enhance the professionalization of the Investment Services results measurement staff, the IFC Management Team has agreed to implement this proposal in FY12. As part of this proposal, a competency framework and professional stream for results measurement staff is being developed by the Development Impact Department (CDI) and IFC Human Resources Department. The competency framework will provide specific guidance to managers as they consider what skills sets (behavioral and technical) are required for different staff grade levels within the results measurement stream. The KM framework mentioned in last year‘s management comment has been delayed and is being implemented this Financial Year. Implementation of the new framework will take place in FY13. Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment IEG recognizes the efforts and progress made to improve indicators, clarify definitions, upgrade IT systems, 2012 clarify attribution policy and enhance the overall quality and professionalism of M&E function. Several of the initiatives cited in Management‘s comments for 2012 are recent and improvement can only be assessed during 218 the course of implementation. In addition to following up on the proposed KM framework, it is also critical to make progress in finalizing and rolling out the Post-Completion follow-up initiative to better capture project development results, as many projects are not adequately mature by PCR and require additional time in order to have a more complete and accurate perspective on the overall development results of AS projects. IEG LOA 2012 Medium IEG Status 2012 Active (Rated) 219 7. HNP 2009 Record STD041REC01 Study Short HNP - 2009 Study Date 39841 Number Name (verified) Headline Expand support for innovative approaches and viable business models IEG Rec Expand support for innovative approaches and viable business models that demonstrate private sector solutions to improve the health of the poor, including expansion of investments in low-cost generic drugs and technologies that address health problems of the poor. Mgmt Response Management agrees with working on innovative approaches and helping private providers to move down- market to serve lower-income groups and markets. IFC has several initiatives already under way to build on its work to date. Some examples include: • Health in Africa initiative. • Working with clients to invest expertise and capital from high-income to low- and lower-middle income countries, e.g., Saudi-German Hospitals, based in Saudi Arabia, opened hospitals in Yemen, Egypt, and Ethiopia with IFC finance, creating a South-South investment. • Output-based aid projects in Yemen and Nigeria, in which poor people get subsidized care in IFC- financed private facilities that otherwise would not exist. • Creating finance facilities for health care small and medium-size enterprises in low-income countries by use of structured finance, combined with technical assistance, that IFC pioneered with banks in Africa and elsewhere to finance education facilities, and building on that knowledge to apply it in health. • Working with clients to move down-market within their country, e.g., working with Apollo Hospitals in India to create hospitals in secondary cities. Indicator 1. Number of SMEs financed (investment); 2. % of people from BoP served by Equity Vehicle for Health in Africa (EVHA) investee companies (Investment); 3. Whether any previously unavailable generic drugs were made available in the country where IFC project was financed; if no new generic drugs were introduced, whether existing drugs were made available at a more affordable price, as a result of an IFC-financed project; 4. Whether a new drug producing technology was introduced as a result of an IFC-financed project. (Investment); 5. Whether an IFC-financed project provided novel drugs to the market, domestic or export. (Investment); 6. Whether generic drugs were manufactured to GMP standards or more stringent quality standards as a result of an IFC-financed project. (Investment); 7. Number of OBA grants provided as % of IFC projects (Advisory Services); 8. Amount of OBA grants provided (Advisory Services); 9. % of poor assisted by each project (Advisory Services). Comment Wholesaling facilities that provide financing to healthcare SMEs give access to financing to smaller players that may not otherwise get access. Therefore, improving access to healthcare services to middle-and low income patients. The number of healthcare SMEs financed by these wholesaling facilities is tracked through the DOTS 220 system and can be reported on an annual basis. The Health in Africa Initiative through the Equity Vehicle for Health in Africa (EVHA), is supposed to provide improve access to health-related goods and services to the BoP. This will be tracked through DOTS. The target is to have at least 50% of people served by EVHA investee companies to be from BoP. Access to medicines is related both to availability and affordability. It is important in the first instance to monitor the range of medicines being made available on the market as a result of the IFC- financed project. Secondly, most IFC-financed projects will increase the availability of generic drugs; this can stimulate price competition, thus lowering the prices of both the originator brand name medicine and other generic drugs. Finally, improving the quality of medicines available on the market is also key. Together these can result in improved access to quality and affordable essential medicines for the poor. The Global Partnership for Output-Based Aid's mandate is to fund, design, demonstrate, and document output-based aid approaches to improve delivery of basic infrastructure and social services to the poor in developing countries. It encourages the use of explicit performance-based subsidies to support the delivery of basic services, such as health care, where policy concerns would justify public funding to complement or replace user-fees. The number and amount of OBA grants can be tracked and reported on annual basis. Mgmt IFC remains committed to leveraging wholesaling facilities to reach a larger number of patients receiving Comment 2011 medical care through small and medium size health care companies. In FY 2010, we committed four facilities: i) US$50 million to Banca Transylvania in Romania (IFC Project # 28708), which is expected to reach 6000 SMEs in health sector, of which 2300 are in rural and frontier regions of Romania., ii) US$25 million Bic Banco (Project # 28626), creating access to finance to 80 SMEs, (iii) US$50 million for FBN in Nigeria - Health, Education and Agribusiness (Project #28220), increasing access to finance to 455 SMEs and; iv) US$25 million for Yapi Kredi in Turkey ( Project #29117), reaching about 550 SMEs. The investment part of the Africa Health Program led to the establishment of a new health focused Equity Fund for Health in Africa and debt facilities through commercial banks in most of the involved focus countries. These activities are occurring in parallel to a continuation of traditional IFC investment activities in Africa. We remain engaged in developing additional opportunities, especially in Africa. IN FY 2011, we committed US$11 million Vinati Organics in India (FC Project #29888), the world's largest supplier of a key active substance to make for ibuprofen, which is expected to increase access to affordable and quality drugs in India and Africa. In addition, we are actively exploring the use of shorter term supplier finance to support Life Sciences companies. Mgt LOA 2011 Medium Mgt Status 2011 Active (Rated) IEG Comment IFC expanded its portfolio in Health, and many includes new approaches. Ultimate success of these 2011 approaches and lessons are to be assessed. 221 IEG LOA 2011 Medium IEG Status 2011 Active (Rated) Mgmt IFC remains committed to working on innovative approaches and helping private providers to move down- Comment 2012 market to serve lower-income groups and markets. In FY11, IFC provided finance to: (i) Amen Sante, part of COMAR in Tunisia, to finance the a private hospital that will provide healthcare paid for by public health insurance with co-payments at half the level of current private clinics in Tunisia. The model is a first in the country, actively encouraged by the Ministry of Health and the commitment was made soon after the Arab Spring in Tunisia; (ii) Grupo Salud de Peru, part of which will be used to complete a large outpatient facility for public patients and partly for a private hospital and cancer center. The PPP will complete and operate a healthcare facility which commenced construction by the public sector in the 1970s but was never completed for lack of funding; and (iii) MedLife in Romania, which is expanding its chain of outpatient clinics and hospitals, which treat both public and private patients, including contracts that treat patients for the public health insurance. The company also provides diagnostic laboratory services for public patients treated in public hospitals. Through the Africa Health Fund (AHF) and the Investment Fund for Health in Africa (IFHA), IFC also made investments to small and medium sized companies providing health related goods and services in Africa reaching BOP. These activities are occurring parallel to IFC‘s direct investments in the health sector in Africa. Recently, we supported Hikma to expand access to more affordable high quality dugs to middle and lower income groups in the Middle East and North Africa EVHA has only made seven (7) investments. In terms of the BoP served by EVHA companies, only one has been independently assessed (there is a lag between investing and making the independent assessment). The independent review of that particular investment (Nairobi Women's Hospital - NWH) found that NWH has met Target A (BoP 3000 > 50%) and Target C (BoP 1000 > 15%) with a 99% level of confidence and fell slightly short of Target B (BoP 3000 > 70%). In other words, almost 70% of clients are from the base of the pyramid (income <$3000/year on a PPP basis) and more than 15% are from the very poor (income <$1000/year on a PPP basis). Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment Management reported progresses based on indicator #1 (number of SMES financed in health sector), #2 % of 2012 people from BOP served by EVHA, #3 introduction of new or cheaper generic drugs, and #5 novel drug market. However, it did not state new development in advisory services activities, including OBA (Indicators 7, 8 and 9), as well as indicators #4 introduction of new drug technology and #6 drug manufacturing based on quality standards. Based on the given indicators, IFC expanded its activities in health further during the year, with focus on down-market to serve lower-income population. Ultimate successes of these approaches, especially to the improving the access to health services to the poor, are to be assessed. 222 IEG LOA 2012 Medium IEG Status 2012 Active Record STD041REC02 Study Short HNP - 2009 Study Date 39841 Number Name (verified) Headline Support public-private partnerships IEG Rec Support public-private partnerships through Advisory Services to government and industry and through its investments, and expand investments in health insurance. Mgmt Response IFC has supported pioneering health public-private partnerships (PPPs) in Romania and Lesotho and continues to work in this area. While health PPPs are a relatively recent development in emerging markets, there is increasing interest in health PPPs as a means to expand and improve services for the public. The work in Lesotho is at the leading edge for emerging-country health PPPs in several aspects. From the investment aspect, partners in health PPPs to date have often been construction companies rather than health providers and have not required capital from IFC. In some of the few cases where it is truly health services, rather than construction and facilities management, that have been provided by the private sector, IFC has financed providers of renal dialysis services and diagnostic services to public health systems. Many of IFC‘s clients who provide health services have prepayment schemes for health care in operation and this makes the most business sense. In some instances, by creating more low-cost local capacity, IFC health-provider clients have made it possible for health insurers to offer new products with lower premiums. Experience to date has shown that the business case for direct investment in stand-alone private health insurance does not exist to the extent envisaged when the 2002 IFC health strategy was devised. The few health insurance operations found to date that actually needed capital have needed only very small amounts, too small to be viable transactions. This segment of the sector is intended to be addressed by the health-sector financing facilities now being developed and in early implementation. For the balance, typically the health insurer is one arm of a larger insurer that is well capitalized from its other operations such as life insurance or is a subsidiary or joint venture of a well-capitalized foreign parent company. Indicator 1. Number of health PPP projects supported (Investment and Advisory Services); 2. IFC support for health insurance when it makes sense. Comment IFC supports PPPs in health that have the potential to expand and improve access to health services. The number of health PPP projects can be tracked and reported on an annual basis. Mgmt On the advisory side, as of 02/28/11, C3P (responsible for PPP transaction advice) had successfully closed 16 Comment 2011 health PPP transactions (3 in FY11 to date), with 8 active mandates and 10 more in the pipeline (active discussions with client governments). Each transaction is estimated to result in additional access of 100,000- 300,000 public patients annually (depending on the type of transaction) and private investment of $20-100 223 million. A new multi-donor advisory funding facility is expected to be established by end FY11, which will enable further scaling up. There will be even greater focus on IDA countries and targeted PPPs which reach the poor and enhance maternal/child health. C3P will also broaden its activities to include contracting for health services for communities linked with extractive industries and large agro-plantations. Mgt LOA 2011 Substantial Mgt Status 2011 Active (Rated) IEG Comment High PPP in health achieved. Support to health insurance by GBOPA. Progress to be monitored. 2011 IEG LOA 2011 Substantial IEG Status 2011 Active (Rated) Mgmt C3P has now secured $9.5 million in three-year funding for advisory work on health PPPs, almost exclusively for Comment 2012 IDA, including $5.25 million from DFID (UK), $1 million from Netherlands and $2.25 million from FMTAAS. This funding will enable a significant ramp-up. There has been one closing (Moldova) health PPP in FY12; we expect 3-4 more this FY. There are 10 active mandates, including two new ones in Benin and Nigeria (Cross River State). Potential mandates are well advanced for Rwanda, Uganda, and Southern Sudan. The Lesotho Hospital (landmark PPP) opened to acclaim in Oct/11. To ramp up our delivery capability, C3P is entering into framework contracts with 2 large firms with combined financial/health expertise. This will enable leveraging more delivery with the same number of IFC staff. Regarding health insurance we continue to promote advisory assistance to governments to help establish low-cost health insurance and contracting capability, as well as the use of GPOBA to help offset the costs of premia and the cost of the provision of care for the poor. However, we have not yet secured a mandate which specifically includes insurance. Finally, regarding health PPPs related to extractive industries, C3P is co-funding with DFID a major review of health delivery in mining communities and the identification of potential opportunities for use of health PPPs in mining areas, principally in Africa, We are also coordinating with SBA on this. We expect this to become a major business line for C3P health within the next 3 years Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment Progress made in generating deals in health PPP through C3P, but IFC has been facing challenges in supporting 2012 health insurance. Because of the continuous attempts in building PPP transactions in health, Substantial rating is justified (largely adopted but not fully incorporated into policy, strategy, or operations). IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) 224 8. PERU CPE Record STD 42 REC Study Short Peru CPE Study Date Number 01 Name (verified) Headline IEG Rec Develop a strategic focus around areas that further economic diversification and offer high potential employment, such as tourism, fisheries, and other manufacturing. Although Peru has some diversification in its economy, a central challenge ahead remains to further the economic diversification into areas that can generate employment and engage a broader share of the population in economic growth. Although tourism has developed rapidly, it remains confined around a few major attractions, with substantial unrealized potential in other areas. At present, there remains only limited value added to Peru‘s extensive natural resources in fisheries. Other manufacturing sectors that might exploit Peru‘s access to world markets through Free Trade Agreements (FTAs) and offer strong employment-generating potential also warrant close attention. An IFC focus on supporting private investment in such areas is likely to enhance its contribution to employment generation and to broadening the benefits of economic growth. Mgmt Response We welcome IFC-related recommendations. Supporting employment- generating industries, and lower tier and more regionally dispersed companies are areas in need of support. Our strategic focus under the current CPS has been trying to address some of those needs. IFC‘s focus under the current CPS includes: (i) working to reduce Peru‘s high informality in its economy with technical assistance; (ii) investing in and developing projects that address bottlenecks in the provision of basic services and physical infrastructure; (iii) investing in industries with strong competitive advantages such as agribusiness and tourism; (iv) extending to new beneficiaries access to SME, housing, and tertiary education finance and building local government capacity to deliver services; and (v) supporting environmentally and socially sustainable growth, notably in extractive industries. Several of our investments and advisory services support the manufacturing, agribusiness, and tourism sectors. One of our key selection criteria is employment generation. Indicator IFC's Peru country strategies support further diversification (i) To the extent there is strong market demand and role for IFC, explicit strategy on how IFC plans to develop business in sectors such as infrastructure (including renewable energy), agribusiness, tourism, fisheries, and manufacturing; (ii) number of projects in these areas Comment Mgmt IFC continued to diversify its IS (see below tables on project count) and AS operations in Peru. In Investment Comment 2012 Services, recent direct IFC investments cover a number of different sectors including, healthcare, personal consumer product manufacturing, education, hydro-electric generation, and natural gas distribution. Through an investment in a private equity company, IFC further diversified in other sectors such as ramp and air logistic, 225 home improvement retail, tiles and sanitary manufacturing and fishing. In addition, IFC supported sub-national initiatives like Lima Municipality (bus transportation system) and Arequipa Municipality (construction of highway), and public companies like Sedapal (the water and waste treatment company of Lima). IFC also supported the local private financial sector to help increase access to finance in underserved areas and low- income segments of the population. Chief among these activities was the assistance provided to two microfinance institutions to support their transformation into regulated financial institutions and expand their reach to provide access to finance to microentrepreneurs and low-income households. Through Advisory Services, IFC has contributed to generate economic diversification in regions other than Lima, as well as opened up opportunities for growth in labor intensive new sectors and reduction of informality. IFC‘s Revenue Management Program has provided technical support to over 30 municipalities in some of the poorest regions where extractive activity takes place, enabling them to increase public investment by approximately US$47 million. This investment generated local employment and benefited an estimated 265,000 people with access to improved services. In addition, approximately 800 rural municipalities throughout the country have received on-line technical support. IFC has also started providing support for improving business regulation procedures for key labor intensive sectors. In November 2011, IFC AS started working, in association with the most important regional stakeholders, on a project for improving the business climate for Tourism in Cusco Region. This initiative aims to reduce the transaction costs to start and operate businesses across all the tourism value chain in Cusco. The project seeks to improve the business climate, particularly for SMEs, by reducing some of the bureaucratic barriers for the private sector to operate formally, trade and invest in tourism. Number of Projects Percent of Projects 226 Balance as of Commitments Balance as of Commitments FY09 FY10 FY11 FY12** FY09 FY10 FY11 FY12** General Mfg 4 1 0 1 General Mfg 13% 11% 0% 14% Health & Ed 1 0 1 1 Health & Ed 3% 0% 14% 14% Agribusiness 3 0 0 1 Agribusiness 10% 0% 0% 14% Services 2 0 0 0 Services 7% 0% 0% 0% Infra / Sub-N 6 3 2 0 Infra / Sub-N 20% 33% 29% 0% FM 9 3 4 4 FM 30% 33% 57% 57% Oil / Mining 5 2 0 0 Oil / Mining 17% 22% 0% 0% Total 30 9 7 7 Total 100% 100% 100% 100% **Estimate **Estimate Mgt LOA 2012 Substantial Mgt Status 2012 Active (rated) IEG Comment IFC has made several positive initiatives to help Peru diversify its economy and realized investments in areas 2012 such as infrastructure, manufacturing, and the social sectors. Efforts to help realize more investment in the employment- generating manufacturing sectors should be sustained. IEG LOA 2012 Substantial IEG Status 2012 Active Record STD 42 REC Study Short Peru CPE Study Date Number 02 Name (verified) Headline IEG Rec Enhance efforts to work with lower tier and more regionally dispersed private companies. Although IFC demonstrated high additionality in the past by investing in emerging and risky sectors, improving conditions have reduced its potential value added among upper-tier companies. Although IFC‘s strategy to support access to finance through the local banking sector remains sound, several segments of the market—particularly smaller companies in the Sierra and Selva regions—are likely to remain too risky for the banking sector to reach in the medium term. A challenge therefore remains for IFC to help accelerate the goal of more broad-based participation in growth by supporting smaller and more regionally dispersed private companies. To date, IFC‘s use of equity instruments has been limited. Given its decreasing competitiveness with the local banking sector, greater use of equity instruments should be actively pursued. Mgmt Response One of IFC‘s main objectives is to reach underserved segments such as S MEs via credit lines and advisory services to local financial institutions, including in regions outside the capital city. Our ability to reach these segments depends also on improving the financial sector and business environment. As correctly pointed out in 227 the paper, several of these improvements have occurred mainly around the capital city, thereby limiting IFC and its potential clients‘ ability to invest in underserved areas. We have adapted our strategic focus. IFC has been partnering with local financial institutions to reach SMEs for both economic and regional diversification, and supporting employment generation. Working with ―Caja Municipales‖ we are reaching small municipalities. Our advisory services supporting SME formalization have a strong focus outside Lima. Furthermore IFC advisory services in extractive industries (revenue management mostly directed at smaller communities in rural areas) and tourism are reaching the provinces notably in the Sierras. Concerning equity, IFC will continue to pursue those opportunities where it can bring high value added and development impact. Indicator IFC's continued focus on the underserved is evident in its strategies and activities Comment Mgmt 1. Strategies Comment 2012 As part of the WBG Peru Country Partnership (CPS) for FY12-16, the IFC under the Sustainable Growth and Productivity Objective, expects to assist the GoP in areas related to business environment facilitation for SMEs and incorporation of local SMEs into the supply chain of extractive industries, which are mainly in Sierra and Selva region. In addition, within IFC‘s LAC Strategy approved by the Management Team is the Amazon Initiative, where we are seeking to protect natural habitats while promoting economic alternatives to deforestation in the Selva region and where a complete study of investment opportunities by an outside consultant has been completed. 2. Activities During the period FY10 - FY12 (committed plus highly probable on the pipeline) and from an investment point of view, IFC has supported financial institutions which focuses on SMEs that have a nationwide coverage (Costa, Sierra and Selva), generating lending to lower tier companies, and has invested directly (insurance company La Positiva) and indirectly (through Enfoca) in equity in second tier companies in the country. IFC also supported the local private financial sector to help increase access to finance in underserved areas and low-income segments of the population. Chief among these activities was the assistance provided to two microfinance institutions to support their transformation into regulated financial institutions and expand their reach to provide access to finance to micro- entrepreneurs and low-income households. 3. Progress According to INEI (the government official statistics office) during the period 2004-2010 - total poverty rates 228 were reduced from 48.6% to 31.3% (57.7% to 37.3% in Selva, 64.7% to 49.1% in Sierra, and 35.1 to 17.7% in Costa regions), suggesting that the poor have benefitted from growth and investments in the region. Mgt LOA 2012 High Mgt Status 2012 Active (Rated, to be made inactive) IEG Comment IFC has realized several important investments that support SME activity. Given the magnitude of the challenge, 2012 intensive efforts should be sustained to support economic activity in the Selva and Sierra regions where employment and business opportunities are scarce and poverty remains high. IEG LOA 2012 Substantial IEG Status 2012 Active (Rated, to be made inactive) 229 9. SAFEGUARDS Record STD 043 REC01 Study Short Safeguards Study Date (verified) Number Name Headline Revise the policy frameworks to harmonize thematic coverage and guidance across the WBG and enhance the relevance of those frameworks to client needs. IEG Rec IFC should: a. Strengthen the provisions on sustainability to address emerging issues, notably climate change and supply chains and their commodity certification. b. Develop more robust approaches to the implementation of the Performance Standards in FI projects, listed equities, and trade finance. c. Strengthen policies and practices on disclosure, including at the local levels. Mgmt Response a. Agreed. IFC has proposed changes through the ongoing review and update of its Sustainability Policy and Performance Standards, which will address climate change, supply chains, and biodiversity, among others. Management feels that the proposals put forward with regard to climate change—including the consideration of low-carbon technology options and resource efficiency, and to strengthen reporting on greenhouse gas emissions—are consistent with good business, and with good international industry practice. Timeline: Following Board approval of the updated Sustainability Policy and Performance Standards. Proposed changes to the Performance Standards include extending supply chain considerations to significant safety issues. Regarding certification, IFC is generally supportive of these schemes and has an active and ongoing engagement in a number of global commodity roundtables. Timeline: Following Board approval of the updated Sustainability Policy and Performance Standards. Disagreed: Management notes that the number of credible certification schemes is still limited, and that these schemes could disadvantage small-scale producers and suppliers, particularly those in developing countries. In addition, IFC clients have varying degrees of control of or influence over their supply chains. Indicator a. Provisions for supply chain risk management and climate change strengthened; Revised Sustainability Framework (SF) approved by the Board. b. (See also Recommendation 3 for Mgt Response and Indicator for IEG Rec ―b‖ above) c. (See also Recommendation 4 for Mgt Response and Indicator for IEG Rec ―c‖ above) Comment Mgmt a. Provisions of supply chain risk management and climate change started to be implemented from Jan 1, 2012. 230 Comment 2012 The revised Sustainability Framework has been approved by the Board on May 12, 2011. b. (See also Recommendation 3 for Mgt Response and Indicator for IEG Rec ―b‖ above) c. (See also Recommendation 4 for Mgt Response and Indicator for IEG Rec ―c‖ above) Mgt LOA 2012 High Mgt Status 2012 Active (to be made inactive) IEG Comment The revised Sustainability Framework that has been officially approved and is now in effect specifically 2012 addresses, inter alia, the issues of climate change, biodiversity, and supply chains. IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) Record STD 043 Study Short Safeguards Study Date (verified) Number REC02 Name Headline Enhance client capacity, responsibility, and ownership. IEG Rec IFC should:  Develop incentives for investment officers to share ownership of the Performance Standards and mainstream their implementation.  Use advisory services to build social and environmental management systems and implementation capacity, especially among small and medium enterprises, financial intermediaries, and clients in countries and sectors with weak environmental and social management.  Mobilize resources at appraisal for energy and clean production audits, using auditors with relevant sector knowledge.  Define areas of influence and requirements to better address supply chain risks and opportunities, particularly related to biodiversity and forestry, expanding the application of material biodiversity along the supply chain for suppliers. Mgmt Response a. Agreed/Ongoing. With regard to staff incentives, these are constantly evolving and incentives related to environmental and social issues have become more prominent in recent times, especially in areas where environmental and social performance is a core aspect of project sustainability. IFC Management has reinforced environmental and social issues as a shared and core agenda and will hold staff accountable for this. Environmental and social due diligence is required for success in approval of investments. IFC will consider opportunities to include environmental and social aspects in the performance management process. b. Agreed. With regard to advisory services, IFC will work to strengthen the capacity of select clients to develop and manage their environmental and social management systems through a mix of tools and approaches, including the selective and strategic use of advisory services. IFC will also use other approaches, as appropriate, including the use of environmental and social specialists who engage with clients in developing action plans and use supervision to verify and support implementation of environmental and 231 social standards; and country-based expertise, especially in middle-income countries. Timeline: Work with advisory services has been initiated and a strategic approach is expected before the end of fiscal 2011. c. Agreed/Ongoing (energy and cleaner production audits). IFC includes energy/cleaner production audits as part of appraisal or ongoing improvement of clients‘ operations when deemed useful and appropriate. Disagreed (resource mobilization approach). IFC disagrees with the proposed resource mobilization approach. Resources are mobilized through different avenues not linked to timing of appraisal and may include funding of audits directly by clients. d. Agreed. IFC requirements on supply chains apply to all sectors with a focus on the highest risks, such as child labor, forced labor, and clearing of critical habitats. The proposed changes to the Performance Standards expand this to include significant safety issues. IFC has included provisions under PS1, PS2, and PS6 to ensure an adequate assessment of supply chains is undertaken as part of appraisal. Timeline: Following Board approval of the updated Sustainability Policy and Performance Standards. Indicators a. Environmental and social issues reinforced as a shared and core corporate agenda. Environmental and social issues are integral part of regional/sectoral strategies; Climate Business Group department established within IFC, climate business related investments targets for FY13, 30% of IFC staff (E+) trained as part of Corporate Sustainability Learning Program. b. AS program supporting implementation of the Sustainability Framework for real sector developed by Q1, FY13. AS program for application of revised SF for FIs developed by Q2, FY13 and available to external users. c. Through PS3 requirements, IFC includes energy/cleaner production audits for clients as deemed appropriate, by Q4, 2011. Recruitment of additional resource efficiency specialist by Q4, FY12. d. The scope of supply chain strengthened is the revised Sustainability Framework. Develop and implement E&S Risk Guidelines for supply chain with emphasis on agro-commodities, by Q4 FY12. Comment Mgmt a. Environmental and social considerations are an integral part of annual regional/sectoral strategies that are Comment 2012 presented to IFC Senior Management. These strategies feed into the IFC Corporate Strategy (the Road Map Paper). The most recent strategy presentations were in Dec 2011. Climate Business Group department has been established in September 2010. Climate business related investments targets established and tracked annually, reporting on FY13 performance to be completed by early FY14. Corporate Sustainability Learning Program has been launched on Feb 13, 2012. b. Ongoing activities in finalizing AS program supporting the Sustainability Framework for real sector clients. An initial framework has been put in place for both real sector and FI clients. c. Ongoing energy audits for clients, additional resource efficiency specialist to be recruited by Q4, FY12. 232 d. The scope of supply chain has been strengthened in the revised Sustainability Framework, approved by the Board on May 12, 2011. E&S risk guidelines for supply chain developed and will start to be implemented by Q4 FY12. Mgt LOA 2012 Medium Mgt Status 2012 Active (Rated) IEG Comment IFC‘s progress is noted and progress on the indicators will be tracked by IEG. 2012 IEG LOA 2012 Medium IEG Status 2012 Active (Rated) Record STD 043 Study Short Safeguards Study Date Number REC03 Name (verified) Headline Revise guidelines, instruments, and incentives to strengthen supervision arrangements. IEG Rec IFC should:  Enhance the supervision of financial intermediaries at the subproject level by developing clear guidelines for applying the Performance Standards at the subproject level and by adopting a systematic approach to environmental and social specialists‘ site visits to selected subprojects.  Use loan covenants, including Conditions of Disbursement to enforce compliance with environmental and social requirements and reporting if the clients lack commitment and are continuously out of compliance. Mgmt Response a. Agreed/Ongoing. IFC has been strengthening the oversight of its investments in and through financial intermediaries at the portfolio, company, and subproject levels. This approach has been developed and is being implemented in accordance with a risk-based approach, which is intended to deploy resources efficiently where the risk is highest and/or performance is poorest. IFC provides guidance on the application of Performance Standards at the subproject level through its ongoing engagement with clients. There is an ongoing multiyear effort to ensure that IFC‘s approach continues to be suitable. The approach to the subproject level supervision will be codified in IFC‘s Environmental and Social Review Procedure. b. Agreed/Ongoing. The use of loan covenants to support compliance with environmental and social requirements is a standing practice. IFC has several instruments to support client compliance. These include specific provisions in the loan agreement or legal documentation and action items linked to disbursement and specific deadlines. There are covenants in legal documentation through which IFC monitors compliance, including a ―policy put‖ in some cases, whereby noncompliance of policy provisions would trigger the option for IFC to sell its shares. Indicators a. FI interpretation note clarifying application of IFC‘s requirements for FIs by Q2, FY12 developed addressing application of relevant requirements including performance standards in cases of project finance and long- term corporate lending at the subproject level. Policy requirements for FI supervision in the Sustainability 233 Framework (Sustainability Policy and ESAP) strengthened. b. Use of loan covenants to enforce compliance with environmental and social requirements is long-standing practice in IFC. Reporting on quarterly basis to CRM on portfolio risks for both real sector and FIs including high risk projects. Comment Mgmt a. FI interpretation note developed by Q2, FY12. The policy requirements for FIs in the Sustainability Framework Comment 2012 have been strengthened and approved by the Board on May 12, 2011 b. IFC uses loan covenants to enforce environmental and social requirements. Ongoing quarterly reporting to CRM on E&S risks for both real and FI sector. Mgt LOA 2012 High Mgt Status 2012 Active (to be made inactive) IEG Comment IEG notes the progress on the Interpretation Note for Financial Intermediaries in providing guidance for FI 2012 categorization (FI-1, FI-2 and FI-3) and for the FI to develop a Social and Environmental Management System, appraise and monitor subprojects and conduct external communication. However, for at least A-category (high risk) subprojects IFC should also request the FI to provide IFC with reports on compliance with Performance Standards and ESAP. It is still unclear how the future Environmental and Social Review Procedure will address environmental and social specialists‘ site visits to selected subprojects. IFC should enforce the environmental and social loan covenants if the clients are continuously out of compliance. IEG LOA 2012 Medium IEG Status 2012 Active Record Number STD 043 REC04 Study Short Safeguards Study Date Name verified) Headline Strengthen safeguards monitoring, evaluation, and completion reporting. IEG Rec IFC should:  Disclose project-level environmental and social information from monitoring and supervision reports.  Make use of independent, third-party, or community monitoring and evaluation for its projects, particularly for projects with involuntary resettlement and higher-risk FI and agribusiness projects. Mgmt Response a. Agreed (disclosure of some project-level information). IFC is reviewing its Disclosure Policy to determine where it is most appropriate to make modifications to the policy and to practices throughout the project life 234 cycle. Disagreed (disclosure of all information). However, this will not include disclosure of all environmental and social reporting from clients since there has to be a balance between client confidentiality and disclosure. IFC will continue its practice of holding its clients responsible for reporting to the local community. Timeline: Following Board approval of the updated Disclosure Policy. b. Agreed (community and select independent/third-party monitoring and evaluation). IFC Management will explore how to strengthen community engagement, participatory monitoring and how, in selected high-risk cases, third party monitoring or advice can be incorporated. Timeline: Following Board approval of the updated Sustainability Policy and Performance Standard. Disagreed (independent/third party monitoring across the board). IFC Management does not see third-party monitoring as an approach that should be appropriate for all projects, but rather one that may be considered in selected higher-risk situations. Since a key priority is to strengthen client capacity and ownership, third-party monitoring should be seen in that context, not just in terms of independent verification. Indicators a. Enhanced post-Board reporting/disclosure requirements in IFC‘s Access to Information Policy; Policy approved by the Board. Disclosure of information from the Environmental and Social Action Plan (ESAP) for approved projects and standard E&S indicators tracked in DOTS. b. Enhanced community engagement and monitoring in the Sustainability Framework; Sustainability Framework approved by the Board. Third party monitoring or advice in selected high risk cases applied as per PS1 requirements. Comment Mgmt a. Based on IFC‘s Access to Information Policy approved by the Board on May 12, 2012, most significant Comment 2012 information from ESAP and E&S DOTS indicators are being disclosed for all projects with CRM on or after Jan 1, 2012. b. Application of PS1 requirements on third-party monitoring for high risks cases is being applied to all projects with CRM on or after Jan 1, 2012. Mgt LOA 2012 High Mgt Status 2012 Active (To Be Made Inactive) IEG Comment a) Medium (partially implemented in AIP and need further tracking of projects); 2012 b) Substantial (almost fully implemented in PS1 and AIP 2012, but high risk FI subprojects need to be addressed) IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) 235 10. AGRIBUSINESS Record Number STD 44 REC01 Study Short Name Agribusiness Study Date (verified) Headline Synergies and complementarities. In the areas that drive productivity, such as irrigation and drainage, agricultural research and extension, access to credit, access to land, transport infrastructure, and the policy environment, complementarities and synergies are key drivers of effectiveness. To take better advantage of these complementarities: IEG Rec a. Step up IFC‘s engagement in Sub-Saharan Africa, including support for public-private partnerships and adapting the integrated trader-processor model for more effective use with small-to-midsize indigenous companies in the agriculture-based economies. b. Set up a knowledge network that links agriculture and agribusiness supply-chain specialists across the World Bank Group to strengthen communication and collaboration among sector departments within the Bank and IFC, as well as across the World Bank Group. Mgmt Response a. Ongoing/Agree IFC‘s business model in the Region is evolving to be more active in IDA countries and IFC is extending its reach to small-scale farmers and small and medium enterprises through: (i) direct investments in larger companies with a significant development reach to farmers and SMEs; (ii) indirect financing of medium-size companies and cooperatives through financial intermediaries; and (iii) indirect financing of smaller size agribusiness farms and enterprises through risk-sharing facilities with financial intermediaries. IFC now has an agribusiness anchor in Africa with a dedicated team of 9 full-time and 4 part-time staff, which is double that of the previous year, and IFC has more than doubled its investments in African agribusiness in the past three fiscal years, reaching a record $270 million in fiscal 2010. b. Ongoing/Agree Management will broaden the existing informal World Bank Group thematic group on food safety to include agribusiness supply-chain specialists in the Bank and IFC. Bank-IFC collaboration on agriculture has increased since the 1998– 2008 IEG review period. Recent examples of joint work include: (i) preparation of the World Bank Group Agriculture Action Plan 2010–2012; (ii) inclusion of both a public and private sector window in the recently launched Global Agriculture and Food Security Program (GAFSP); (iii) the Responsible Agro-Investment (RAI) initiative, including the recently launched RAI Toolkit, and; (iv) preparation of a new Bank Group framework for engaging in oil palm. In addition, there is increasing Bank- IFC in-country coordination. Examples include: (i) a review of smallholder participation in the Liberian rubber sector; (ii) possible interventions in the cocoa sector in Côte d‘Ivoire; (iii) commercial agricultural development in Ghana; iv) a first loss agribusiness finance facility in Cambodia; and (v) an agribusiness logistics study in the Philippines. Indicators 236 a. See below  Growth in volume/number of projects in Sub Saharan Africa  Increase in volume/number of wholesale financing of small and medium scale farmers through large agribusiness companies, traders, financial institutions (including trade finance) , e.g., warehouse receipt and trade financing, agriculture financing, infrastructure, etc. b. See below:  Joint Agribusiness Strategic Action Plan FY12-14  Joint World Bank-IFC Agriculture Action Plan FY13-15 Comment Mgmt Comment a. See below 2012  After achieving a record level of $270 million in FY10, IFC‘s annual investment commitments in African agribusiness is now averaging double 2008 levels and are expected to exceed $500 million in FY12.  IFC’s engagement in Sub-Saharan Africa (AFR) going forward. IFC‘s 2012-14 Agribusiness Strategic Action Plan (ASAP) provides an updated framework for IFC‘s engagement in Sub-Saharan Africa (SSA). In recognition of the importance of the sector and its smallholder-based structure in most African countries, IFC is adapting its business model to step up its involvement and offer relevant financial products and services, including: (i) direct investments in larger companies with a significant development reach to farmers and MSMEs; (ii) indirect financing of medium sized companies and cooperatives through financial intermediaries; (iii) short term commodity-based lending through its Global Warehouse Finance Program (GWFP); (iv) trade finance to support commodity export and critical commodity import needs during the financial crisis through the Critical Commodity Finance Program (CCFP)  As shown below, IFC has significantly increased its wholesale financing activities (financial institutions, traders, new products) in Sub Saharan Africa (SSA) from a negligible level in the IEG review period (1998 to 2008), As of end of 3rd quarter of FY12, wholesale financing activities represent 40% of overall agribusiness financing in the region and are expected to almost 60% by the end of the year. 237 IFC Agribusiness Wholesale Financing Commitments in Sub Saharan Africa CAF Agri Wholesaling incl. CAF Agri Mobilization Wholesaling as % of (USD Million) CAF Agri Total FY10 92.8 34% FY11 73.0 38% FY12-as of March 31, 2012 177.5 40% FY12 - Projected 398.5 59% b. See below  Collaboration among sector departments within the Bank and IFC. Management has broadened informal links between the World Bank and IFC. In addition to the examples provided in the original response, new examples of joint Bank-IFC work include: (i) Preparation and implementation of the IFC Agribusiness Strategic Action Plan FY12-14, (ii) Preparation of the forthcoming World Bank Group Agriculture Action Plan:2013-2015; (iii) Completion of The World Bank Group Framework and IFC Strategy for Engagement in the Palm Oil Sector; (iv) Regular joint meetings on the IFC‘s agribusiness strategic action plan and implementation practice which includes Bank staff and staff from both the advisory and investment arms of the IFC; (v) Cross institution learning between staff from the agriculture and private sector development sectors of the Bank and staff from IFC in the development of several new Bank agriculture commercialization projects (for example in Ghana, Burkina Faso, Senegal and Tanzania) (vi) Inclusion of both a public and private sector windows in the recently launched Global Agriculture and Food Security Program (GAFSP); (iv) The Responsible Agro-Investment (RAI) initiative, including the recently launched RAI Toolkit; and (v) Preparation of the Ukraine Agriculture Investment Policy Note. Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment On a) IEG notes the progress IFC Agribusiness Wholesale Financing Commitments in Sub Saharan Africa and 2012 looks forward to verifying the projected numbers in coming years. It would be useful to know how many small and medium scale farmers have been reached through this wholesale approach. On b), IEG finds substantial progress and agrees with IFC‘s self-rating. 238 IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) Record STD 44 REC02 Study Short Agribusiness Study Date Number Name (verified) Headline Knowledge and capacity building. Experience points to the importance of capacity and how analytical work can highlight issues and raise awareness—particularly when capacity is weak: IEG Rec a. Establish mechanisms to confirm ex ante if project M&E frameworks are adequate, with clear, relevant, and realistic objectives; thorough cost-benefit analysis; appropriate indicators; and adequate baseline data. Mgmt Response a. As for IFC, the tracking of financial, economic and environmental outcomes has progressively been built into the appraisal of new investments and monitoring of the portfolio since IFC deployed its Development Outcome Tracking System (DOTS). For new investments, coverage of IFC‘s standardized agribusiness indicators is at 100 percent. Monitoring and evaluation of development impacts is an area of priority for IFC, and indicators will continue to be reviewed and improved continuously for their relevance. In particular, agribusiness indicators will be adjusted as part of IFC‘s forthcoming implementation of its Corporate Development Goals. Indicators a. Revision of Agribusiness DOTS indicators to include additional dimensions (Productivity, Food Security, Sustainable Land Management) Comment Mgmt  At IFC, monitoring and evaluation of development impacts is an area of priority. Agribusiness indicators Comment 2012 were revised and finalized in 2011 as part of IFC‘s forthcoming implementation of its IFC Development Goals. New indicators include productivity, food availability (as proxy for food security) and sustainable land management. Guidelines for identifying and tracking these indicators were provided to staff. Starting CY2012, Board papers of agribusiness projects already reflect these new indicators, if relevant and applicable.  IFC also continues to track the impact of its investments on direct and indirect employment by gender through its DOTS system. Recognizing the potential to leverage the impact of its clients‘ influence over supply chains IFC is also initiating case studies to identify the business case for gender sensitivity. 239 Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment Progress on the indicator is noted and rated as substantial. IEG looks forward to seeing evidence/actual data for 2012 these new indicators in the coming years of tracking this recommendation. IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) Record STD 44 REC03 Study Short Name Agribusiness Study Date (verified) Number Headline Efficiency and sustainability. The impact of increased resource flows into agriculture will depend on the efficiency of resource use and the financial, social, and environmental sustainability of investments: IEG Rec a. Ensure that gender concerns are adequately mainstreamed and monitored in World Bank and IFC agriculture operations. b. Expand the application of IFC Performance Standards to material biodiversity and other environmental and social aspects along the supply chain for primary suppliers (and for secondary suppliers to the extent the client has leverage), and enhance IFC support to the development and application of internationally accepted commodity certification systems. Mgmt Response a. Gender is mainstreamed across sectors and industries through explicit requirements in IFC‘s Sustainability Policy and Performance Standards (PS). The social assessment process required in PS1 provides guidance on disaggregating stakeholder groups by key social identities, including gender, and ensuring that any potential adverse impacts are addressed. Throughout the PS, there is explicit reference to addressing risks, ensuring opportunities, and providing appropriate consultation for women. As examples, PS2 addresses non- discrimination; PS4 addresses gender-disaggregated aspects of health and vulnerability; PS5 addresses women‘s tenure and livelihoods in cases of resettlement; and PS7 & and PS8 both include specific attention to women‘s views in decision-making processes. As part of its PS review and update, IFC proposes to strengthen client‘s requirements not to employ trafficked persons as part of a revised version of the Performance Standards (version 1). IFC has been tracking the implications of its investments on employment by gender through its DOTS system. It agrees with IEG that more can be done to mainstream and monitor gender concerns in agribusiness. This will be done as part of the next revision of the sector‘s standardized indicators. b. Management agrees that IFC policies did not contain a requirement to examine supply chains in the early years of the review period; this was only addressed in the IFC Performance Standards (PS) which became effective April 30, 2006. PS 1 requires that the impacts associated with supply chains be assessed in two cases: 240 (1) where the resource utilized by the project is ecologically sensitive (e.g., wood products) or (2) in the case where low labor cost is a factor in the competitiveness of the item supplied (e.g., textile and some agribusiness activities); then child and forced labor should be examined for supply chains. IFC‘s approach to supply chains has been to focus client actions on the most immediate and serious risks in their supply chains—such as child labor, forced labor, and potential clearing of critical habitats. As part of its PS review and update, IFC proposes to: (i) strengthen its supply chain assessment methodology as part of project appraisal; (ii) make changes to the PS by adding significant occupational health and safety issues as a new risk factor to be considered in the supply chain assessment; and, (iii) to continue supporting certification schemes, through investments and advisory services, including engagement in a number of global commodity roundtables. The ongoing review and update of IFC‘s Sustainability Policy and Performance Standards will address climate change, supply chains, and biodiversity issues among others. IFC is generally supportive of international certification schemes and has an active engagement in a number of global commodity roundtables. In addition, IFC has been actively working with its clients (especially producers and traders) to increase the traceability and certification of their products in their respective supply chains. Efforts in the coffee, cocoa, and cotton sectors, among others, are under way to develop corporate policies that emphasize purchasing of sustainable products, and compliance with national environmental legislation and IFC‘s PS requirements. Indicators a. Adoption of revised Performance Standards with added emphasis on gender considerations – Continued tracking of the implications of IFC investments on employment by gender through DOTS and case studies. b. Adoption and implementation of revised Performance Standards with added emphasis on supply chain considerations. Comment Mgmt a. Gender Considerations. Comment 2012  IFC‘s revised Performance Standards (PS) have been implemented and effective since January 1, 2012. The revised Performance Standards have explicit gender dimensions as follows: PS1 – Gender perspective is included in the context of consultation and participation: Specific requirement: Clients are required to conduct an informed consultation and participation 241 process that capture women‘s views, concerns and priorities about impacts, mitigation mechanisms and benefits of the project. PS2 - Gender issues are covered in the perspective of non-discrimination and equal employment opportunity, occupational health and safety and forced labor. Specific requirements: Through non-discrimination and equal opportunity requirements in PS2, IFC‘s clients are required to take measures to prevent and address harassment, intimidation, and exploitation, especially in regard to women. Also in terms of Occupational Health and Safety requirements, clients are required to take into account specific threats to women. In terms of forced labor, PS2 requires from clients to not employ trafficked persons with a specific reference to women. PS5 – Gender requirements are addressed as part of community engagement and involuntarily resettlements. Specific requirements: As part of community engagement requirements, PS5 requires clients to ensure that women‘s perspectives are obtained and their interests factored into all aspects of resettlement planning and implementation. In cases of involuntarily resettlements, clients are required to adopt measures that would provide equal protection for women. PS7 – Gender is included in the perspective of land and natural resource assessment and the management and use of project resources. Specific requirements: In cases where clients are proposing to locate a project on and/or commercially develop natural resources, as part of the assessment of land and natural resource, PS7 explicitly requires from clients to develop a gender inclusive assessment and specifically consider women‘s role in the management and use of the project resources.  IFC is tracking direct employment impact - and indirect employment when feasible - of its investments as part of its DOTS system. The experience shows, however, that it is both difficult technically to track indirect employment on a consistent basis, and that there is limited strategic value in the numbers themselves. It is more important to identify business models that: (i) promote opportunities for women by increasing market access and finance for women entrepreneurs; (ii) reduce gender-based barriers in the business environment; and (iii) create business opportunities to 242 promote improved working conditions. To that effect, under the Women Business Initiative, IFC will identify and assess in its agribusiness investments portfolio client companies that have established gender-sensitive business models to draw lessons and disseminate. b. Supply Chain and Certification Considerations  As part of IFC‘s revised Performance Standards, IFC has: (i) strengthened its supply chain assessment methodology as part of project appraisal; (ii) added significant occupational health and safety issues as a new risk factor to be considered in the supply chain assessment; and, (iii) continued supporting certification schemes, through investments and advisory services, including engagement in a number of global commodity roundtables.  IFC is generally supportive of international certification schemes and has an active engagement in a number of global commodity roundtables. In addition, IFC has been actively working with its clients (especially producers and traders) to increase the traceability and certification of their products in their respective supply chains. Efforts in the coffee, cocoa and cotton sectors, among others, are underway to develop corporate policies that emphasize purchasing of sustainable products, and compliance with national environmental legislation and IFC‘s PS requirements. Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment a) IFC‘s revised Performance Standards (PS) have been implemented since January 2012 and justify a substantial 2012 rating. IEG looks forward to the dissemination of lessons on the implications of IFC investments on employment by gender as part of IFC‘s Women Business Initiative in the coming years. On b) IEG will keep tracking IFC‘s implementation of the revised Performance Standards with regard to supply chain considerations. IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) 243 11. CLIMATE CHANGE Record STD45 REC 01 Study Short Climate Change II Study Date Number Name (verified) Headline IEG Rec  The World Bank and IFC should— Create incentives and mobilize resources to support effective pilot, demonstration, and technology transfer projects that have a clear logic of demonstration and diffusion. This will include mobilizing Global Environment Fund and other concessional funds to mitigate World Bank borrower risk; reshaping incentives for staff and managers; providing adequate resources for the design and supervision of complex projects; and making available specialized expertise in technology transfer and procurement through a real or virtual technology unit. Mgmt Response  IFC‘s clean-tech investment practice will be housed in the newly-created Climate Business Group. CLEANTECHNET is a practice group that meets virtually and in person to share knowledge and issues in the technology space. Indicators  Climate Business Group's CleanTech Unit strategy for promoting clean technology in IFC investments in place  Notable increase in IFC's investments with clean tech over the next three years  Staff in Climate Business Group's CleanTech Unit to be recognized for demonstrated special efforts in technology transfer, diffusion, and procurement Comment Mgmt  Cleantech Investment Unit (was established in early FY 11 as a complement and catalyst to IFC‘s mainstream Comment 2012 departments. Current team consists of 10 investment professionals.  In October 2011, a cleantech strategy roadmap was presented to the Board promoting cleantech innovations and facilitating North/South-South transfer of innovative cleantech investments.  Established a $60 million Cleantech Innovation Facility– a facility for higher risk, early innovative investments with high development impact that includes a $15 million of investment for GEF which helps protect IFC‘s downside.  Closed 7 transactions as of March 2012 (over $80 million in commitments)  To align staff and managers incentives to promote cleantech investment across mainstream industry groups, Cleantech Investment unit has taken to a 100% shadow booking system, whereby the booking and portfolio management falls entirely to the Industry Departments. This allows IFC to take on a thematic approach towards cleantech, enabling better alignment of interest across IFC investment practices to catalyze early 244 stage investment across sectors. Mgt LOA 2012 Substantial Mgt Status 2012 Active (Rated) IEG Comment The CleanTech investment unit is consistent with the Climate Change II evaluation‘s call for a more venture- 2012 capital-like approach to climate mitigation. It has been actively placing investments, many of which have a clear logic of diffusion (e.g. through support of a new technology which seeks to achieve significant market penetration via lower prices or better performance) or via demonstration (for instance of technologies suitable for municipal procurement, where successful demonstration in one city sparks replication in others). There is room to further work on complementing firm-specific investments with industry-level support for diffusion, as envisioned in the CleanTech strategy.. IEG LOA 2012 Substantial IEG Status 2012 Active (Rated) Record STD 45 REC02 Study Short Climate Change II Study Date Number Name (verified) Headline IEG Rec  The WBG should—Place greater emphasis on large-scale energy efficiency scale-up, as measured by energy saved and generating capacity avoided. This includes support for efficient lighting and exploring the scope for accelerating the global phase-out of incandescent light bulbs. It includes continued and expanded support for reductions in transmission and distribution losses. And it includes proactive search by IFC for large-scale, catalytic investments in energy efficiency. There is scope to coordinate World Bank support for demand-side energy efficiency policies with IFC support for more efficient manufacturing and more efficient products. Mgmt Response  IFC intends to increase its climate-related lending from 10 percent of annual commitments in fiscal 2009 to 20–25 percent in fiscal 2015, and will undertake a proactive search for suitable investments. Energy efficiency is expected to be a significant contributor to meeting this target. IFC is piloting a set of approaches for estimating avoided emissions associated with real sector investments. IFC agrees with the potential for investments in manufacturing of more efficient products and is actively seeking such opportunities, having made several such investments in fiscal 2010. Indicators More than $3 billion in annual climate business, with at least 20% of its annual long-term finance (LTF) and at least 10% of its annual Trade and Supply Chain (TSC) commitments to be climate friendly by FY15. The target is a percentage of volume and thus the $3 billion figure is dependent on actual growth in total lending. Comment 245 Mgmt  IFC continues to mainstream climate considerations into all of its activities across all industries and regions. Comment 2012 In FY11, IFC committed $1.7 billion in climate-related investments, representing 14 percent of own-account volumes (13 percent in FY10).  IFC targets and tools and methods for carbon footprinting and measurement of carbon avoidance have evolved with experience since clean energy definitions and targets were first announced in 2004 and climate specific measures were first applied to real sector investments in 2009. The current approach continues to evolve based on operational experience and new needs for development indicators. Mgt LOA 2012 Medium Mgt Status 2012 Active (Rated) IEG Comment IEG recognizes IFC‘s work on carbon footprinting and its continued significant investments in energy efficiency. 2012 However, neither the comments nor the indicator respond to IEG‘s recommendation, which focuses on quantification of energy savings, and expression of targets in results-oriented terms: energy savings rather than commitments. The carbon footprinting work could provide a foundation for quantifying energy savings. Evidence on large-scale catalytic investments in energy efficiency was not presented. IEG LOA 2012 Medium IEG Status 2012 Active (Rated) 246 Draft 2012 MIGA Management Action Record (MAR) 247 IEG FY12 Responses to and Ratings of Active Recommendations FINALIZED APRIL 14, 2012. IEG CONTACT: BIDJAN NASHAT (X.30779) IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) I. Strategy Recommendations #1. Strengthen and Align Management’s FY12 Assessment Medium Incentives to MIGA’s Strategic  STATUS ASSESSMENT: COMPLETED While overall MIGA has taken some and Operational Goals.  Management continues to focus on driving new steps to implement this MIGA needs to put in place a business towards its strategic priority areas recommendation and the alignment of suitable performance (SPAs). FY12 results to date demonstrate strong business with the Strategic Priority management system that aligns alignment between new guarantees issued and Areas seem to improve, the staff incentives to the strategic priorities: as of mid-February 2012, 57% shortcomings identified in the 2010 achievement of MIGA‘s of MIGA‘s new projects address at least one report have not been addressed corporate goals and priorities, priority area, and account for 78% of the volume according to the Management and enables the Agency to of new issuance. comments. It remains unclear, if the deliver its targeted  These results stem from the considerable performance targets refer to Definitive development results and emphasis that has been placed on strengthening Applications only or to actual Board- business volume. An MIGA‘s business development capacity – efforts approved and signed deals. appropriate incentive system which have included creating a new Vice Furthermore, the Management aligned to delivering ―value- President level position to oversee all operational comments are silent about linking driven-volume‖ that will fully activity and the introduction of two regional hub performance targets to development support the implementation of presences (Asia and Europe) to get staff closer to outcome performance, an important 25IEG's assessment of the Status of Implementation is based on information provided by MIGA under the column ―MIGA Management Assessment of Status of Implementation‖. RATINGS: ―Completed‖= MIGA has fully adopted the recommendation or broadly incorporated and mainstreamed into policy, strategy, and/or operation s; ―Substantial‖ = MIGA has taken steps to adopt/incorporate the recommendation into policy, strategy or operations to a substantial degree; ―Medium‖= MIGA has taken some steps to adopt/incorporate the recommendation into policy, strategy or operations, but not to a signif icant degree (or not in key areas); ―Low‖= MIGA’s actions and/or plans for adoption of the recommendation are in a very preliminary stage. 248 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) MIGA‘s next strategy FY12-15 new clients. enabler for value-driven volume. IEG should be in place by the start  MIGA‘s underwriting department is organized in rates the implementation status of that strategy period. To that a structure that aligns well with the SPAs, and medium. end, MIGA is strongly sector teams and underwriters have performance encouraged to assess whether targets (both volume and number of projects) its current performance against which they are assessed through the OPE management system and the process. incentives embedded in it is  MIGA is able influence the profile of its portfolio compatible with its corporate through its marketing and business development goals and commitment to outreach which is carefully directed towards delivering a far higher and priority areas. Business development missions, growing volume of conference attendance and marketing events are developmentally sound all designed with this in mind. guarantees, and based on that assessment, take the necessary actions so that a suitable incentive system is in place that fully supports achievement of the Agency‘s business volume and development effectiveness goals. IFC‘s experience in aligning staff incentives with its institutional goals of delivering business volume and development outcomes may be relevant to MIGA‘s efforts to develop an incentive system that supports its ―value drive volume‖ goals. (2010 IER) 249 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) #2. MIGA needs to underpin Management’s FY12 Assessment Medium its engagement in conflict-  STATUS ASSESSMENT: MEDIUM MIGA‘s Conflict Affected and Fragile affected countries with a game  In MIGA‘s FY12-14 Strategy, providing Economies Facility (CAFEF) will be an plan. guarantees into conflict affected countries was important element in its game plan Such a game plan would selected as one of MIGA‘s four strategic priority for engaging in conflict-affected enable MIGA to enhance its areas. This means that MIGA will place internal countries. However, elaboration and development effectiveness in priority on underwriting such projects and will Board approval of the facility are still conflict-affected countries and pay special attention to developing new business pending. would allow the Agency to opportunities in this area. Results in this area are utilize its administrative broken out and publicly reported. The new special module on budget and capital more  MIGA is continuing to develop its CAFEF facility guaranteeing projects in conflict effectively. This includes for the providing guarantees into conflict-affected country that is being incorporated developing some staff with countries and fragile economies in order to be able into the Agency‘s staff learning specialist knowledge and to extend the Agency‘s reach and effectiveness. program are encouraging indication expertise of the issues and The proposed facility relies heavily on leveraging of implementing the needs of operating in a conflict partners and in this context MIGA is maintaining recommendation. environment; aligning MIGA's a close dialogue with a number of key donors who Hence IEG rates the implementation business functions--such as are interested in being involved. The external status medium. marketing, business environment will play a causal role in development, client aftercare, determining the timing and level of donor and risk management--to participation.. Management is continuing to push effectively support this forward with the planning meanwhile. business; and assessing the  MIGA has strengthened links to OCCPS in the financial implications of its context of developing the CAFEF concept. engagement in conflict-affected  Historically, MIGA has underwritten projects in countries. MIGA may also post-conflict environments, and there is good staff wish to assess its project-level experience in this area, both on the underwriting risk management approach side as well as from the perspective of the country (i.e., contract exclusions) to risk analysis. 250 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) assure itself of the consistency  MIGA has initiated a special module on of its stated strategic priority guaranteeing projects in conflict country that is and its actual being incorporated into the Agency‘s staff implementation.(2009 IER) learning program.  MIGA has hired a consultant in the West Bank & Gaza to represent the Agency and identify potential new business in this high priority destination. #3. Should MIGA decide to Management’s FY12 Assessment Not rated until Board approval of pursue Trust Funds in the CAFEF.  Management agrees that to the limited extent future [to supports its post MIGA works with and through Trust Funds, conflict engagement], it every effort should be made to ensure they are should enhance their efficiency and effectiveness. efficient and effective.  Any Trust Funds that MIGA develops going This includes implementing forward, including the proposed CAFEF, will multi-country and un-tied draw on lessons of experience from the World Trust Funds with broad and Bank. flexible mandates that have no  MIGA does not expect to introduce any other new restrictions with respect to the Trust Funds at this time. eligible investments or investors. Trust Fund arrangements should be underpinned by realistic objectives (including with respect to expected demand) and a results framework. (2009 IER) 251 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) II. Development Impact and Effectiveness Recommendations #4. Apply Lessons from (Self-) Management’s FY12 Assessment Substantial Evaluations to Future  STATUS ASSESSMENT: SUBSTANTIAL Management‘s activity in this area Guarantee Projects.  Management is working with IEG to design a new shows that MIGA has embraced the MIGA needs to ensure that training program for staff, based on evaluation concept of learning from evaluation lessons distilled in evaluation work, which will become a routine module in the and self-evaluation. Once more self- and self-evaluations are semi-annual MIGA Learning Week. evaluations are available the learning applied to new operation by  MIGA Self-Evaluation: MIGA is only in year 3 of experience will become richer and means of an effective feedback piloting self-evaluation. MIGA is learning how to have more impact on MIGA‘s future loop. To that end, MIGA needs do good evaluations by doing them. This has guarantee projects. to include lessons learned from entailed heavy investment in program design, past projects in MIGA‘s training, etc. Underwriting Paper, develop a  Dissemination of results remains a priority. To system that allows lessons date, MIGA has discussed early learnings of the 6 distilled from evaluations to be completed evaluations in departmental briefings, easily accessed by all BBLs, workshops on findings as well as in early underwriting teams; screening meeting for new projects. incorporate ―lessons learned‖  As the stock of self evaluations grows, it will be into the management review appropriate to distill common lessons. In this process for projects; and regard, MIGA will employ a range of tools, systematically disseminate including short synoptic notes, searchable data evaluation findings and bases, and in house workshops. lessons. More generally, MIGA  IEG Evaluations: MIGA will continue to pay needs to establish a culture of careful attention to IEG evaluations and ensure learning and continuous that sector teams are closely involved in the improvement (―kaizen‖) with review processes (e.g. the MIGA response to the strong encouragement and recent IEG evaluation on Financial Sector 252 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) leadership from management. Guarantees), which is an efficient learning method (2010 IER) and feedback loop.  With both IEG and MIGA ramping up dissemination activities, staff will be in a position to draw relevant lessons for future guarantees. III. Institutional Effectiveness Recommendations #5. Strengthen and Align Management’s FY12 Assessment Substantial Business Development to  STATUS ASSESSMENT: SUBSTANTIAL MIGA has taken substantial steps to Support MIGA’s Strategy.  Multiple steps have been taken by MIGA to strengthen its business development. MIGA needs to revamp its strengthen the Agency‘s new business Taking stock of tangible results business development function development capacity, and to do so in a way that delivered through these initiatives to reverse the current emphasizes areas of strategic priority. over several years will prove the stagnation in guarantee  In FY11, MIGA established its Asian Hub to help sustainability of these initiatives. issuance and enable the Agency expand business coming from Asian clients – a to consistently meet or exceed nontraditional source of business for MIGA Management should however still its business volume, historically. Amongst other things, this will help consider some of the actual activities development effectiveness and bring in new southern investors. suggested in the recommendation, i.e. strategic priority goals. An  In FY12, MIGA‘s European Hub was created to reviewing the business development appropriately revamped bring MIGA closer to European clients as well as function with respect to the business development function African and Middle Eastern business appropriateness of staff roles and should be in place by the time opportunities. responsibilities, the adequacy and of the next strategy period,  Over the past two years, MIGA has entered into suitability of staff incentives for FY12-15. To that end MIGA formal business development arrangements with business development, and the cost- would benefit by reviewing its the IF, as well as with the World Bank‘s Treasury effectiveness of the activities business development function and FEU departments. The purpose of these undertaken. with respect to the arrangements is to leverage the greater reach of 253 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) appropriateness of staff roles MIGA‘s sister institutions and and responsibilities for  In FY12, MIGA‘s sector teams have developed business development, the detailed business development strategies that adequacy and suitability of reflect strategic priority areas. staff incentives for business development, and the cost- effectiveness of the activities undertaken, and based on the findings of such a review, taking the actions necessary to reverse the current stagnation and build up MIGA‘s guarantee business volume to allow selectivity, consistent with the Agency‘s accountability for meetings its targets for value-driven volume in the next strategy period. (2010 IER) #6. Improve its client Management’s FY12 Assessment High (to be retired next year pending relationship management, tangible results)  STATUS ASSESSMENT: COMPLETED including after-care, to Ongoing progress in MIGA‘s client  MIGA has taken notable steps to improve client enhance MIGA's value added relationship management is noted. and increase client retention. relationship management, including most notably the introduction of MIGA‘s two field Hubs. While Pending tangible results IEG will keep Improving client relationships tracking this recommendation for could be the most cost-effective MIGA is investing considerably in identifying and attracting new clients, at the same time there is another year and reconsider for marketing that MIGA could retirement in 2013. undertake to increase its also increased emphasis on retaining existing business volume. Managing clients. The Hubs will play an important part in this agenda. 254 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) client relationship requires a  Additionally, MIGA‘s Contract Management and focused and coherent business Portfolio Services team (CMPS) has been moved development plan into the Operations Department, in order to bring implemented by a staff with this function closer to underwriting teams. CMPS expertise in the guarantee staff are responsible for managing contracts in the business and/or financial portfolio and this presents an important markets. (2009 World Bank opportunity for after-care and to nurture client Group Guarantee Instruments) relationships that can yield future business Improve the effectiveness of its opportunities. business development and client relationship functions. (2008 IEG-MIGA Annual Report) #7. Measure Project-level Management’s FY12 Assessment Medium Financial Results to Inform IEG notes the ongoing activities with  STATUS ASSESSMENT: MEDIUM Strategy Development. regard to informing the agency‘s  The MDBs Evaluation Cooperation Group- To enable MIGA to grow its pricing including identifying critical Working Group on Private Sector Evaluation, business in a strategic and revenue and cost dynamics in MIGA‘s which sets global best practice for evaluating financially sustainable manner, portfolio. IEG looks forward to private sector projects, has noted in its Good MIGA should start to measure learning how these initiatives will Practice Standards for Private Sector Investment project-level financial results so inform MIGA‘s operations and Operations (4th Ed.) that the evaluation of that is can ensure that any strategy. individual guarantee project‘s contribution to cross-subsidies benefit only MIGA‘s financial results is not useful and that those projects with high ex-ante ―this EP (evaluation principal) is not relevant for development outcomes. MIGA‖. MIGA is in agreement with this view. Developing this system before the start of the next strategy  MIGA is however working on a number of period FY12-15 would enable activities that will help further inform the MIGA to use information on Agency‘s pricing, including: project level profitability to o Reassessing the assumptions employed in the pricing model 255 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) inform its operations and o Refining the assessment of costs relating to strategy development. (2010 different types of projects and activities IER) o Identifying critical revenue and cost dynamics in the portfolio.  This work is ongoing and well underway. It is taking place in the broader context of an overall review MIGA is undertaking with regard to risk management, which in turn plays a role in project selection. #8. Revise the policy Management’s FY12 Assessment Medium frameworks to harmonize  STATUS ASSESSMENT: SUBSTANTIAL MIGA‘s intentions to adopt the thematic coverage and  As a policy matter, MIGA and IFC follow the performance standards are noted and guidance across the WBG and enhance the relevance of those same Performance Standards. will be credited after the action has frameworks to client needs.  IFC has recently revised its Performance been completed.  IFC, MIGA, and the Standards and is beginning implementation. World Bank should MIGA is reviewing these changes and expects to jointly adopt and use a fully adopt the revisions (as long as they fit the shared set of objective MIGA business model) in order to maintain criteria to assess social harmonization. and environmental risks  With respect to the consistency of categorization to ensure adequacy and across institutions, MIGA continues to be consistency in project conservative in the way in which it categorizes categorization across projects, and maintains an active dialogue with the WBG, using the colleagues in other parts of the Group whenever more inclusive criteria questions arise on this point. The staff capacity is still at the same for category A, and  With respect to increasing the capacity of the level as in FY10. However, MIGA refining the Environmental and Social Unit, MIGA has taken a considered the monitoring categorization system to number of important steps that squarely address recommendation as a top priority and this issue, including the strengthening of MIGA‘s 256 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) address the bunching of Environmental and Social Team, which has their weakest part. As a result, MIGA higher and lower risk increased significantly in terms of staff size since had developed a monitoring strategy projects within the FY06. that was endorsed by their senior current category B.  With respect to the recommendation for category- management. They have started to  Increase the capacity of B, Small Investment Program projects, MIGA implement their monitoring program the Environmental and Management will be reviewing the Agency‘s last year. Social Unit to the level Disclosure Policy now that the IFC has completed needed to provide its review of its policy. credible assurance on  In the meantime, MIGA will still post its SPG for performance against the all SIP projects, which may include a more standards for every detailed explanation of environmental and social project. Should MIGA issues, and will attach the project ESIA if be unable to increase its warranted by the nature of the project. It should resources devoted to be noted that SIPs are subject to all requirements implementation of of PS1, which require MIGA clients to undertake Performance Standards, local disclosure (including any ESIA) and it should revise its community consultations as warranted by the Policy on Social and nature of the project. Environmental Sustainability to disclaim any responsibility for monitoring the projects‘ social and environmental performance and ensuring that they comply with the standards. Under this option, MIGA‘s role 257 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) would be limited to reviewing the client‘s assessment of the project‘s environmental and social risks against the standards, identifying corrective actions as needed, and securing the client‘s commitment to implement these actions.  Require that category-B, Small Investment Program projects follow the same disclosure requirements as for regular category-B projects. (2010 Safeguards and Sustainability Policies in a Changing World) #9. Enhance client capacity, Management’s FY12 Assessment Negligible responsibility, and ownership. MIGA‘s intended actions to date are  STATUS ASSESSMENT: SUBSTANTIAL  Focus the due diligence noted but IEG‘s rating can only reflect  With respect to the first part of the reviews of financial actions completed. recommendation, MIGA Management agrees and sector projects on the wishes to clarify that in the case of guarantees Social and provided in support of shareholder loans from a Environmental 258 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) Management Systems of parent bank to a subsidiary, it looks to the SEMS developing-country that the parent company imposes as a matter of subsidiaries the project corporate policy on itself and its subsidiaries (at supports, rather than local project enterprise level), which the the corporate policies of subsidiaries are expected to follow. the parent banks.  MIGA Management plans to start examining how  Expand the size and the client (i.e., corporate parent) implements its eligibility of the Trust policies at the local project enterprise level during Fund for Addressing the guarantee period, but will conduct this Environmental and monitoring exercise on a selective basis. Social Challenges to all  With respect to the second part of the low-capacity clients on recommendation Management also agrees in the basis of need. (2010 principle. The first three pilot years showed that Safeguards and the Trust fund is indeed a useful tool and resource Sustainability Policies in a for MIGA‘s clients. Management sees value in Changing World) maintaining and extending this facility, but at the same time notes that the ability to comply with this recommendation will largely be driven by donors‘ willingness to contribute additional funding. #10. Strengthen safeguards Management’s FY12 Assessment Negligible monitoring, evaluation, and There has not been any reported  STATUS ASSESSMENT: MEDIUM completion reporting follow up action with regard to the  With respect to the first point: As noted in the  Disclose project-level second recommendation since FY10 formal Management Response provided to this environmental and (note that IEG only rates the second report, Management disagrees with the social information from sub-recommendation as MIGA had recommendation, since there are client supervision reports. disagreed to the disclosure of project- confidentiality issues that frequently are relevant  Develop a credible level information). when it comes to this matter. MIGA plans to mechanism to ensure continue its practice of it being the client‘s 259 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) that Performance responsibility to report to the local community. Standards are adhered  With respect to the second point: This is a matter to by financial sector that is currently under discussion in the context of projects. (2010 the GPS4 Evaluation Standards. Safeguards and  MIGA ensures that for financial sector projects the Sustainability Policies in a project enterprise has a Social and Environmental Changing World) Management System (SEMS) consistent with MIGA‘s Policy and Performance Standards. This is ongoing.  In the case of guarantees provided in support of shareholder loans from a parent bank to a subsidiary, MIGA looks to the SEMS that the parent company imposes as a matter of corporate policy on itself and its subsidiaries (at the local project enterprise level), which the subsidiaries are expected to follow. MIGA Management plans to start examining how the client (i.e., corporate parent) implements its policies at the local project enterprise level during the guarantee period, but will conduct this monitoring exercise on a selective basis.  Lastly, Management does not believe that third party monitoring would be needed or cost effective for all projects, but would note that in high risk situations, it is current practice to require independent (third party) assessments as warranted by the nature of the project. 260 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) III. Institutional Effectiveness Recommendations #11. Address internal Management’s FY12 Assessment Medium weaknesses that reduce  STATUS ASSESSMENT: SUBSTANTIAL While progress is noted, it remains efficiency and slow  MIGA has introduced new streamlined guarantee unclear in how far the various responsiveness without lowering MIGA's financial, procedures aimed at increasing efficiency and streamlining and improvement efforts social and environmental enhancing client service. yielded tangible outcomes. Tracking standards.  Application procedures have been streamlined to these through suitable results reduce response time, and the early screening of indicators may be the best way to These include organizational new enquiries has been simplified to similarly issues in staffing, performance show implementation of this provide faster feedback to clients. recommendation. review, and incentives as well  Underwriting guidelines have been modernized, as consideration of matters the process of screening projects and final such as inflexibility on approval by the Project Review Committee has guarantee contract terms and been streamlined, and overall transaction conditions. However, documentation is being revamped to increase efficiency in its underwriting efficiencies. A key objective of this streamlining process must not come at the effort is also to improve teamwork to maximize expense of quality, risk synergy in guarantee transaction team activities. mitigation, safeguards, and  Recent business results provide an indication that development impacts of the measures to improve processing whilst maintain projects it insures. (2009 World rigor and quality have met with at least some Bank Group Guarantee success. FY11 saw a record level of Board- Instruments) approved new exposure issued and the largest number of projects supported in the previous five years. There was strong alignment with the Agency‘s strategic priority areas. The FY12 results to date have also been strong and there is a solid and well diversified pipeline of prospective new 261 IEG’s Assessment Outstanding IEG MIGA Management Assessment of Status of of Status of Implementation25 Recommendations for MIGA Implementation (Completed, Substantial, Medium, (Completed, Substantial, Medium, Management Low) Low) business ahead. Notes: These 11 recommendations tracked in the MATR derive from the IEG-MIGA and IEG joint reports listed below: Independent Evaluation of MIGA’s Development Effectiveness: Achieving Value-Driven Volume (2010) Independent Evaluation of MIGA’s Development Effectiveness: Enhancing MIGA’s Risk Mitigation in IDA and Conflict-Affected Countries (2009) World Bank Group Guarantee Instruments (2009) IEG-MIGA Annual Report (2008) IEG-MIGA Annual Report (2007) Safeguards and Sustainability Policies in a Changing World 262