IEG Independent Evaluation Group 50359 Independent Evaluation of IFC's Development Results KNOWLEDGE FOR PRIVATE SECTOR DEVELOPMENT THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions ­ the International Bank for Reconstruction and Development (IBRD); International Finance Corporation (IFC); the International Development Association (IDA); the Multilateral Investment Guarantee Agency (MIGA); and the International Center for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their environment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-IFC independently evaluates IFC's investment projects and Advisory Services operations that support private sector development. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to World Bank Group's Boards of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the World Bank Group's work, and to provide accountability in achieving its objectives. IEG seeks to improve World Bank Group work by identifying and disseminating lessons learned from experience and by framing recommendations drawn from evaluation findings. I N D E P E N D E N T E V A L U A T I O N G R O U P Independent Evaluation of IFC's Development Results 2009 Knowledge for Private Sector Development 2009 http://www.ifc.org/ieg Washington, D.C. 2009 © International Finance Corporation (IFC) 2121 Pennsylvania Avenue NW Washington, D.C. 20433, USA Telephone: 202-473-1000 Internet: http://www.ifc.org All rights reserved This volume, except for the "IFC Management Response to IEG-IFC" and "Chairperson's Summary" is a product of the In- dependent Evaluation Group (IEG) and the findings, interpretations, and conclusions expressed herein do not necessarily reflect the views of IFC Management, the Executive Directors of the World Bank Group, or the governments they represent. This volume does not support any general inferences beyond the scope of the evaluation, including any inferences about IFC's past, current, or prospective overall performance. The World Bank Group does not guarantee the accuracy of the data included in this publication and accepts no respon- sibility whatsoever for any consequences of their use. The boundaries, colors, denominations, and other information shown on any map in a publication do not imply any judgment on the part of the World Bank Group concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank Group encourages dissemination of its work and will normally grant per- mission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; facsimile: 978-750- 4470; Internet: http://www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, , The World Bank, 1818 H Street NW Washington, D.C. 20433, USA; facsimile: 202-522-2422; e-mail: pubrights@worldbank.org. Cover photos: Gas company workers beside containers in China; loan officer from an IFC-client microfinance institution meeting a borrower in Tanzania; a wind turbine farm in the United States. IFC Photo Collection. ISBN: 978-0-8213-7986-8 e-ISBN: 978-0-8213-7995-0 DOI: 10.1596/978-0-8213-7986-8 Library of Congress Cataloging-in-Publication Data have been applied for. World Bank InfoShop Independent Evaluation Group­IFC E-mail: pic@worldbank.org E-mail: AskIEG@ifc.org Telephone: 202-458-4500 Telephone: 202-458-2299 Facsimile: 202-522-1500 Facsimile: 202-522-0931 Printed on Recycled Paper Contents vii Abbreviations ix Definitions of Evaluation Terms xi Acknowledgments xiii Foreword xv Avant-propos xvii Prólogo xix Arabic Foreword xxi Executive Summary xxvii Résumé analytique xxxv Resumen ejecutivo xliii Arabic Executive Summary li IFC Management Response to IEG-IFC lvii Chairperson's Summary: Committee on Development Effectiveness (CODE) lix Advisory Panel Statement 1 1 Strategic Context 1 Growing Participation of the Private Sector in Development 2 Global Financial Crisis 4 Implications for IFC 5 Part I: Financing Development 7 2 Performance of IFC Investment Operations 7 Portfolio Pattern 10 Project Development Results 20 Impact and Implications of the Global Financial Crisis 27 Part II: Knowledge for Development 29 3 Performance of IFC Advisory Services 29 Knowledge, Development, and the Private Sector 31 Growth of IFC Advisory Services and Strategic Considerations 35 Organizational Alignment of Advisory Services 40 Delivery of IFC Advisory Services 51 Results of IFC Advisory Services iii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 65 4 Conclusions and Recommendations 65 Conclusions 67 Recommendations 69 Appendixes 71 A: Project Sample Representativeness--Investment Operations 73 B: Project Evaluation Methodology--Investment Operations 79 C: Further Details on Results Characteristics--Investment Operations 81 D: Risk-Adjusted Expected Development Outcome Regression: Model Specification, Analysis, and Preliminary Results 85 E: Chronology of IFC Advisory Services 89 F: Advisory Services Facilities, by Region 91 G: Project Sample Representativeness--Advisory Services 95 H: Project Evaluation Methodology--Advisory Services 97 I: Findings from IFC-Commissioned Reviews of IFC Advisory Services 101 J: High-Level Comparison of IFC Advisory Services with Other Multilateral Development Banks 115 Endnotes 121 Bibliography Boxes 12 2.1 How Are Project Development Outcomes Rated? 13 2.2 IFC Investment Outcome Rating 13 2.3 Illustrations of High and Low Project Development Outcomes 14 2.4 Measuring IFC Work Quality 21 2.5 Projects under Implementation in the Downturn Are Most Vulnerable to the Crisis 51 3.1 How Does IFC's Delivery Approach Compare with That of Other Development Institutions? 53 3.2 How Is Development Effectiveness Rated? 54 3.3 What Does Strong Advisory Services Development Performance Look Like? 55 3.4 Illustrations of Low Development Impact 61 3.5 Illustrations of IFC Additionality 62 3.6 How Is IFC Additionality Rated? Figures 2 1.1 Stronger Growth Has Generally Been Associated with Increased Private Capital Flows 2 1.2 Knowledge Accumulation Is Key for Future Productivity 3 1.3 Growing Share of Bank Group Financing to the Private Sector 3 1.4 Bank Group Knowledge Services Are Increasingly Aimed at the Private Sector 8 2.1 IFC's per Client Exposure Has Doubled in the Last Ten Years 9 2.2 IFC and World Bank IDA Operations Increased over FY06­FY08 9 2.3 IFC Has Made Up a Higher Share of Multilateral Development Bank Finance in Asia, MENA, and LAC 10 2.4 Project Development Outcomes and IFC Investment Returns Improved in the Last Three Years 11 2.5 Improvement in 2006­08 Followed Historically Weak Performance in 2004­05 iv CONTENTS 11 2.6 Country Business Climate Risk Improved in Most Regions 12 2.7 IFC Work Quality Improved Again 15 2.8 Strong IFC Work Quality Can Help Clients Overcome Risk 15 2.9 Strong Additionality Is Important for Effective Risk Mitigation 15 2.10 Better Ratings Again in Europe and Central Asia and in Latin America and the Caribbean 16 2.11 IFC Appraisal Quality and Realized Additionality Were Much Weaker in East Asia and the Pacific and in Sub-Saharan Africa 16 2.12 External Risks Were Highest in Sub-Saharan Africa 17 2.13 Performance Was Strong in IFC's Strategic Sectors 18 2.14 Environmental and Social Effects Performance Weakened in 2008 18 2.15 Environmental and Social Effects Performance Has Declined Sharply for Financial Intermediary Operations 19 2.16 Project Development Outcomes and IFC Profitability Were Strongly Correlated 20 2.17 On a Cumulative Basis, High/High Outcomes Were Achieved Half the Time 21 2.18 Best Results When IFC Investments Were Made in the Wake of a Crisis 25 2.19 Global and Regional Investments Tend to Perform Less Well Than Single- Country Investments 32 3.1 Rapid Growth in Advisory Services Operations and Staff 32 3.2 Seventy-Seven Percent of Advisory Services Staff Are Based in the Field 34 3.3 Access to Finance Is the Largest Business Line 34 3.4 Highest Share of Operations Is in Sub-Saharan Africa 37 3.5 Delivery Structure for IFC Advisory Services Projects in Africa 39 3.6 Country Assistance Strategies Provide Limited Coverage of Other Knowledge Providers 39 3.7 Discussion of Activities and Complementarities with Others in Project Approvals Is Weak 41 3.8 Donor Commitments, by Region, FY05­FY08 42 3.9 Most Advisory Operations Featured No Client Contributions 45 3.10 Project Completion Report Quality, by Business Line 46 3.11 Project Completion Report Quality, by Region 47 3.12 About One-Fifth of Advisory Services Operations Are Linked to IFC Investment Services Operations 48 3.13 Advisory and Investment Services Linkages Are Greatest for Access to Finance Operations 52 3.14 Majority of Operations in IDA Countries Are Similar to Official Aid Pattern 54 3.15 Strategic Relevance Was Often Rated High, Impact Achievement Much Less So 55 3.16 Environmental and Social Sustainability Project Ratings Have Lagged Behind Those of Other Business Lines 56 3.17 Ratings of Operations, by Region 57 3.18 Country Conditions: Better Ratings Where Country Business Climate Risk Is High 57 3.19 The Overall Relationship between Advisory and Investment Services Performance Is Moderate 58 3.20 Better Advisory Services Ratings When Combined with IFC Loans Rather Than Equity 58 3.21 Stronger Performance in Repeat or Combined Advisory Services Operations v I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 58 3.22 Client Commitment: Better Ratings When Client Contributed 59 3.23 Project Implementation: Local Presence Is Associated with Better Results 59 3.24 Higher Role & Contribution, Better Ratings 59 3.25 Better M&E Quality, Better Ratings 60 3.26 Similar Performance for Projects That Began between Periods 2003­05 and 2006­08 60 3.27 Slightly Lower Performance for Recently Completed Operations Tables 8 2.1 Methodologies Employed by IEG to Evaluate IFC Investment Operations 10 2.2 IFC Tended to Invest in Countries with Lower Prior Levels of Foreign Direct Investment to GDP 22 2.3 In Argentina, Performance Fell Dramatically as the Business Environment Deteriorated 25 2.4 Selected Lessons from Regional and Global Investments 30 3.1 Methodologies Employed by IEG to Evaluate IFC Advisory Services 35 3.2 IFC's Main Advisory Services Client Is Government 35 3.3 Government Clients Predominate for Business Enabling Environment and Infrastructure Work 40 3.4 Two Main Funding Sources: Donors and IFC 41 3.5 Donor Funding Leverage Has Been Highest in South Asia 42 3.6 Middle East and North Africa Region and Infrastructure Have the Most, Though Still Limited, Cost Contributions by Clients 43 3.7 Some Free or Near-Free Advisory Services Operations Cross-Subsidize IFC Investments 44 3.8 Ratio of Staff to Consultant Expenses Is Roughly 1:1 47 3.9 Few External Reviews Have Thus Far Focused on Impact 52 3.10 By Business Line, Resources Have Tended to Be Allocated to Countries in Greatest Need 56 3.11 Selected Ratings, by Product vi ABBREVIATIONS A2F Access to Finance (business line) ADB Asian Development Bank AfDB African Development Bank AS Advisory Services BEE Business Enabling Environment (business line) CA Corporate Advice (business line) COI Conflict of interest DANIDA Danish International Development Agency DFID Department for International Development E&S Environment and safeguards EBRD European Bank for Reconstruction and Development EIB European Investment Bank ERR Economic rate of return ESS Environmental and Social Sustainability ESW Economic and sector work FDI Foreign direct investment FI Financial intermediary FIAS Foreign Investment Advisory Services FRR Financial rate of return FY Fiscal year GDP Gross domestic product IBRD International Bank for Reconstruction and Development IDA International Development Association IDB Inter-American Development Bank IEG Independent Evaluation Group IFC International Finance Corporation IMF International Monetary Fund INF Infrastructure (business line) IS Investment Services IT Information technology M&E Monitoring & evaluation MDB Multilateral development bank MIGA Multilateral Investment Guarantee Agency PCR Project Completion Report PEP Private Enterprise Partnership PSD Private sector development RAEDO Risk-Adjusted Expected Development Outcome SME Small- and medium-sized enterprise TA Technical assistance vii DEFINITIONS OF EVALUATION TERMS Investment operations: Company: The entity implementing the project and, generally, IFC's investment counter-party. For financial markets operations, it refers to the financial intermediary (or fund manager), as distinct from its portfolio of IFC-financed sub-project companies. Operation: IFC's objectives, activities, and results in making and administering its investment. Project: The company objectives, capital investments, funding program, and related business activities being partially financed by IFC's investment selected for evaluation. Example: "Through this operation IFC provided $55 million for the company's $100 million cement manufacturing expansion project in the form of a $20 million A-loan, a $30 million B-loan from commercial banks and a $5 million equity investment." Financial markets All projects where the company is a financial intermediary or financial projects: services company, including agency lines and private equity investment funds. Non-financial markets All other projects; sometimes referred to as "real-sector" projects. projects: Advisory Services operations: Outcomes of Outcomes refer to implementation of recommendations or advice. AS operations: Impacts of Impacts refer to the changes that occurred following the AS operations: implementation of recommendation. Example: An AS operation recommended that the country amend the leasing law to incorporate best practice in similar markets in the region. Outcome--the country amended the leasing law in accordance with the recommendation. Impact--the leasing industry became attractive to potential sponsors as evidenced by new companies that were established following the amendment of the leasing law. ix Acknowledgments This report was prepared by a team led by The evaluation was produced under the guid- Daniel Crabtree, drawing upon research and ance of Stoyan Tenev, Head of Macro Evaluation, contributions from Ana Belen Barbeito, Nicholas IEG-IFC, and Amitava Banerjee, Manager, IEG- Burke, Unurjargal Demberel, Hiroyuki Hata- IFC; and under the overall leadership of Marvin shima, Kris Hallberg, Beata Lenard, Maria Elena Taylor-Dormond, Director, IEG-IFC. Pinglo, Stephen Pirozzi, Cherian Samuel, and Izlem Yenice. Rosemarie Pena and Marylou Kam- The report benefited substantially from the con- Cheong provided general administrative sup- structive advice and feedback from many staff port to the study team. Helen Chin edited the at IFC, and also from a number of Independent report for publication; Sid Edelmann, Sona Evaluation Group (IEG) colleagues in IFC, the Panajyan, and Shunalini Sarkar managed its World Bank, and MIGA. Peer review was provided production and dissemination. by Bruce Murray and Nils Fostvedt. xi I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Director-General, Evaluation, World Bank Group Vinod Thomas Director, IEG-IFC Marvin Taylor-Dormond Head of Macro Evaluation Stoyan Tenev Task Manager Daniel James Crabtree Study Team Evaluation Officer Hiroyuki Hatashima Evaluation Officer Cherian Samuel Evaluation Analyst Izlem Yenice Consultant Unurjargal Demberel Consultant Kris Hallberg Program Assistant Marylou Kam-Cheong Program Assistant Rosemarie Pena xii Foreword I FC has been undergoing transformation of both its investment and advi- sory services operations in recent years. In particular, a sharp growth in its Advisory Services (AS) is changing the nature of the organization. IFC now has more staff in the field for AS than for Investment Services (IS), its traditional core business. Independent Evaluation of IFC's Development impact. This will need to change in IFC's response Results 2009 takes stock of the development per- to the current crisis, so that the tension between formance of IFC's investment operations, and ex- protecting the portfolio and responding to op- amines, for the first time, the development portunities can be effectively managed. effectiveness of its AS--thus offering the first ho- listic review of IFC's development results. On IS, The risks and opportunities brought on by the cur- the report finds that 72 percent of operations rent crisis extend to AS as well. The crisis exposes reaching early operating maturity between 2006 gaps in sustainable business practices and business and 2008 met or exceeded their financial, eco- regulation globally, thus offering IFC an opportu- nomic, environmental, and social benchmarks, nity for greater impact in these areas. But in order and made contributions to private sector devel- to do so, bold actions are needed. IFC's AS activ- opment beyond just the project. This is a signifi- ities--fueled by donor money and IFC's own fund- cant improvement over the 63 percent achieved ing--have grown in a largely unchecked manner, between 2005 and 2007. Meanwhile, 70 percent raising concerns about the long-run sustainability of AS operations reviewed between 2006 and 2008 of the current business model. Recent measures achieved high development effectiveness ratings. are intended to initiate a broad AS institutional re- But these development results do not yet reflect alignment aimed at tackling these challenges. the sharp deterioration in global economic con- ditions, which has just now begun to affect the eco- The monitoring and evaluation (M&E) system nomic environment in most developing countries. for AS was only introduced in 2006. Nonethe- less, it is possible to discern several patterns in Experience suggests there are considerable risks performance. First, project development effec- to development results but crises can also offer new tiveness has been strongest in Southern Europe opportunities that need to be grasped. Projects ap- and Central Asia, and weakest in Latin America and proved in the years prior to a crisis were about 15 the Caribbean. Second, results were significantly percent less likely to achieve good results than oth- better for infrastructure, business enabling en- erwise. In the wake of past crises, investing was vironment, and corporate advice operations, likely to lead to better results. But measures to pro- and weaker in the case of environmental and tect the portfolio have tended to crowd out the social sustainability operations--a particular con- proactive pursuit of new opportunities to broaden cern in Africa and for IFC's work with financial xiii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 institutions. Third, key drivers of performance To enhance its development effectiveness and appear to be: country conditions and client com- additionality (unique role and contribution), IFC mitment; local presence and ownership; pro- should formulate an overall strategy for its advi- grammatic approaches, as opposed to one-off sory services, addressing the need for a clear interventions; and the quality of IFC additional- vision and business framework. At the same time, ity and M&E. In this context, effective pricing of it must pursue more programmatic AS interven- AS is fundamental because it should provide in- tions, improve execution of the AS pricing policy, centives to improve all aspects of the AS business. and strengthen AS performance measurement. Vinod Thomas Director-General Evaluation xiv Avant-propos L 'IFC a connu ces dernières années une période de transformations, tant du point de vue de ses opérations d'investissement que de ses activités de services-conseil. La croissance rapide de ces dernières, en particulier, entraîne un changement de nature de l'organisation. L'IFC compte désormais sur le terrain plus de personnel pour les services-conseil que pour les services d'investissement, qui étaient traditionnellement son coeur de métier. Le présent rapport du Groupe indépendant d'éva- bles, mais les crises peuvent aussi offrir des op- luation présente un bilan des réalisations des portunités nouvelles, qu'il faut savoir exploiter. La opérations d'investissement de l'IFC au service du probabilité que les projets approuvés dans les développement et, pour la première fois, de l'ef- années précédant une crise produisent des ré- ficacité de ses services-conseil, offrant ainsi la sultats satisfaisants est de 15 % environ inférieure première appréciation globale de sa contribution à ce qu'elle est pour les autres. Investir dans la fou- effective au développement. lée de crises passées offre une meilleure proba- bilité de bons résultats. Mais les mesures prises S'agissant des projets d'investissement, il est pour protéger le portefeuille ont souvent pour constaté que 72 % des opérations parvenues à effet de supplanter la poursuite dynamique d'op- leur régime de croisière entre 2006 et 2008 ont at- portunités nouvelles qui permettraient d'élargir teint ou dépassé leurs valeurs de référence finan- l'impact des projets. Il y a là un aspect qu'il fau- cières, économiques, environnementales et sociales, dra modifier dans la manière dont l'IFC réagira à et ont contribué au développement du secteur la crise actuelle, de manière à bien gérer les ten- privé au-delà du projet proprement dit, ce qui sions entre la protection du portefeuille et le marque une nette amélioration par rapport aux parti à tirer d'opportunités nouvelles. 63 % enregistrés entre 2005 et 2007. Parallèlement, 70 % des opérations de services-conseil examinées Les risques et les opportunités découlant de la entre 2006 et 2008 se sont vu attribuer une note éle- crise actuelle touchent également les services- vée du point de vue de leur efficacité au plan du conseil. La crise met en évidence des lacunes développement. Mais ces résultats ne rendent pas dans le monde entier, en matière de pratiques encore compte de la grave détérioration des condi- commerciales durables, de réaction pour atté- tions économiques mondiales, qui ne commence nuer les effets du changement climatique et de ré- que maintenant à retentir sur le climat écono- glementation des activités commerciales, offrant mique dans la plupart des pays en développement. ainsi à l'IFC une occasion de développer son im- pact dans ces domaines. Mais pour y parvenir, il L'expérience donne à penser que les risques pour faudra des interventions hardies. Les activités de les résultats de développement sont considéra- conseil de l'IFC, alimentées par les contributions xv I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 des donateurs et son financement propre, se sont les activités où l'IFC travaille avec des institutions développées sans contrôle pour une bonne part, financières. Troisièmement, les principaux dé- ce qui suscite des préoccupations pour la viabi- terminants des résultats semblent être : les condi- lité à long terme du modèle d'activités actuel. tions dans le pays et l'engagement du client ; les Des mesures récentes ont eu pour objet d'enta- approches-programmes, plutôt que les interven- mer un réalignement institutionnel des services- tions ponctuelles ; la qualité de la valeur ajoutée conseil visant à rectifier ce qui doit l'être. par l'IFC, et de ses activités de suivi et d'évalua- tion. Dans ce contexte, il est fondamental que le Le système de suivi et d'évaluation n'a été intro- prix des services-conseil soit fixé de manière ef- duit pour les services-conseil qu'en 2006, mais il ficace, car cela devrait inciter à améliorer tous les est possible de dégager plusieurs tendances dans aspects des activités des services-conseil. les réalisations. Premièrement, c'est en Europe méridionale et en Asie centrale que l'efficacité Pour renforcer l'efficacité de son action au service des projets au plan du développement a été la plus du développement et sa valeur ajoutée (son rôle marquée, et c'est en Amérique latine et dans les et son concours sans équivalent), l'IFC devra dé- Caraïbes qu'elle a été la plus faible. Deuxième- finir une stratégie globale pour ses services-conseil, ment, les résultats ont été nettement meilleurs qui énonce clairement une perspective et un pour les projets concernant l'infrastructure, l'ins- schéma d'activité. Parallèlement, elle devra cher- tauration d'un cadre porteur pour les entreprises, cher à inscrire les interventions des services- et les conseils aux sociétés, alors qu'ils ont été conseil dans une démarche plus globale de moins satisfaisants pour les opérations axées sur programme, améliorer l'application des politiques la viabilité environnementale et sociale--ce qui est de fixation des prix des services-conseil, et ren- particulièrement préoccupant en Afrique et pour forcer la mesure des réalisations de ces services. Vinod Thomas Directeur général Évaluation xvi Prólogo E n los últimos años, las operaciones de servicios de inversiones y de ase- soría de la IFC han venido experimentando transformaciones. En es- pecial, un pronunciado crecimiento de sus servicios de asesoría está modificando las características del organismo, que cuenta actualmente con más funcionarios de servicios de asesoría sobre el terreno que los que destina a su esfera de actividad básica tradicional: los servicios de inversiones. En la presente Evaluación Independiente de los La experiencia indica que los resultados en tér- Resultados de Desarrollo se pasa revista al de- minos de desarrollo están expuestos a riesgos sempeño de las operaciones de inversiones de la considerables, pero las crisis pueden también IFC en términos de desarrollo y se examina, por ofrecer nuevas oportunidades que es preciso primera vez, la eficacia en términos de desarro- aprovechar. La probabilidad de que los proyectos llo de sus servicios de asesoría, por lo cual se aprobados en los años que precedieron a una ofrece el primer examen integral de los resulta- crisis obtuvieran buenos resultados fue alrededor dos obtenidos por la Corporación en términos de de 15% menor que la de los restantes. Invertir desarrollo. después de las crisis del pasado ofreció mayores probabilidades de obtener mejores resultados, Con respecto a los servicios de inversiones, en el pero las medidas destinadas a proteger la cartera informe se constata que el 72% de las opera- han tendido a desplazar la búsqueda proactiva ciones que llegaron a un vencimiento operativo de nuevas oportunidades de ampliar el impacto. anticipado entre 2006 y 2008 cumplieron o ex- Esto tendrá que cambiar en la respuesta de la IFC cedieron sus parámetros de referencia finan- a la crisis actual, para poder manejar eficazmente cieros, económicos, ambientales y sociales, y la tensión entre protección de la cartera y res- contribuyeron al desarrollo del sector privado puesta a las oportunidades. más allá del proyecto, lo que implica una mejora significativa con respecto al 63% logrado entre Con respecto a los servicios de asesoría, los ries- 2005 y 2007. En comparación, el 70% de las ope- gos y oportunidades que plantea la crisis actual raciones de servicios de asesoría examinadas también se extienden a ellos. La crisis pone de ma- entre 2006 y 2008 lograron altas calificaciones en nifiesto vacíos en procedimientos operacionales cuanto a eficacia en términos del desarrollo. No sostenibles, mitigación del cambio climático y obstante, esos resultados relativos al desarrollo reglamentos de negocios en todo el mundo, lo aún no reflejan el profundo deterioro de las con- que ofrece a la IFC la oportunidad de suscitar un diciones económicas mundiales, que recién ahora mayor impacto en esas esferas. Pero para lograrlo han comenzado a afectar al entorno económico se requieren medidas audaces. En gran medida, de la mayoría de los países en desarrollo. las actividades de los servicios de asesoría de la xvii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 IFC, alimentadas por dinero de los donantes y labor realizada por la IFC con instituciones fi- recursos financieros propios de la IFC, han au- nancieras. Tercero, los siguientes son, al parecer, mentado en forma incontrolada, lo que genera factores determinantes clave del desempeño: preocupaciones acerca de la sostenibilidad a largo condiciones del país e identificación del cliente plazo del actual modelo de actividad. Las medi- con sus operaciones; utilización de enfoques pro- das recientes están destinadas a poner en marcha gramáticos, en lugar de intervenciones aisladas; una amplia realineación institucional de los ser- calidad de la adicionalidad de la IFC y del sistema vicios de asesoría encaminada a hacer frente a esos de seguimiento y evaluación. En este contexto es desafíos. fundamental la determinación efectiva de pre- cios de los servicios de asesoría, que previsible- El sistema de seguimiento y evaluación de los mente creará incentivos para mejorar en todos sus servicios de asesoría recién se introdujo en 2006, aspectos las actividades propias de los servicios pese a lo cual es posible discernir varias modali- de asesoría. dades de desempeño. Primero, el más alto nivel de eficacia en el desarrollo de proyectos se registró Para lograr mayor eficacia en términos de desarrollo en las regiones de Europa meridional y Asia cen- y adicionalidad (papel singular y contribución), la tral, y el más bajo en la región de América Latina IFC debería formular una estrategia global para sus y el Caribe. Segundo, los resultados fueron con- servicios de asesoría, atendiendo la necesidad de siderablemente mejores en materia de infraes- una visión y un marco de negocios más claros. Al tructura, condiciones propicias para los negocios mismo tiempo, debe tratar de realizar más inter- y operaciones de asesoría para empresas, y menos venciones de servicios de asesoría programáticas, satisfactorios en el caso de las operaciones de mejorar la ejecución de la política de precios de sostenibilidad ambiental y social, que fueron mo- tales servicios y fortalecer la medición del de- tivo de especial preocupación en África y para la sempeño en materia de servicios de asesoría. Vinod Thomas Director general de Evaluación xviii . . . 15 . . . . 72 . 2008 2006 . 63 .2007 2005 70 . 2008 2006 . . . . . . xix I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 ( ) .2006 . . . . : . . . Vinod Thomas xx Executive Summary ver the last decade, many developing countries have experienced O strong economic growth, typically accompanied by falling levels of poverty. The private sector has been a key contributor to this growth, mainly through new capital investment, but also through fostering innovation and entrepreneurship, helping to create jobs, and opening up new markets. Developing countries with the highest levels of World Bank) assumes particular importance. Con- private investment and those that have made the cerning the first role, IFC's founding articles state biggest strides in bridging knowledge and tech- that the Corporation should invest in viable pri- nology gaps with the developed world--from vate sector projects in developing countries for India to the Baltic States--have generally grown which "sufficient private capital is not available on the quickest. reasonable terms."1 In such crisis times, the onus is on IFC to ramp up its financing efforts. But The current global financial crisis places many of IFC's second role as a knowledge provider (to- these hard-won gains under severe threat. The cri- gether with the World Bank) is also important, par- sis began in the developed world, but has since ticularly as policy makers and administrators focus spread to the developing world, and has partic- on business regulations, good governance, and the ularly affected countries with economies more environmental and social sustainability of growth. connected to global markets. Import demand IFC provides advice that helps to shape the con- from developed countries is falling, and compa- ditions for sustainable private sector develop- nies in developing countries, both large and small ment--for example, through promoting more (particularly small), have also found that funds for effective regulation--and to enhance the capac- new investment have dried up, or have become ity, skills, and behavior of actors involved with much more expensive and more difficult to ob- private sector enterprise in the field (including ef- tain. Private capital flows to developing countries fective management of the social and environ- in 2009 are expected to be, at best, about half their mental effects of private activities). level in 2007 (of $1 trillion). Past crises suggest that it may take some years for these flows to return This Independent Evaluation of IFC's Development to their precrisis levels. More generally, the crisis Results (IEDR) looks at each of these roles in turn: has led policy makers and analysts to reevaluate IFC's effectiveness in financing development the role of markets and the private sector, par- through its growing portfolio of investment op- ticularly where the value of effective regulation, erations, with an emphasis on IFC's experience prudential oversight, and fiduciary management during previous crises and in helping clients mit- was wrongly deemphasized or ignored. igate investment risks (Part I); and--for the first time and as the main theme of this report--the In times like these, IFC's dual role as a financier Corporation's experience organizing and deliver- and as a provider of knowledge (together with the ing its Advisory Services (AS) interventions, that xxi I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 is, knowledge services the IFC provides to either achieving high outcomes in 2005­07. On a cu- private companies or governments in support of mulative basis, since independent evaluation private sector development (Part II). In terms of started in 1996 and up to and including 2008, 62 results, the report focuses on IFC investment op- percent of projects (70 percent by volume) have erations that reached early operating maturity be- achieved high development outcome ratings. tween 2006 and 2008, and IFC AS projects with Project Completion Reports during the same pe- Stronger overall results in recent years reflected riod. The review of AS development effectiveness several factors: i) the exit of a particularly weak per- comes with certain caveats, given that the moni- forming cohort of projects, which matured in toring and evaluation system was only introduced 2005 (51 percent of projects maturing in 2005 re- in 2006, and considering the often intangible na- alized high development outcomes, compared ture of knowledge transmission. Nonetheless, the with 75 percent maturing in 2008); ii) more fa- report, for the first time, provides a combined ac- vorable economic conditions in much of the de- count of both arms of IFC's business--invest- veloping world (until late 2008, by which time ments and AS--including situations where these most evaluated projects had been substantially instruments have been offered to the same client. implemented); iii) improvement in IFC project The report also complements a recent IEG eval- appraisal and structuring quality; iv) the conscious uation of the effectiveness of World Bank eco- move by IFC toward larger projects, which have nomic and sector work and technical assistance, been likely to achieve higher ratings than smaller which was completed in 2008.2 projects, due in part to greater internal scrutiny; and v) especially strong performance in Europe Financing Development and Central Asia, and in Latin America and the IFC's portfolio of investment operations (loans, Caribbean, where the majority of mature opera- equity, and other financial products) continued to tions are located. In these regions, business con- grow in the last year. The cumulative volume of ditions were most supportive and IFC work quality active investment activities increased by about a was strongest. South Asia exhibited improving quarter, from $32.7 billion in fiscal year (FY) 2007 performance, with higher IFC work quality than to $40 billion in FY 2008. The number of invest- in the past. ments rose by a lesser order (8 percent), reflect- ing a general preference for larger investment Performance lagged considerably in East Asia and operations (increasingly involving corporate fi- the Pacific, and in the mainly low-income Middle nance rather than project finance), and a more East and North Africa, and Sub-Saharan Africa-- wholesale approach to reaching small and me- with barely half of the projects in these regions dium enterprises (SMEs), that is, through finan- meeting or exceeding specified benchmarks and cial intermediaries and larger companies. standards. External conditions were partly re- sponsible--projects in Sub-Saharan Africa and A growing portfolio provides opportunities to Middle East and North Africa generally featured extend the Corporation's development reach. high levels of country, sponsor, and product com- IEG's evaluations of investment operations that petitiveness risks--but the quality of IFC's work reached early operating maturity between 2006 and contribution to the project tended to have a and 20083 show that IFC's project development larger impact. This was especially the case in East results improved overall. More specifically, 72 Asia and the Pacific, where nearly 40 percent of percent of evaluated projects (85 percent by vol- projects exhibited low quality of IFC additionality. ume) achieved outcomes that, on balance, met or There is evidence of better screening and ap- exceeded project financial, economic, and envi- praisal work in Middle East and North Africa and ronmental and social benchmarks and standards, improved supervision quality in Sub-Saharan Africa. and made positive contributions to private sector development beyond the project. This compares Among IFC's strategic sectors, project perfor- with 63 percent of projects (75 percent by volume) mance showed continued improvement in health xxii E X E C UT I V E S U M M A RY and education, it was better in agribusiness, and development risk. At the same time, IFC has con- remained strong in infrastructure and financial siderably strengthened its internal risk manage- markets. At the same time, performance lagged in ment processes and its capacity to bear and nontelecommunications information technology manage financial risks appears to have improved (software and Internet).4 In other sectors, oil, gas, significantly in recent years. Importantly, evalua- mining, and chemicals projects achieved relatively tion suggests that investments approved in the poor ratings. Risk exposure was clearly a factor in wake of the crisis (i.e., at the bottom of the busi- weak nontelecommunications information tech- ness cycle) will tend to have better development nology projects, most of which were small opera- results. Thus, there are also upside opportunities tions involving inexperienced sponsors and unclear that need to be grasped. product competitiveness. However, work quality in this sector was also well below par, with high The experience of past crises underlines two key ratings in just 40 percent of cases. Improved work responses by IFC: first, careful portfolio risk man- quality was in evidence in the health sector, where agement, particularly projects in early imple- IFC showed that it had learned lessons from past mentation; and second, IFC additionality. The experience, but the portfolio has not achieved latter is particularly important in two respects: much diversity. Oil, gas, mining, and chemicals i) in acting as an honest broker in restructurings; projects did not meet benchmarks for a number and, ii) in pursuing a well-timed and targeted ap- of reasons: technical weaknesses of the sponsor; proach to new operations, particularly through the higher than expected asset acquisition cost; and signaling effect IFC interventions can provide to in one case, unsatisfactory environmental com- other investors. pliance. Environmental and social effects ratings were stable for real sector projects, but remained Knowledge for Development weak in financial intermediary operations, re- IFC AS have been growing rapidly, with an active flecting the need to strengthen client capacity and portfolio approaching $1 billion and employing securing their commitment, as well as addressing 1,262 staff, a sevenfold increase in the last seven shortfalls in IFC supervision and additionality. years. As a result, the nature and face of IFC has changed significantly: AS staff now make up the The development results reported above do not majority of the Corporation's presence in the yet reflect the sharp deterioration in global eco- field in developing countries.5 The rapid growth nomic conditions, which has just now begun to of AS has happened in a largely unchecked man- affect investment returns in most developing ner. This is well illustrated in the emergence of countries. The development results reported here more than 50 AS products, 18 regional facilities largely reflect project experience during 2003­08, covering seven regions, 13 global business units, a period of unprecedented growth in emerging and about half of AS work being contracted out markets. Most evaluated projects had been sub- to short-term consultants. stantially implemented, and some had been closed by late 2008 when the crisis started to affect the Important strategic questions need to be ad- developing world. dressed. These include whether, in grafting such a substantial knowledge business onto a financ- The development results of maturing operations ing institution, IFC has the appropriate balance of are, however, expected to decline in the coming efforts between AS and Investment Services (IS) years. Past evaluation shows that projects ap- to ensure maximum development impact. Qual- proved in the years prior to the crisis (and being ity trade-offs are also possible, given substantial implemented during the downturn) are most at organizational change, a high reliance on rela- risk from a development perspective. Approxi- tively new staff (60 percent have been with IFC less mately 40 percent of IFC's portfolio (62 percent than three years), and outsourcing work through by volume) falls into this category, thus the Cor- some 1,300 short-term consultants each year. poration is exposed to considerable downside There is also increased possibility of conflict of in- xxiii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 terest or market distortion--where AS is offered has been fundamental for achieving results, and together with financing, or is provided at less has been particularly noticeable among business than market value. enabling environment operations in IDA countries with high business climate risk, and in some pack- IFC deploys its AS in the pursuit of general ob- ages of services, such as SME linkage projects in jectives that are common with those for IFC in- agribusiness, manufacturing and extractive sec- vestments. These objectives include focusing on tors. Such packaging raises potential conflicts of frontier markets (including International Devel- interest, which must be tackled effectively, and opment Association, or IDA, countries and fron- needs appropriate pricing. Intrinsic constraints in tier regions of non­IDA countries, as well as SMEs capturing the impact of AS are compounded by and agribusiness), strategic sectors, such as fi- the relatively weak application of M&E guidelines nance, infrastructure, health, and education, and to date by IFC staff. support for environmental and social sustain- ability (including climate change). The allocation Over the last five years, IFC's management has of AS resources has been largely aligned with taken action to enhance its AS effectiveness through these priorities. That is, IFC AS has generally tar- efforts to strengthen AS organizational alignment geted high-need destinations, such as IDA coun- and delivery processes. Efforts to bring greater tries and Africa in particular. structure and clarity include: categorizing AS ac- tivities into five business lines; consolidating some Relevance, however, does not guarantee impact. global and regional facilities; classifying products Fifty-two percent of IFC's AS projects, where rat- by level of maturity; developing AS staff compe- ings could be assigned, were rated high on tencies; AS training; and establishing an AS vice achieved development impact. Projects rated sub- presidency. IFC's attention to the delivery of AS stantially higher on other dimensions of per- has focused on establishing mechanisms and sys- formance, such as strategic relevance, output, tems to ensure: adequate, sustainable funding; and outcome achievement, with an overall devel- client commitment; sound project design and im- opment effectiveness success rate of 70 percent. plementation; and robust M&E of performance. Ratings did not change significantly for projects IFC's efforts in these areas appear to compare fa- that began before (as opposed to projects initiated vorably with measures taken by other multilateral after) the major organizational changes in 2005/06. development banks, for example, in the intro- By region, ratings have been substantially better duction of a pricing policy (which broadly seeks to in Southern Europe and Central Asia, and weaker build client commitment and reduce possible mar- in Latin America and the Caribbean. Evaluated ket distortion by limiting any subsidies to public global projects also did not perform well. By busi- goods), and an M&E system, which seeks to cap- ness line, while the variation in results is less pro- ture outcomes and impacts, as opposed to just out- nounced than by region, infrastructure, business puts. The momentum of transformation continues enabling environment, corporate advice, and ac- with the recent introduction of new policies, pro- cess to finance tend to perform better than en- cedures, and guidelines related to pricing, conflict vironmental and social sustainability. of interest, funding, and governance. Key drivers of results have been client commit- The professionalization of AS, however, remains ment (as evidenced by contribution to project a work in progress and significant organizational costs and especially so for environmental and so- issues still persist: overlapping and parallel im- cial sustainability projects), strong project design plementation structures in several regions (Sub- and implementation, IFC's proximity to the client Saharan Africa, East Asia and the Pacific, and South as defined by IFC's local presence and involve- Asia); few well-established products outside of ment, programmatic (rather than one-off) inter- finance and infrastructure; lack of clarity about ventions, and effective M&E. Strong additionality how AS and IS are best integrated in different xxiv E X E C UT I V E S U M M A RY contexts; limited consideration of IFC's compar- funding, is an opportunity for the Corporation to ative advantages relative to other knowledge serv- push harder in the direction of value-based pric- ice providers at the strategic and project levels; ing, and to encourage other development insti- and no umbrella AS strategic framework to weave tutions to do likewise. different strands together. Recommendations There are also substantial gaps in delivery that This review comes at a time of deep distress in fi- need to be addressed--particularly in matching nancial markets and a severe downsizing in private corporate intent with consistent implementation economic activities. It reminds us of the critical im- on the ground. This applies with respect to the portance of sustainable development in the pri- execution of the pricing policy, as well as ensur- vate sector, for which regulatory frameworks are ing good quality project design and implemen- important and excessive deregulation costly. In tation, and effective collaboration with other these circumstances, this review provides further actors, including the World Bank. Getting the findings on what IFC might do to enhance de- right staffing mix has been a particular challenge, velopment effectiveness and additionality: with a heavy reliance on short-term consultants and relatively new staff (as compared to those in- Operations during the Crisis: volved with investment operations). The chosen · Effectively manage the tension between mix has major implications for the quality and protecting the portfolio and responding continuity of IFC's AS, and the preservation of to opportunities during crisis. In the past, global knowledge leadership. At all stages of de- this tension has not always been managed ad- livery, M&E data provided by staff and short-term equately and IFC has missed opportunities to consultants (in particular) has remained unreli- have a deeper impact. Experience suggests able. Relatedly, IFC-commissioned reviews of AS the importance of arrangements to isolate facilities, products and projects, while offering portfolio problems from new business devel- insights into the organization and delivery of AS, opment, to mitigate conflicts of interest that have exhibited shortcomings in independence may impede effective collaboration with the and design. World Bank and the IMF, and to establish clear rules of engagement in crisis response, par- Charging effectively for IFC's AS is perhaps the ticularly for staff in the field. Experience also most important step going forward. Effectively indicates the important role IFC and the World charging clients for services will introduce a mar- Bank Group must play in promoting sound ket test for AS and is likely to have a positive im- frameworks for prudent financial risk man- pact on all aspects of the business, such as creating agement and safeguards to ensure sustainable incentives for: greater client buy-in, stronger proj- private sector development. This is especially ect design and implementation, stronger M&E, relevant today, as the world reexamines the development of products that best meet demand, roles of governments and markets in the wake and ensuring IFC additionality. In the immediate of the financial crisis. term, IFC would need to strictly implement the current pricing policy, which is largely cost-based IFC Advisory Services: (i.e., the price the client is expected to pay is a pro- portion of the cost of the project). Over time, · Set out an overall strategy for IFC AS that efforts should be made to move to a market value- addresses the need for a clear vision and based approach to pricing, so that IFC does not business framework, and is closely linked run the risk of crowding out other knowledge with IFC's global corporate strategy. Fol- providers. IFC investments are priced according to lowing years of unchecked growth and recent this principle for the same reason. The current eco- organizational changes, the role of AS in IFC's nomic crisis, and its likely effects on donor and IFC business model needs to be addressed. The xxv I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 strategy would need to better articulate IFC facility, product, and project reviews. In the comparative advantages in AS, as well as objec- medium term, it would pay to introduce an tives and goals for AS in different contexts (a Expanded Project Completion Report system source of confusion among staff), and to con- (akin to the Expanded Project Supervision Re- sider the best staffing combinations (internal or port system for investment operations, and external, global or local staff), delivery unit or- carried out later than the Project Completion ganization, incentives, and performance mea- Report to better capture impacts), more pro- sures to help realize these objectives and goals. grammatic impact evaluation and impact re- · Pursue more programmatic AS interven- search, the setting of results-based targets for tions. Evaluation shows that IFC has achieved AS in its corporate scorecard, and regular bench- better results in AS projects that have been marking of IFC AS activities and systems with carried out in conjunction with other AS in- other providers of knowledge services, in- terventions. One-off activities have been less ef- cluding other multilateral development banks fective. However, programmatic efforts of this and commercial providers. In the longer term, kind have been in the minority (about a fifth the aim could be to establish a specialized re- of all AS projects), and IFC should accordingly search unit focused on generating and bringing seek to expand this type of intervention. together private sector development knowl- · Improve execution of the AS pricing pol- edge work. icy through greater client contributions. Over the longer term, it would be important to This report was reviewed by an advisory panel of seek client contributions that reflect value and international experts in the area of knowledge impact (i.e., not just cost) to create a true test and development. Panel members were: Carl of client demand, to promote incentives for Dahlman, Luce Associate Professor of Interna- better AS delivery, and to ensure IFC is being tional Relations and Information Technology, additional. Georgetown University School of Foreign Ser- · Strengthen AS performance measure- vice; Acha Leke, Partner, McKinsey & Company; ment and internal knowledge manage- and Laurence Prusak, founder and former Di- ment. In the short term, it would be important rector, Institute for Knowledge Management. In a to have more hands-on M&E support in the joint statement, included in this publication, the field, post-project completion follow-up, bet- panel agreed with the above recommendations, ter lessons-capture (including from dropped and suggested additional steps IFC may take in the or terminated projects), and more arms-length same direction. xxvi Résumé analytique N ombre de pays en développement ont connu ces dix dernières an- nées une forte croissance économique, généralement accompagnée d'un recul de la pauvreté. Le secteur privé a apporté un concours cru- cial à cette croissance, essentiellement par de nouveaux investissements en capital, mais aussi par l'innovation et la création d'entreprises, qui ont contri- bué à créer des emplois et à ouvrir de nouveaux marchés. Les pays en développement où l'investissement les décideurs et les analystes à réévaluer le rôle privé a été le plus important et ceux qui ont le des marchés et du secteur privé, surtout là où la mieux réussi à réduire les écarts de savoirs et de valeur d'une réglementation, d'un contrôle pru- technologies avec les pays développés -- de l'Inde dentiel et d'une gestion fiduciaire efficaces avait aux pays baltes -- sont généralement ceux où la à tort été déconsidérée ou ignorée. croissance a été le plus rapide. Ce sont des moments où le rôle double de l'IFC, La crise financière mondiale actuelle fait peser une à la fois prestataire de financement et de savoirs grave menace sur une bonne part de ces progrès (conjointement avec la Banque mondiale) revêt durement acquis. Elle a commencé dans les pays une importance particulière. Son premier rôle, développés, mais s'est étendue ensuite aux pays celui de bailleur de fonds, est inscrit dans ses sta- en développement, et a touché particulièrement tuts, qui stipulent que l'IFC doit investir dans des ceux dont l'économie était le plus étroitement liée projets viables du secteur privé dans les pays en aux marchés mondiaux. La demande d'importa- développement « lorsqu'il n'est pas possible de tions des pays développés recule, et les entreprises se procurer à des conditions raisonnables les ca- des pays en développement, grandes et petites pitaux privés nécessaires ».1 En temps de crise, (ces dernières surtout), ont constaté aussi qu'il n'y comme maintenant, il appartient à la Société de avait plus de fonds pour de nouveaux investisse- développer ses efforts de financement. Mais elle ments, ou qu'ils étaient devenus beaucoup plus assume aussi (avec la Banque mondiale) un rôle coûteux et plus difficiles à obtenir. On s'attend à important de prestataire de savoirs, surtout ce que les courants de capitaux privés vers les pays lorsque les décideurs et les administrateurs met- en développement n'atteignent au mieux en 2009 tent l'accent sur la réglementation des affaires, la que la moitié environ du volume qu'ils avaient en- bonne gouvernance, et la viabilité environne- registré en 2007 (qui était de 1 000 milliards de mentale et sociale de la croissance. Ce rôle sup- dollars). L'expérience des crises passées donne à pose que la Société offre des avis qui contribuent penser qu'il faudra probablement plusieurs années à la définition de conditions favorables au dé- avant que ces courants retrouvent leur niveau veloppement durable du secteur privé -- en- d'avant la crise. Plus généralement, la crise a porté courageant par exemple une réglementation plus xxvii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 efficace -- et à améliorer les capacités, les com- dernier. Le volume cumulé des investissements pétences et le comportement des acteurs dont dé- actifs a augmenté d'un quart environ, passant de pendent sur le terrain les entreprises du secteur 32,7 milliards de dollars pour l'exercice 07 à 40 mil- privé (y compris à faire gérer plus efficacement les liards de dollars pour l'exercice 08. Le nombre des effets sociaux et environnementaux des activités investissements n'a pas connu un accroissement privées). du même ordre (8 %), ce qui traduit une préfé- rence générale pour les opérations d'investisse- Dans la présente évaluation indépendante des ré- ment importantes (le financement des entreprises sultats obtenus par l'IFC au plan du développe- prenant de plus en plus le pas sur le financement ment, chacun de ces deux rôles est examiné tour des projets), et une démarche plus globale attei- à tour : l'efficacité avec laquelle la Société finance gnant les petites et moyennes entreprises (PME) le développement avec son portefeuille de plus en par le biais d'intermédiaires financiers et d'en- plus important des opérations d'investissement, treprises plus importantes. l'accent étant mis en particulier sur l'expérience de l'institution au cours de crises antérieures et sur Le développement du portefeuille offre des pos- l'aide qu'elle apporte aux clients pour atténuer les sibilités d'étendre le rayon d'action de la Société risques de l'investissement (Première partie) ; et au service du développement. Les évaluations l'expérience de la Société pour ce qui est d'orga- réalisées par l'IEG des opérations d'investisse- niser et d'exécuter ses interventions de services- ment ayant atteint leur régime de croisière entre conseil -- les services que l'IFC fournit à des 2006 et 20083 font apparaître globalement une sociétés privées ou à des gouvernements dans le amélioration des résultats pour le développe- domaine du savoir pour soutenir le développe- ment des projets de l'IFC. Pour être plus précis, ment du secteur privé (Deuxième partie) ; ce 72 % des projets évalués (soit 85 % en volume) deuxième aspect étant traité pour la première ont abouti à des réalisations qui, tout bien consi- fois, il constitue le thème principal du rapport. déré, ont atteint ou dépassé les valeurs de réfé- S'agissant des résultats, le rapport est consacré aux rence et les normes financières, économiques, opérations d'investissement de l'IFC atteignant environnementales et sociales du projet, et con- leur régime de croisière entre 2006 et 2008 et aux couru utilement au développement du secteur projets de services-conseil de l'IFC pour lesquels privé par-delà le projet proprement dit. Ce pour- les rapports d'achèvement datent de la même pé- centage de projets aboutissant à des réalisations riode. L'examen de l'efficacité des services-conseil de haut niveau n'était en 2005­2007 que de 63 % est assorti de mises en garde, car le système de suivi (75 % en volume). Si on prend les pourcentages et d'évaluation n'a été introduit qu'en 2006, et le cumulés -- les évaluations indépendantes ayant transfert des savoirs est souvent de nature intan- commencé en 1996 et allant jusqu'à 2008 com- gible. Le rapport n'en offre pas moins, pour la pris -- 62 % des projets (70 % en volume) ont ob- première fois, un tableau des deux volets d'acti- tenu des appréciations élevées pour leur effet sur vité de l'IFC (investissement et services-conseil), le développement. y compris pour les situations où les deux activités ont été offertes au même client. Il vient aussi en Si les résultats sont globalement plus satisfaisants complément d'une évaluation récente du Groupe dans les premières et les dernières années de d'évaluation indépendante (IEG) portant sur l'ef- la période considérée, cela tient à plusieurs fac- ficacité des études sectorielles et économiques teurs : i) la fin d'une cohorte de projets aux ré- et de l'assistance technique de la Banque mondiale, sultats particulièrement médiocres parvenus à achevée en 2008.2 leur régime de croisière en 2005 (51 % de ces dernier projets ont abouti à des réalisations de Financement du développement haute qualité, contre 75 % de ceux atteignant Le portefeuille des opérations d'investissement de leur régime de croisière en 2008) ; ii) des condi- l'IFC (prêts, prises de participation, et autres pro- tions économiques plus favorables dans une duits financiers) a continué à se développer l'an bonne partie des pays en développement (jus- xxviii R É S U M É A N A LY T I Q U E qu'aux derniers mois de 2008, moment où pour celui de l'infrastructure et celui des marchés fi- l'essentiel la plupart des projets évalués avaient nanciers. Mais, ils n'ont pas été à la hauteur pour été exécutés) ; iii) une amélioration de l'évalua- les technologies de l'information autres que les tion préalable et de la structuration des projets de télécommunications (logiciels et Internet).4 Dans l'IFC ; iv) la préférence délibérément donnée par les autres secteurs, les projets visant le pétrole, le l'IFC aux projets de plus grande ampleur, qui ont gaz, les industries extractives et chimiques ont ob- une meilleure probabilité d'obtenir une haute tenu des résultats plutôt médiocres. L'exposition appréciation que les petits, en partie parce qu'ils au risque a manifestement joué pour la faiblesse sont suivis de plus près par la Société ; et v) des des résultats des projets informatiques autres résultats particulièrement satisfaisants en Europe que de télécommunications, qui étaient pour la et en Asie centrale, et dans la région Amérique la- plupart de petites opérations avec des entités tine et Caraïbes, où se trouvent la majorité des parrainantes inexpérimentées et une compétiti- opérations ayant atteint leur régime de croisière. vité mal définie des produits. Mais la qualité du Dans ces régions, le climat a été particulièrement travail dans ce secteur est également très en- favorable aux entreprises, et la qualité des activi- dessous de la moyenne : elle n'a été jugée élevée tés de l'IFC a été la plus élevée. Les résultats de que dans 40 % tout juste des cas. L'amélioration la région Asie du Sud s'améliorent, la qualité des de la qualité du travail est nette dans le secteur activités de la Société y étant plus élevée que par de la santé, où l'IFC a montré qu'elle avait tiré les le passé. enseignements de son expérience passée, mais le portefeuille ne s'était guère diversifié. Les projets Les résultats ont été beaucoup moins bons dans visant le pétrole, le gaz, les industries extractives les pays d'Asie de l'Est et du Pacifique et dans les et chimiques n'ont pas atteint les valeurs de ré- régions Moyen-Orient et Afrique du Nord et férence pour différentes raisons : lacunes tech- Afrique subsaharienne, qui comptent principale- niques de l'entité parrainante ; coûts d'acquisition ment des pays à revenu faible ; dans ces régions, d'avoirs plus élevés que prévu ; et, dans un cas, la moitié à peine des projets a atteint ou dépassé mauvais respect des règles environnementales. Les les valeurs de référence et les normes spécifiées. notes obtenues pour les effets environnemen- Les conditions externes en ont été en partie la taux et sociaux sont stables pour les projets phy- cause, les projets en Afrique subsaharienne et siques, mais restent faibles pour les opérations dans la région Moyen-Orient et Afrique du Nord d'intermédiation financière, signe qu'il faut ren- étaient généralement caractérisés par des niveaux forcer les capacités des clients et s'assurer de leur élevés de risques (pays, entité parrainante et com- engagement et qu'il faut aussi remédier aux in- pétitivité des produits) -- mais, dans l'ensemble suffisances aux plans de la supervision et de la va- c'est la qualité des activités de l'IFC et son leur ajoutée par la Société. concours au projet qui ont eu un impact plus marqué. C'est particulièrement vrai de la région Les résultats de dont on vient de faire état ne Asie de l'Est et Pacifique, où dans près de 40 % portent pas encore la marque de la grave dété- des projets la valeur ajoutée par l'IFC a été in- rioration de la situation économique mondiale, suffisante. Des éléments permettent de penser dont les effets commencent seulement à se faire que le tri et l'évaluation préalable des projets ont sentir sur les retours sur investissement dans la été meilleurs dans la région Moyen-Orient et plupart des pays en développement. Les résultats Afrique du Nord, et que la qualité de l'encadre- de développement indiqués ici renvoient pour ment s'est améliorée en Afrique subsaharienne. l'essentiel à des projets réalisés entre 2003 et 2008, période pendant laquelle les marchés émer- Parmi les secteurs stratégiques de l'IFC, les ré- gents ont connu une croissance sans précédent. sultats des projets ont continué à s'améliorer La plupart des projets évalués avaient été réalisés dans le domaine de la santé et celui de l'édu- pour l'essentiel, et certains étaient clos avant la cation, se sont améliorés dans celui des agro- fin de 2008, moment où la crise a commencé de industries, et sont restés de bonne qualité dans se faire sentir dans les pays en développement. xxix I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 On s'attend toutefois à ce que les résultats des répartis entre sept régions et 13 services écono- opérations atteignant leur régime de croisière miques mondiaux, et que la moitié à peu près des soient moins bons au cours des années à venir. activités de services-conseil sont confiées à des Les évaluations antérieures montrent que les pro- consultants à court terme. jets approuvés dans les années précédant une crise (et exécutés en période de contraction de En conséquence, des questions stratégiques im- l'économie) sont ceux qui sont les plus exposés portantes se posent. On peut se demander en par- au risque du point du vue du développement. En- ticulier si, greffant une activité de savoir aussi viron 40 % du portefeuille de projets de la Société importante sur une institution financière, la So- (62 % en volume) relèvent de cette catégorie, et ciété parvient à l'équilibre voulu entre services- l'IFC est donc exposée à un risque considérable conseil et services d'investissement pour obtenir à cet égard. Mais, il faut voir aussi qu'elle a nota- l'impact maximum sur le développement. Il est blement renforcé ses procédures internes de ges- possible aussi que la recherche de qualité ait tion des risques et que sa capacité de supporter été un peu sacrifiée, les modifications organisa- et de gérer les risques financiers semble s'être bien tionnelles ayant été notables, le personnel rela- améliorée ces dernières années. L'évaluation tivement nouveau (60 % des effectifs ont moins donne à penser, ce qui est important, que les in- de trois ans d'ancienneté à l'IFC) étant fortement vestissements approuvés tout de suite après la sollicité, et des activités étant externalisées chaque crise (c'est-à-dire quand la conjoncture est au année à quelque 1 300 consultants à court terme. plus bas) donnent généralement de meilleurs ré- Il existe aussi une possibilité accrue de conflit sultats au plan du développement. Il existe donc d'intérêt ou de distorsion du marché, lorsque les des opportunités qu'il faut savoir saisir. services-conseil sont offerts avec le financement, ou fournis à un prix inférieur à celui du marché. L'expérience des crises passées souligne l'im- portance pour la Société de jouer principalement La Société assure ses services-conseil pour at- sur deux tableaux : premièrement, la gestion teindre des objectifs généraux qui sont communs prudente des risques du portefeuille, particuliè- aux services-conseil et aux investissements de rement des projets en début d'exécution ; et l'IFC. Il s'agit notamment de concentrer les deuxièmement, la valeur ajoutée de l'institution. interventions sur les marchés pionniers (dont Ce deuxième aspect est particulièrement impor- les pays bénéficiant d'un financement de l'IDA et tant à deux égards : i) en jouant le rôle d'inter- les régions pionnières d'autres pays, ainsi que médiaire impartial dans les restructurations ; et les PME et les agro-industries), sur les secteurs ii) en choisissant avec soin la cible et le moment stratégiques -- finances, infrastructure, santé et de ses interventions, la Société envoie en effet un éducation -- et sur l'appui à la viabilité environ- signal fort aux autres investisseurs. nementale et sociale (changements climatiques compris). L'affectation des ressources de services- Le savoir au service du développement conseil a été pour l'essentiel alignée sur ces prio- Les services-conseil de la Société ont connu une rités. Autrement dit, les services-conseil ont croissance rapide, avec un portefeuille actif proche généralement été ciblés sur les destinations où les de 1 milliard de dollars et un personnel de 1 262 besoins sont aigus, pays bénéficiant d'un finan- agents, soit sept fois plus qu'il y a sept ans. Cette cement de l'IDA et Afrique en particulier. évolution a fortement modifié la nature et la re- présentation de l'IFC : dans les pays en dévelop- Mais la pertinence des choix n'est pas une garantie pement, le personnel des services-conseil forme d'impact. En ce qui concerne l'impact sur le dé- désormais l'essentiel des effectifs de la Société sur veloppement, 52 % des projets de services-conseil le terrain.5 Pratiquement aucune limite n'a été im- de l'IFC, parmi ceux qui pouvaient être notés, ont posée à cette croissance rapide. Cela apparaît obtenu une note élevée. Les appréciations ont clairement quand on constate qu'il y plus de 50 été nettement plus élevées pour d'autres aspects produits des services-conseil, 18 centres régionaux des résultats, tels que la pertinence stratégique, xxx R É S U M É A N A LY T I Q U E les produits, et l'obtention de réalisations, l'effi- chant à en améliorer l'alignement organisationnel cacité globale pour le développement étant bonne et les modes d'exécution. On a cherché ces der- dans 70 % des cas. Les appréciations n'ont pas été nières années à mieux structurer les services- très différentes pour les projets démarrés avant conseil et à les rendre plus clairs, notamment par (plutôt qu'après) les grands changements orga- les mesures suivantes : répartition des activités nisationnels de 2005/2006. Par région, les appré- de services-conseil en cinq catégories ; fusion de ciations ont été nettement meilleures pour certains mécanismes mondiaux et régionaux ; l'Europe du Sud et l'Asie centrale, et moins bonnes classification des produits par degré de matu- pour l'Amérique latine et les Caraïbes. Les projets rité ; valorisation des compétences du personnel mondiaux évalués n'avaient pas non plus eu de des services-conseil ; formation aux services- bons résultats. Par catégorie d'activités, les varia- conseil ; création d'une vice-présidence pour les tions étant toutefois moins prononcées qu'entre services-conseil. S'agissant de l'exécution de ces régions, les projets visant l'infrastructure, le climat services, la Société s'est efforcée de mettre en favorable aux entreprises, les conseils aux entre- place des dispositifs et des systèmes garantissant prises et l'accès à des financements avaient eu dans : un financement suffisant et viable ; l'engage- l'ensemble de meilleurs résultats que ceux visant ment des clients ; une conception et une bonne la viabilité environnementale et sociale. exécution des projets ; un suivi et une évaluation robustes des résultats. Ce que l'IFC a fait en ce sens Les facteurs déterminant les résultats ont été l'en- semble soutenir favorablement la comparaison gagement des clients (manifesté par la contribu- avec les mesures prises par d'autres banques mul- tion aux coûts des projets, en particulier pour tilatérales de développement, par exemple en ce les projets visant la viabilité environnementale et qui concerne l'introduction de politiques de fixa- sociale), une conception et une exécution adé- tion des prix (qui, de manière générale, visent à quate des projets, la proximité de l'IFC par rap- développer l'engagement des clients, et à réduire port au client (définie par la présence locale et les risques de distorsion du marché, en limitant l'implication de la Société), le caractère pro- les subventions aux biens publics), et d'un système grammatique (et non pas ponctuel) des inter- de suivi et d'évaluation qui permette de mesurer ventions, et l'efficacité du suivi et de l'évaluation. les issues et les impacts plutôt que les seuls pro- La forte valeur ajoutée par l'IFC a joué un rôle fon- duits. Le mouvement de transformation se pour- damental pour l'obtention de résultats, ce qui a suit, de nouvelles politiques, procédures et été particulièrement facile à constater dans les ac- directives ayant récemment été introduites rela- tivités visant un climat favorable dans les pays tivement à la fixation des prix, aux conflits d'intérêt, IDA où le risque lié aux conditions économiques au financement et à la gouvernance. est élevé, et dans certains ensembles de services, tels que les projets d'établissement de liens avec Mais on n'a pas fini de donner un caractère plus les PME dans l'agro-industrie, les industries ma- professionnel aux services-conseil, et des pro- nufacturières et extractives. La constitution de blèmes organisationnels notables persistent : ce type d'ensembles peut donner lieu à des structures d'exécution parallèles ou qui se che- conflits d'intérêt, auxquels il importe de bien vauchent dans plusieurs régions (Afrique subsa- parer, et le prix en est à établir soigneusement. Les harienne, Asie de l'Est et Pacifique, Asie du Sud) difficultés que l'on rencontre toujours pour me- ; peu de produits bien établis en dehors de la fi- surer l'impact des services-conseil sont encore ac- nance et de l'infrastructure ; manque de précision crues par le fait que jusqu'à présent, le personnel sur la meilleure manière d'intégrer services-conseil de l'IFC n'a pas appliqué très strictement les di- et services d'investissement dans différents rectives de suivi et d'évaluation. contextes ; peu de prise en compte des avan- tages comparatifs de l'IFC par rapport à d'autres Au cours des cinq dernières années, la direction prestataires de services dans le domaine du savoir, de l'IFC a pris des mesures pour renforcer l'effi- à l'échelon stratégique et à celui des projets ; et cacité des services-conseil de la Société, cher- manque de cadre stratégique global coiffant les xxxi I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 services-conseil et permettant de combiner har- fonction du marché, afin que l'IFC ne risque pas monieusement différents axes d'activité. d'évincer les autres prestataires de savoirs. C'est pour la même raison que le prix des investisse- On observe aussi des déficiences dans certains do- ments de l'IFC est fixé selon ce principe. La crise maines, dont il faudra se préoccuper, surtout si économique actuelle, et les effets qu'elle aura l'on veut que les résultats obtenus sur le terrain probablement sur le financement des donateurs reflètent bien le but recherché par l'institution. et celui de l'IFC, donnent à la Société l'occasion Cette considération s'applique à l'application des de chercher plus énergiquement à fixer les prix en politiques de fixation des prix autant qu'à la qua- fonction de la valeur, et d'encourager les autres ins- lité de la conception et de l'exécution des projets, titutions de développement à faire de même. et à une bonne collaboration avec les autres acteurs, dont la Banque mondiale. Le dosage Recommandations judicieux du personnel a donné beaucoup de L'évaluation présentée ici vient à un moment de mal, du fait qu'une bonne part des activités est profonde crise des marchés financiers, et de confiée à des consultants à court terme, et à des contraction grave des activités économiques pri- membres relativement nouveaux du personnel vées. Elle nous rappelle l'importance cruciale (si on les compare à ceux qui s'occupent des in- d'un développement durable du secteur privé, vestissements). La combinaison choisie retentit pour lequel des dispositifs réglementaires sont in- fortement sur la qualité et la continuité des dispensables, la déréglementation excessive se services-conseil de l'IFC, et sur le maintien de sa payant fort cher. Dans ces circonstances, on avance position de pointe en matière de savoirs. À tous ici d'autres recommandations que l'IFC pourrait les stades de l'exécution, les données de suivi et suivre pour améliorer son efficacité pour le dé- d'évaluation communiquées par le personnel et veloppement et sa valeur ajoutée : les consultants à court terme (surtout) sont res- tées peu fiables. De même, les examens par l'IFC Opérations pendant la crise : de ses mécanismes, produits et projets de services- conseil, s'ils ont livré des idées sur l'organisation · Bien gérer la tension entre la protection et l'exécution des services-conseil, ont manqué du portefeuille et le parti à tirer des op- d'indépendance et la conception n'en a pas été portunités en temps de crise. Cette ten- entièrement satisfaisante. sion n'a pas toujours été bien gérée par le passé, et la Société n'a pas tiré parti d'occasions d'ap- L'étape suivante la plus importante pour pro- profondir son impact. L'expérience donne à gresser est peut-être d'arriver à fixer le bon prix penser qu'il est important de faire la distinction pour les services-conseil de l'IFC. En facturant entre les problèmes des opérations en porte- convenablement les services aux clients, on sou- feuille et les nouveaux projets, d'atténuer les mettra les services-conseil de la Société à un test conflits d'intérêt qui peuvent empêcher de col- de marché, et on obtiendra très probablement laborer utilement avec la Banque mondiale et un impact favorable sur tous les aspects des in- le FMI, et de définir clairement les domaines terventions : en incitant les clients à davantage s'in- d'intervention face à la crise, en particulier pour vestir dans les projets, en améliorant la conception le personnel sur le terrain. Elle indique aussi qu'il et l'exécution des projets, en renforçant le suivi et appartient à l'IFC et au Groupe de la Banque l'évaluation, en définissant des produits qui ré- mondiale d'assumer un rôle important en faveur pondent mieux à la demande, et veillant à ce qu' de dispositifs bien pensés de gestion prudente l'IFC offre une valeur ajoutée. Dans l'immédiat, la des risques financiers, et de garanties proté- Société aura à appliquer strictement les politiques geant le développement durable du secteur de prix en vigueur, qui sont pour l'essentiel basées privé. Ces considérations sont particulièrement sur les coûts (le prix que le client est appelé à payer pertinentes à l'heure actuelle, au moment où est proportionnel aux coûts du projet). À plus la crise financière fait repenser dans le monde long terme, il faudra chercher à fixer les prix en le rôle des gouvernements et celui des marchés. xxxii R É S U M É A N A LY T I Q U E Rôle de l'IFC pour les services-conseil : en interne. À court terme, il importe d'avoir un appui plus concret de suivi et d'évaluation · Définir une stratégie globale pour les sur le terrain, d'assurer le suivi après l'achè- services-conseil de l'IFC, répondant à la né- vement des projets, de tirer les enseignements cessité d'une perspective et d'un schéma des projets abandonnés ou éliminés, et d'as- d'activité clairs, et en liaison étroite avec surer un examen indépendant des méca- la stratégie d'ensemble de la Société. Après nismes, des produits et des projets par des des années de croissance incontrôlée et à la suite entités qui en soient plus éloignées. À moyen des transformations organisationnelles récentes, terme, il serait utile de mettre en place un il est temps de réfléchir au rôle que jouent les système de rapports étendus d'achèvement services-conseil dans le modèle d'activité de de projet (XPCR) (semblable au système de l'IFC. Cette stratégie devra mieux préciser les rapports étendus de supervision de projets avantages comparatifs de la Société en matière -- système XPSR -- mis en place pour les de services-conseil, les buts et objectifs pour- opérations d'investissement, les rapports suivis par ces services dans différents contextes étant réalisés après les rapports d'achève- (qui donnent lieu à des incertitudes pour le ment de projets afin de mieux cerner les im- personnel), et donner des indications sur les pacts), d'évaluer les impacts et de mener la meilleures combinaisons de personnel (per- recherche sur ces derniers selon une approche- sonnel interne/personnel extérieur, personnel programme, de fixer aux services-conseils mondial/personnel local), sur l'organisation des des objectifs axés sur les résultats dans sa fiche services chargés de l'exécution, sur les moyens de réalisations, et de jauger régulièrement les d'incitation et d'évaluation des résultats, afin activités et les systèmes de services-conseil d'aider à atteindre ces buts et objectifs. de l'IFC par rapport à d'autres prestataires · Replacer les interventions des services- de services dans le domaine du savoir, dont conseil dans une approche programme. les autres banques multilatérales de dévelop- L'évaluation montre que l'IFC a obtenu de pement et les sociétés commerciales. À plus meilleurs résultats avec les projets de services- long terme, on pourrait songer à créer un ser- conseil réalisés conjointement à d'autres in- vice de recherches spécialisées qui s'occupe- terventions de services-conseil. Les projets rait de susciter et de réunir des travaux sur les autonomes ont eu moins d'efficacité. Pourtant savoirs relatifs au développement du secteur les projets inscrits dans un programme sont privé. moins nombreux (le cinquième environ de l'ensemble des projets de conseil), et la Société Le rapport qui suit a été revu par un Groupe devrait donc chercher à développer ce type consultatif de spécialistes internationaux du savoir d'intervention. au service du développement. Les membres en · Améliorer l'application des politiques de étaient le professeur Carl Dahlman, Luce, pro- fixation des prix des services-conseil. À fesseur adjoint (Relations internationales et tech- plus long terme, il serait important d'obtenir nologie de l'information) à la School of Foreign des clients qu'ils évaluent les interventions en Service de Georgetown University ; Acha Leke, termes de valeur et d'impact (et non pas seu- Partenaire du cabinet McKinsey & Company ; et lement de coûts), ce qui permettrait de mieux Laurence Prusak, fondateur et ancien directeur de cerner la demande et de créer de meilleures in- l'Institute for Knowledge Management. Dans une citations pour la prestation de services-conseil, déclaration conjointe, figurant dans la présente pu- et qui garantirait la valeur ajoutée des inter- blication, le groupe a souscrit aux recommanda- ventions de l'IFC. tions énoncées plus haut, et suggéré d'autres · Renforcer l'évaluation des résultats des mesures allant dans le même sens que pourrait services-conseil et la gestion du savoir prendre l'IFC. xxxiii Resumen ejecutivo E n la última década, muchos países en desarrollo experimentaron un vigoroso crecimiento económico, en general acompañado por la dis- minución de los niveles de pobreza. El sector privado contribuyó en forma decisiva a ese crecimiento, principalmente a través de nuevas inversiones de capital, pero también de innovaciones y espíritu empresarial, contribuyendo a crear puestos de trabajo y abrir nuevos mercados. En general, el crecimiento más acelerado se dio antes de que esas corrientes vuelvan a alcanzar sus en los países en desarrollo que registran los más niveles anteriores a la crisis. Ésta, en forma más altos niveles de inversión privada y en que se ha general, ha llevado a los responsables de políticas logrado cerrar en mayor medida los vacíos de y a los analistas a revisar sus conceptos sobre el conocimientos y tecnología que los separan del papel de los mercados y el sector privado, espe- mundo desarrollado. India y los Estados bálticos cialmente en los casos en que se incurrió en el son ejemplos al respecto. error de dejar de hacer hincapié en la importan- cia de una reglamentación, supervisión pruden- La actual crisis financiera mundial amenaza gra- cial y administración fiduciaria eficaces, o no se vemente muchos de esos logros, arduamente tuvo en cuenta esa importancia. alcanzados. La crisis se inició en el mundo desa- rrollado, pero desde entonces se ha propagado al En épocas como la actual asume especial impor- mundo en desarrollo, y ha afectado especialmente tancia la doble función de la IFC como fuente de a los países cuyas economías están más estre- financiamiento y como proveedora de conoci- chamente vinculadas con los mercados mundia- mientos (junto con el Banco Mundial). Con res- les. La demanda de importación proveniente de pecto al primero de esos cometidos, el Convenio los países desarrollados está disminuyendo, y en Constitutivo de la IFC establece que la Corpora- los países en desarrollo grandes y pequeñas com- ción debe invertir en proyectos viables para el sec- pañías (especialmente estas últimas) también se tor privado en los países en desarrollo para los han visto confrontadas con el agotamiento del fi- cuales `el capital privado suficiente no se en- nanciamiento para nuevas inversiones, o con un cuentre disponible en condiciones razonables'.1 muy pronunciado aumento del costo de este úl- En esas épocas de crisis, la carga de incrementar timo y una mayor dificultad para obtenerlo. Según el financiamiento recae sobre la IFC, pero ésta se prevé, en 2009 la afluencia de capital privado (junto con el Banco Mundial) también cumple un a los países en desarrollo alcanzará a lo sumo papel importante como proveedora de conoci- un nivel de alrededor de la mitad del de 2007 mientos, en especial en un período en que los res- (US$1 billón). Las crisis anteriores llevan a pensar ponsables de políticas y los administradores que pueden tener que transcurrir algunos años centran la atención en la reglamentación de los xxxv I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 negocios, una buena gestión y la sostenibilidad diente (IEG, por sus siglas en inglés), completada ambiental y social del crecimiento. Esta función en 2008, sobre la eficacia de los estudios econó- implica el suministro de una asistencia que con- micos y sectoriales y la asistencia técnica del tribuya a dar forma a las condiciones apropiadas Banco Mundial.2 para un desarrollo sostenible del sector privado --por ejemplo, promoviendo una reglamenta- Financiamiento para el desarrollo ción más eficaz-- y tienda a reforzar la capacidad, La cartera de operaciones de inversión de la IFC las aptitudes y el proceder de los actores que (préstamos, inversiones de capital y otros pro- realizan actividades conjuntas sobre el terreno ductos financieros) siguió aumentando el año pa- con las empresas del sector privado (aspecto que sado. El volumen acumulativo de las actividades de comprende un eficaz manejo de los efectos so- inversión activas aumentó alrededor de un 25% ciales y ambientales de las actividades privadas). --de US$32.700 millones a US$40.000 millones-- entre los ejercicios de 2007 y 2008. El número de En la presente Evaluación Independiente de los inversiones aumentó en menor proporción (un Resultados de Desarrollo (IEDR, por sus siglas 8%), lo que refleja una preferencia general por en inglés) de la IFC se examinan sucesivamente operaciones de inversión de mayor porte (orien- esas funciones: la eficacia de la IFC en cuanto a tadas cada vez más a financiamiento institucional, financiamiento del desarrollo a través de su cre- en lugar de financiamiento para proyectos), y con ciente cartera de operaciones de inversión, ha- un enfoque más "mayorista" para llegar a peque- ciendo hincapié en la experiencia lograda por la ñas y medianas empresas (es decir, a través de in- Corporación en crisis anteriores y ayudando a termediarios financieros y compañías más grandes). los clientes a mitigar los riesgos de la inversión (parte I), y, por primera vez, y por lo tanto como Una cartera creciente brinda oportunidades tema principal del presente informe, la expe- para ampliar el horizonte de desarrollo de la riencia adquirida por la Corporación en la orga- Corporación. Las evaluaciones del IEG sobre las nización y ejecución de sus intervenciones en operaciones de inversión que llegaron a su ven- materia de servicios de asesoría (SA), que con- cimiento operativo anticipado entre 2006 y 20083 sisten en servicios de conocimiento que la IFC pro- muestran una mejora global de los resultados porciona a compañías privadas o gobiernos para en términos de desarrollo de los proyectos de la respaldar el desarrollo del sector privado (parte IFC. Más concretamente, el 72% de los proyectos II). En cuanto a resultados, en el informe se exa- evaluados (el 85% por volumen) lograron resul- minan las operaciones de inversión de la IFC que tados que en términos generales cumplieron o llegaron a su vencimiento operativo anticipado excedieron los parámetros e indicadores de re- entre 2006 y 2008, y los proyectos de servicios de ferencia financieros, económicos, ambientales asesoría dispensados por la IFC en el mismo pe- y sociales de los proyectos, y realizaron contri- ríodo con informes de terminación de proyectos. buciones positivas al desarrollo del sector pri- Con respecto al examen de la eficacia en térmi- vado más allá del proyecto. En comparación, en nos de desarrollo de los servicios de asesoría se 2005­07 el 63% de los proyectos (el 75% por vo- formulan ciertas advertencias, dado que el sistema lumen) lograron buenos resultados. En cifras de seguimiento y evaluación recién se introdujo acumulativas, como la evaluación independiente en 2006, y habida cuenta del carácter a menudo se inició en 1996 y llegó a 2008 inclusive, el 62% intangible de la transmisión de conocimientos. No de los proyectos (el 70% por volumen) lograron obstante, el informe presenta por primera vez buenas calificaciones de cuanto a resultados en una reseña coordinada de ambos brazos de las términos de desarrollo. actividades de la IFC (inversiones y servicios de asesoría), incluidas situaciones en que esos ins- Los resultados globales más satisfactorios logrados trumentos se ofrecieron al mismo cliente. En el en los años más distantes obedecieron a varios fac- informe también se complementa una reciente tores: i) la salida de una cohorte de desempeño evaluación del Grupo de Evaluación Indepen- especialmente inadecuado de proyectos que xxxvi R E S U M E N E J E C UT I VO alcanzaron su madurez en 2005 (el 51% de los pro- y educación; mejoró en agroindustrias, y siguió yectos en proceso de maduración en 2005 logra- siendo vigoroso en infraestructura y mercados ron buenos resultados en términos de desarrollo, financieros. En tecnología de la información no en comparación con el 75% en proceso de ma- relacionada con telecomunicaciones (software e duración en 2008); ii) condiciones económicas Internet) el desempeño fue menos satisfactorio.4 más favorables en gran parte del mundo en de- En otros sectores --petróleo, gas, minería y sarrollo (hasta fines de 2008, en que la mayoría productos químicos-- los proyectos alcanzaron de los proyectos evaluados se habían ejecutado calificaciones relativamente insatisfactorias. Evi- en considerable medida); iii) mejoras en cuanto dentemente el riesgo explica en parte los resul- a calidad de la evaluación inicial y estructuración tados inadecuados de los proyectos de tecnología de los proyectos de la IFC; iv) deliberada orien- de la información no relacionada con telecomu- tación de la IFC hacia proyectos de mayor escala, nicaciones, la mayoría de los cuales consistieron con mayor probabilidad de alcanzar altas califi- en pequeñas operaciones con patrocinadores no caciones que los proyectos más pequeños, en experimentados y falta de claridad en cuanto a parte debido a una más severa fiscalización interna, competitividad de productos. No obstante, la ca- y v) desempeño especialmente sólido en Europa lidad del trabajo realizado en ese sector fue tam- y Asia central, y en América Latina y el Caribe, en bién francamente inferior a la par, ya que sólo fue que se realiza la mayoría de las operaciones que elevada en el 40% de los casos. La calidad del tra- han madurado. En esas regiones las condiciones bajo mejoró manifiestamente en el sector de la para los negocios son más propicias, y es más só- salud, en que la IFC dio muestras de haber asi- lida la calidad del trabajo de la IFC. En Asia meri- milado la experiencia del pasado, aunque poco se dional hubo mejoras en cuanto a desempeño y logró en cuanto a diversificación de la cartera. calidad del trabajo de la IFC. Varios factores impidieron que los proyectos de petróleo, gas, minería y productos químicos al- Estuvo considerablemente a la zaga la región de canzaran los indicadores de referencia: fallas téc- Asia oriental y el Pacífico; lo propio aconteció con nicas de los patrocinadores; costo de adquisición las regiones de Oriente Medio y Norte de África, de activos mayor del previsto, y un caso de cum- y África al sur del Sahara, cuyos países son princi- plimiento insatisfactorio de normas ambientales. palmente de ingreso bajo. Apenas la mitad de los Las calificaciones de efectos ambientales y socia- proyectos ejecutados en esas regiones alcanza- les fueron estables para los proyectos del sector ron o superaron los parámetros e indicadores de real, pero siguieron siendo insatisfactorias en las referencia especificados, lo que obedeció en parte operaciones para intermediarios financieros, lo a condiciones externas, ya que los proyectos rea- que refleja la necesidad de fortalecer la capacidad lizados en las regiones de África al sur del Sahara de los clientes y lograr su identificación con los y Oriente Medio y Norte de África en general pre- proyectos, y de hacer frente a las fallas de super- sentaron altos niveles de riesgo de país, de pa- visión y adicionalidad de la IFC. trocinador y de competitividad de productos, pero la calidad del trabajo y la contribución de la Los resultados en términos de desarrollo arriba IFC a los proyectos tendieron a suscitar mayores mencionados no reflejan aún el pronunciado de- impactos. Así sucedió en especial en Asia oriental terioro de las condiciones económicas mundiales, y el Pacífico, en que casi el 40% de los proyectos que recién ahora comienza a afectar a la rentabi- alcanzaron bajos niveles en cuanto a calidad de la lidad de las inversiones en la mayoría de los paí- adicionalidad de la IFC. Se ha comprobado un ses en desarrollo. Los resultados de ese género mejor trabajo de selección y evaluación inicial en que aquí se mencionan reflejan en gran medida Oriente Medio y Norte de África, y mejor calidad la experiencia de los proyectos en el período de la supervisión en África al sur del Sahara. 2003­2008, en que se registró un crecimiento sin precedentes en los mercados emergentes. La En los sectores estratégicos de la IFC, el desem- mayoría de los proyectos evaluados se habían peño de los proyectos siguió mejorando en salud ejecutado en considerable medida, y algunos se xxxvii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 habían cerrado a fines de 2008, cuando la crisis está próximo a los US$1.000 millones, y el personal comenzó a afectar al mundo en desarrollo. dedicado a esa labor está compuesto por 1.262 funcionarios, es decir que se septuplicó en los úl- Se prevé, en cambio, un deterioro de los resul- timos siete años. En consecuencia, las caracte- tados en términos de desarrollo de las operacio- rísticas y la apariencia de la IFC han cambiado nes que vayan alcanzando su madurez en los significativamente. En la actualidad el personal des- próximos años. Las evaluaciones anteriores mues- tinado a servicios de asesoría representa la mayor tran que la mayoría de los proyectos aprobados parte de la presencia de la Corporación sobre el en los años anteriores a la crisis (y que se están terreno en los países en desarrollo.5 El acelerado ejecutando en el período de cambio desfavorable crecimiento de esos servicios en general no ha tro- de la coyuntura) corren riesgos desde la pers- pezado con obstáculo alguno, como lo ilustra cla- pectiva del desarrollo. Esta categoría comprende ramente el surgimiento de más de 50 productos aproximadamente el 40% de las operaciones de de servicios de asesoría, 18 servicios regionales la cartera de la IFC (el 62% por volumen), con lo que abarcan siete regiones, 13 unidades de ne- cual la Corporación está expuesta a un conside- gocios mundiales, a lo que se agrega el hecho de rable riesgo de que no se alcancen los objetivos que alrededor de la mitad de la labor relacionada fijados en materia de desarrollo. Al mismo tiempo con dichos servicios se da en contrato a consul- la IFC ha fortalecido considerablemente sus pro- tores a corto plazo. cesos internos de gestión de riesgos, y su capa- cidad de soportar y manejar riesgos financieros A continuación se plantean importantes pregun- parece haber mejorado significativamente en los tas estratégicas. Una de ellas consiste en saber si últimos años. Un hecho importante que surge al reunir una proporción tan considerable de los de la evaluación es que las inversiones aprobadas negocios del conocimiento en una institución tras la crisis (es decir en el punto más desfavora- financiera, la IFC cuenta con el equilibrio de es- ble del ciclo económico) tenderán a lograr mejores fuerzos apropiado entre servicios de asesoría y ser- resultados en términos de desarrollo. En conse- vicios de inversiones como para lograr el máximo cuencia, también existirán oportunidades de al- impacto posible en el desarrollo. También es po- canzar resultados que superen las proyecciones, sible llegar a soluciones de compromiso de cali- y es necesario aprovecharlas. dad adecuada, dado el considerable cambio institucional, el alto grado de utilización de per- La experiencia de crisis anteriores pone especial- sonal relativamente nuevo (el 60% de los funcio- mente de manifiesto la necesidad de dos respuestas narios trabajan en la IFC desde hace menos de tres clave de parte de la IFC: primero, una cuidadosa años), y la tercerización de la labor a través de la gestión de riesgos de la cartera, especialmente en utilización, cada año, de 1.300 consultores a corto el caso de los proyectos que se encuentran en eta- plazo. Ha aumentado también la posibilidad de pas tempranas de ejecución; segundo, velar por conflictos de intereses o distorsiones del mer- la adicionalidad de la contribución de la IFC. Esta cado, cuando los servicios de asesoría se ofrecen última respuesta es especialmente importante en junto con el financiamiento o se dispensan a un dos aspectos: i) la actuación de la Corporación valor inferior al de mercado. como intermediario honesto en las reestructura- ciones, y ii) la búsqueda de un enfoque oportuno La IFC despliega sus servicios de asesoría procu- y bien focalizado frente a las nuevas operaciones, rando alcanzar objetivos generales, comunes con especialmente a través del efecto de señal para los de las inversiones de la Corporación: ocu- otros inversionistas que pueden suscitar las inter- parse de los mercados de frontera (incluidos paí- venciones de la IFC. ses de la Asociación Internacional de Fomento --AIF-- y regiones de frontera de países que no Conocimiento para el desarrollo reciben financiamiento de la AIF, así como pe- Los servicios de asesoría de la IFC han venido cre- queñas y medianas empresas y agroindustrias) y ciendo rápidamente; el monto de la cartera activa de sectores estratégicos (finanzas, infraestruc- xxxviii R E S U M E N E J E C UT I VO tura, salud y educación), y dar respaldo a la sos- condiciones propicias para los negocios realiza- tenibilidad ambiental y social (incluido el cambio das en países de la AIF, en que el riesgo que afecta climático). En gran medida la asignación de re- al clima de negocios es alto, y en algunos paque- cursos de los servicios de asesoría ha sido con- tes de servicios, como el de los proyectos de vin- gruente con esas prioridades. En otros términos, culación con pequeñas y medianas empresas en los servicios de asesoría de la IFC en general se los sectores de agroindustrias, manufacturero y han orientado hacia destinos en que mucho se ne- extractivo. La formación de paquetes da lugar cesitan, como los países de la AIF, y en especial a potenciales conflictos de intereses que es ne- de África. cesario abordar eficazmente y señalar los precios apropiados. Las dificultades inherentes a la de- Pertinencia, sin embargo, no es garantía de im- terminación del impacto de los servicios de pacto. El 52% de los proyectos del servicio de ase- asesoría se ven agravadas por la aplicación, rela- soría de la IFC que fue posible calificar ocuparon tivamente insatisfactoria hasta la fecha, de las di- un alto rango en cuanto a impacto en el desarro- rectrices de seguimiento y evaluación por parte llo logrado. La calificación de los proyectos fue sus- del personal de la IFC. tancialmente más alta en otras dimensiones del desempeño, como pertinencia estratégica, pro- En los últimos cinco años la administración de la ducto y logro de resultados, en que se alcanzó una IFC, para lograr mayor eficacia a través de los ser- tasa global de éxito del 70% en cuanto a eficacia vicios de asesoría, ha procurado fortalecer la ali- en términos de desarrollo. Las calificaciones no va- neación institucional y los procesos de ejecución. riaron significativamente por el hecho de que los Los siguientes son algunos de los esfuerzos rea- proyectos hubieran sido iniciados antes o des- lizados en los últimos años para estructurar y pués de las grandes reformas institucionales rea- aclarar mejor los servicios de asesoría de la IFC: lizadas en 2005/06. Por región, las calificaciones han clasificación de las actividades de servicios de sido sustancialmente mejores en Europa meri- asesoría en cinco líneas de negocios; consolida- dional y Asia central, y menos satisfactorias en ción de algunos servicios mundiales y regionales; América Latina y el Caribe. Los proyectos mundiales clasificación de productos por niveles de madu- evaluados tampoco lograron resultados adecuados. rez; desarrollo de competencias del personal de Por línea de negocios, aunque la variación de los los servicios de asesoría; capacitación en servicios resultados es menos pronunciada que a escala de asesoría, y establecimiento de una Vicepresi- regional, los proyectos de infraestructura, condi- dencia de Servicios de Asesoría. La atención de la ciones propicias para los negocios, asesoría a em- IFC en cuanto a prestación de servicios de asesoría presas y acceso al financiamiento tienden a lograr se ha centrado en el establecimiento de meca- mejores resultados que los de sostenibilidad am- nismos y sistemas que garanticen un financia- biental y social. miento adecuado y sostenible; identificación de los clientes con los proyectos, diseño y ejecu- Han sido factores determinantes de los resultados ción bien concebidos de los proyectos, y sólido la identificación de los clientes con las operacio- seguimiento y evaluación de desempeño. La labor nes (reflejada en contribuciones al costo de los realizada por la IFC en esos ámbitos parece ha- proyectos, en especial para proyectos de soste- ber dado mejores resultados que la realizada por nibilidad ambiental y social), sólido diseño y eje- otros bancos multilaterales de desarrollo, por cución de los proyectos, proximidad de la IFC al ejemplo en cuanto a introducción de una política cliente, definida por la presencia y participación de precios (encaminada en términos generales a local en la Corporación, intervenciones progra- crear identificación de los clientes con sus pro- máticas (en lugar de intervenciones por única yectos y reducir posibles distorsiones del mercado vez) y seguimiento y evaluación eficaces. Una limitando los subsidios a los bienes públicos), y fuerte adicionalidad ha sido un factor fundamental establecimiento de un sistema de seguimiento y para lograr resultados, y ha sido especialmente evaluación a través del cual se procura captar in- perceptible entre las operaciones de creación de formación sobre resultados e impactos, y no tan xxxix I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 sólo productos. El impulso de la transformación servicios de asesoría muestran fallas en cuanto a se mantiene a través de la reciente introducción independencia y diseño. de nuevas políticas, procedimientos y directrices relacionados con fijación de precios, conflictos de Quizás el paso más importante para avanzar con- intereses, financiamiento y conducción adecuada. siste en que la IFC cobre efectivamente los ser- vicios de asesoría que presta. Ello representará una La profesionalización de los servicios de asesoría, prueba de mercado para dichos servicios y es en cambio, sigue siendo una labor en curso y probable que repercuta positivamente sobre todos subsisten considerables problemas instituciona- los aspectos de las actividades creando incentivos les: estructuras de ejecución superpuestas y pa- para generar mayor aceptación en los clientes, dar ralelas en varias regiones (África al sur del Sahara, mayor solidez al diseño y la ejecución de los pro- Asia oriental y el Pacífico y Asia meridional); pocos yectos, fortalecer el seguimiento y la evaluación, productos firmemente establecidos, salvo los de crear productos más adecuados para satisfacer la finanzas e infraestructura; falta de claridad acerca demanda y garantizar la adicionalidad de la IFC. de la mejor manera de integrar los servicios de En el plazo inmediato sería necesario que la IFC asesoría con los de inversiones en diferentes aplicara estrictamente la actual política de precios, contextos; escasa consideración de las ventajas que en gran medida se basa en los costos (el pre- comparativas de la IFC, a nivel estratégico y de pro- cio que se prevé que pague el cliente es una pro- yectos, frente a otros proveedores de servicios del porción del costo del proyecto). Con el tiempo conocimiento, e inexistencia de un marco estra- se debería tratar de basar los precios en el valor tégico paraguas de servicios de asesoría para en- de mercado para que la IFC no corra el riesgo de lazar diferentes pasos. desplazar a otros proveedores de conocimien- tos. Es por esa razón que los precios de las in- Hay también considerables vacíos, que es necesario versiones de la Corporación se basan en ese llenar, en materia de ejecución, especialmente en principio. La actual crisis económica y sus pro- cuanto a coincidencia del designio institucional con bables efectos sobre el financiamiento prove- una ejecución sobre el terreno compatible con esa niente de donantes y de la IFC constituyen una intención. Esta observación se aplica a la ejecución oportunidad para que la Corporación se esfuerce de la política de precios y al logro de un diseño y más en basar los precios en el valor de mercado ejecución cualitativamente satisfactorios de los e inste a hacer lo propio a otras instituciones de proyectos, así como una eficaz colaboración con asistencia para el desarrollo. otros actores, incluido el Banco Mundial. Uno de los problemas ha consistido en lograr la combi- Recomendaciones nación apropiada del personal, habiéndose recu- El presente examen se da en un contexto de pro- rrido en gran medida a consultores a corto plazo fundas dificultades en los mercados financieros y y a funcionarios relativamente nuevos (en com- grave reducción de la escala de las actividades eco- paración con los que toman parte en operaciones nómicas privadas, lo que nos recuerda la decisiva de inversión). La combinación elegida influye po- importancia de un desarrollo sostenible en el derosamente sobre la calidad y continuidad de los sector privado, para el que revisten importancia servicios de asesoría de la IFC, y sobre la preser- los marcos regulatorios y una excesiva desregu- vación del liderazgo mundial en materia de co- lación resulta costosa. En tales circunstancias este nocimiento. En todas las etapas de la ejecución, examen lleva a constataciones adicionales sobre los datos de seguimiento y evaluación propor- lo que podría hacer la IFC para lograr mayor efec- cionados (en especial) por funcionarios y con- tividad en el desarrollo y mayor adicionalidad: sultores a corto plazo aún no son confiables. Un Operaciones durante la crisis: hecho conexo es que los exámenes de servicios, productos y proyectos de los servicios de aseso- · Realizar una eficaz gestión de la tensión ría encargados por la IFC, si bien permiten cono- entre protección de la cartera y respuesta cer aspectos de la organización y prestación de a las oportunidades que se presenten du- xl R E S U M E N E J E C UT I VO rante la crisis. En el pasado esa gestión no ciones de servicios de ese estilo. Menos efica- siempre ha sido adecuada, y la IFC ha perdido ces han sido las actividades realizadas por única oportunidades de suscitar impactos más pro- vez. No obstante, la labor programática de este fundos. La experiencia indica la importancia de tipo ha representado la minoría (alrededor de contar con sistemas que aíslen los problemas un quinto de los proyectos de servicios de ase- de la cartera del desarrollo de nuevos negocios, soría), por lo cual la IFC debería tratar de am- mitigar conflictos de intereses que puedan di- pliar ese tipo de intervenciones. ficultar una eficaz colaboración con el Banco · Mejorar la ejecución de la política de fi- Mundial y el FMI, y disponer de claras normas jación de precios de los servicios de ase- de participación en respuesta a la crisis, espe- soría. A más largo plazo sería importante tratar cialmente para el personal que actúa sobre el de obtener contribuciones de los clientes que terreno. La experiencia revela también el im- reflejen el valor y el impacto (es decir, no tan portante papel que deben cumplir la IFC y el sólo el costo), para crear una genuina prueba Grupo del Banco Mundial como promotores de de demanda de los clientes, incentivos para una sólidos marcos de gestión prudente del riesgo mejor prestación de servicios de asesoría, y financiero y salvaguardias que garanticen un de- como garantía de la adicionalidad de la IFC. sarrollo sostenible del sector privado, lo que re- · Fortalecer la medición del desempeño de viste especial importancia en la actualidad, en los servicios de asesoría y la gestión interna que el mundo está reconsiderando las funcio- del conocimiento. A corto plazo sería impor- nes que deben cumplir los gobiernos y los tante disponer de un mayor respaldo práctico mercados a raíz de la crisis financiera. de seguimiento y evaluación sobre el terreno, seguimiento posterior a la culminación de los Papel de la IFC en materia de servicios proyectos, captación de enseñanzas de proyec- de asesoría: tos abandonados o terminados, y más exámenes · Establecer una estrategia global para los a distancia de servicios, productos y proyectos. servicios de asesoría de la IFC, atendiendo A mediano plazo, sería conveniente introducir la necesidad de una visión y un marco de un sistema de informes ampliados de termina- negocios claros y estrechamente vincu- ción de proyectos (semejante al sistema de in- lados con la estrategia institucional mun- formes ampliados de supervisión de proyectos dial de la IFC. Al cabo de años de crecimiento para operaciones de inversiones y realizado des- incontrolado y recientes reformas institucio- pués del informe de terminación de proyecto, nales, es necesario ocuparse del papel de los para identificar mejor los impactos), evaluacio- servicios de asesoría en el modelo de actividad nes e investigaciones de impactos más progra- de la IFC. Sería preciso articular mejor, en la es- máticas, estableciendo objetivos basados en trategia, las ventajas comparativas que posee la resultados para los servicios de asesoría en su IFC en materia de servicios de asesoría, los ob- puntaje institucional, y determinar regularmente jetivos y metas de dichos servicios en diferen- parámetros de referencia de actividades y sis- tes contextos (tema que genera confusión entre temas de servicios de asesoría de la IFC con los funcionarios) y considerar las mejores com- otros proveedores de servicios de conocimiento, binaciones de personal posibles (en cuanto a incluidos otros bancos multilaterales de desa- personal interno o externo, y mundial o local), rrollo y proveedores comerciales. A más largo la organización de la unidad de ejecución, los plazo el objetivo podría consistir en establecer incentivos y medidas de desempeño que con- una unidad de investigación especializada que tribuyan a alcanzar esos objetivos y metas. se ocupe de generar y reunir trabajos de cono- · Llevar a cabo intervenciones de servicios cimiento para el desarrollo del sector privado. de asesoría más programáticas. La evalua- ción muestra que la IFC ha alcanzado mejores El presente informe fue examinado por un Grupo resultados en proyectos de servicios de asesoría de Asesoramiento de expertos internacionales ejecutados en conjunción con otras interven- especializados en la esfera del conocimiento y el xli I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 desarrollo, integrado por el Profesor Carl Dahl- rector del Instituto para la Gestión del Conoci- man, Profesor Asociado Luce de Relaciones In- miento. En una declaración conjunta incluida en ternacionales y Tecnología de la Información de esta publicación el Grupo se manifestó de acuerdo la Escuela de Servicio Exterior de la Universidad con las recomendaciones que anteceden y sugi- de Georgetown; Acha Leke, Socio de McKinsey & rió a la IFC posibles pasos adicionales en la misma Company; y Laurence Prusak, fundador y ex Di- dirección. xlii . . . ) . ( . > i.< . ) . ( . . ( ) . 2009 ) 1 ) 2007 .( .( . : xliii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 2006 iii2008 7 ( ) . 85) 72 ( .( ) . 2008 2006 63 2007-2005 .( 75) . 62 2008 1996 ( 70) 2006 . . ) ( (1 : . 51 ) 2005 2005 75 ii.2008 (2 (2008 ) 2008 (3 ( ) (4 . ( 40 2007 32.7 .2008 (5 ( 8) ) ( . ) . .( . . xliv . ) ( . . . . 2008-2003 . 40 . 2008 . . . ) ( . . 62) 40 ( iv.( ) . . . ) . ( . . . 40 : . . (1 : : xlv I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 (2 ( . ) .( . 1 1,262 . . : . . 52 v . . 50 13 18 . 70 . . 2006/2005 . . . . 60 ) ( 1,300 . . ) . ( . ) ) xlvi . . ( : ) ( . . . . . . . ) : .( . . : .() . . ) ( . . xlvii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 . : . . . ) .( · . . . . . ) ( . ) ( . . : · . · . . . . ) ( . · . xlviii ) ( . . · . . : Carl Dahlman Luce . Acha Leke Laurence McKinsey & Company ) . Prusak . ( xlix IFC Management Response to IEG-IFC Independent Evaluation of IFC's Development Results 2009: Knowledge for Private Sector Development* M anagement welcomes IEG's Independent Evaluation of IFC's Devel- opment Results 2009. The report reviews the development results of Investment Services (IS) projects evaluated from 2006 to 2008 that were approved between 2001 and 2003 and Advisory Services (AS) operations eval- uated between 2006 and 2008 that were approved between 1996 and 2008. It is the first IEG report that includes evaluations of both IS and AS operations. Introduction still unknown. Under these conditions, IFC is in- IFC is operating in an unprecedented and chal- creasingly proactive in protecting its portfolio lenging environment today. The financial crisis that clients and innovating new business models to re- started in the developed economies has now be- spond to the crisis. A third party assessment of come a global economic crisis, adversely affecting IFC's experience in development, such as this our clients to varying degrees. Private capital flows report, plays an important role in informing IFC's are down significantly, global financial institutions strategic response during this crisis. are curtailing lending and exports are falling, lead- ing to an expected overall contraction in eco- While we are pleased that the independent eval- nomic growth. The crisis is still unfolding, and the uation found that IFC achieved strong develop- extent of its impact on development results is ment results in both IS and AS, we note that the ongoing global slowdown and sharp decline in market conditions are not yet reflected in these *Discussed by the IFC Board Committee on Development Ef- results. Development outcomes of IS operations fectiveness (CODE) on March 11, 2009 and subsequently con- sidered by the IFC Board on a no objection basis. Released by are at a record high at 72 percent (85 percent by IFC in accordance with IFC's Policy on Disclosure of Information. volume), i.e., nearly three-quarters of operations li I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 met or exceeded market, financial, economic, en- tiative to capture and disseminate lessons of ex- vironmental and social performance benchmarks perience across IFC. The momentum continues, and standards, and made positive contributions and in the last year alone management has es- to private sector development beyond the proj- tablished a dedicated Vice Presidency for Advisory ect. IFC's performance in health and education, Services; strengthened policies, procedures, and while based on a relatively small sample, is note- guidelines dealing with matters from the admin- worthy, achieving high development outcome in istration of trust funds to the management of 100 percent of evaluated operations. This confirms records; and undertaken a second major review that appropriately structured private participa- of our AS product offerings. We will also shortly tion in social sectors is good for development. IFC be announcing refined organizational arrange- work quality has improved again. Results of ini- ments with clearly defined accountabilities that are tiatives IFC undertook to strengthen appraisal consistent across the business. These measures and supervision, such as increased decentraliza- augur well for even stronger development results tion and enhanced risk management, are be- going forward. As the report acknowledges, IFC's coming more evident in IFC's development efforts in these areas compare very favorably with performance. IFC is taking advantage of this mo- measures taken by other multilateral develop- mentum by further deepening its initiatives such ment banks (MDBs). as in environmental supervision and client ca- pacity building in the financial sector, especially The report also indicates that IFC's responses to in more difficult regulatory conditions. Going past crises were relevant and effective. While pri- forward, IFC expects to stay focused on both vate sector investors generally hold back in times portfolio and new business opportunities and of crisis, IFC remained committed to its devel- challenges in light of the current global crisis. opment role. IFC's investments in high-profile strategic companies and restructuring of major In AS operations, 70 percent of evaluated projects existing projects sent powerful positive signals achieved a satisfactory or better rating in Devel- at a time when market confidence was waning in opment Effectiveness (DE)--a synthesis rating crisis countries. Demand for IFC was strong, es- of five development dimensions comprising strate- pecially for its risk mitigation, knowledge, and gic relevance, output, outcome, impact, and effi- innovation. Overall, projects approved in the ciency. Significantly, most of the projects evaluated wake of a crisis achieved better results than pre- by IEG were designed and, in many cases, im- crisis projects. Existing projects that were in the plemented before Management fully implemented early stages of implementation were most vul- the raft of recent actions intended to strengthen nerable and were hit hardest by the crisis. The the impact of our AS business. Those actions nature, quality, and speed of IFC's portfolio and have included: organizing the business into five new business responses proved crucial in the business lines; establishing rigorous project review, success of IFC's operations in past crises. approval, and supervision processes; creating a rigorous monitoring and evaluation system; in- We agree with the overall direction of the report's stituting a pricing policy to strengthen client com- recommendations. Our responses to the recom- mitment to implementation and ensure any mendations are set out below. subsidies are justified by the balance of public and private benefits involved; reviewing products Response to Specific Recommendations based on performance and categorizing them by Recommendation 1: Effectively manage the level of maturity; establishing protocols to pro- tension between protecting the portfolio and re- mote effective World Bank Group coordination sponding to opportunities during crisis. when engaging with government clients; strength- ening financial management systems; develop- Response. Management agrees with providing ap- ing AS staff competencies and training; and propriate focus on both protecting our portfolio launching a major knowledge management ini- and responding to opportunities during the cri- lii I F C M A N AG E M E N T R E S P O N S E TO I E G - I F C sis. IFC has crafted separate but coordinated re- matic initiatives that are clearly separate from sponses to the current crisis. portfolio operations. These initiatives are being structured for greater development impact and are IFC's first priority is to work with its portfolio targeted at specific liquidity and other financing clients to help them weather the crisis and at the needs, as well as advisory demands arising from same time protect IFC's portfolio. Portfolio work the current crisis. Initiatives already starting to gain capacity has been enhanced where it is most traction include: needed. Nearly all portfolio managers are now based in the field because this is critical to un- · Bank Recapitalization Fund: A global equity derstanding client issues quickly and resolving fund to recapitalize banks, for up to $5 billion them expeditiously. More investment and corpo- from IFC and other investors. rate services staff have been assigned to portfolio · Trade Initiatives: Global Trade Finance Pro- work and IFC is further strengthening its human gram: doubling to $3 billion. Continued focus resources to ensure that it has adequate requisite on banks in IDA/frontier markets; other initia- skills in complex restructuring and recovery op- tives are also being developed. erations. IFC has undertaken several initiatives · Microfinance Liquidity Facility: A $500 million to closely supervise its portfolio, including: facility to instill confidence in the microfinance . industry, jointly with KfW Initial contributions · Deepening of portfolio stress testing by de- (IFC $150 million) will focus on short-term debt. veloping structured stress-testing methodolo- · Infrastructure Crisis Facility: Facility to support gies and disseminating them throughout the viable privately funded infrastructure projects Corporation; facing financial distress. IFC expects to mobi- · Assembling a new team dedicated to portfolio lize between $1.5 billion and $5 billion. oversight and compliance testing. This team will · Sovereign Fund Initiative: A fund of at least monitor portfolio management processes and $1 billion, of which IFC would provide up to activities globally in order to ensure best prac- $200 million to invest in frontier markets. tice and will be developing a portfolio scorecard · Advisory Services: Refocusing existing pro- for investment departments on all aspects of grams on financial sector and infrastructure; portfolio management; new programs in risk management, loan port- · Enhancing portfolio intelligence activities to folio workout/non-performing loan manage- develop finer methodologies for all portfolio ment; scaling up select programs. valuations as well as single and group exposure aggregation. IFC has established a new investment subsidiary, the IFC Asset Management Company, LLC, which Non-investment departments are also increasingly will initially carry out both the Sovereign Fund Ini- engaged in helping meet the needs of our port- tiative and the Bank Recapitalization Fund. Such folio clients. For example, the Special Operations a move clearly separates these specific new busi- Department has started to get involved early in the ness initiatives from IFC's portfolio operations. investment project cycle to coach Investment Of- ficers and teach lessons learned from restructur- As the IEG report acknowledges, IFC-World Bank ing. Along with the central Portfolio Management cooperation will continue to play an important Department, the Special Operations Department role in IFC's goal of achieving greater development is currently being reinforced with more senior-level impact. Significant progress has been achieved at resources and an expanded and more proactive the level of strategy, policies, systems, and proj- mandate to allow for early risk identification and ects. In developing the crisis response, for ex- heightened portfolio supervision. ample, IFC has coordinated with the Bank in developing its special initiatives, e.g. the Infra- In terms of new business opportunities during the structure Crisis Facility Fund and the Bank Re- crisis, IFC has established a number of program- capitalization Fund. Going forward, IFC will track liii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 its performance in strengthening Bank Group this approach is implemented by a single project cooperation through its Corporate Scorecard. encompassing interventions at all three levels. In other cases, the approach is implemented through Recommendation 2: Set out an overall strategy a series of projects sequenced to address prior- for IFC Advisory Services, addressing the need ity constraints or to ensure strong client com- for a clear vision and business framework that is mitment. In yet other cases, IFC interventions more closely linked with IFC's global corporate are designed to complement the activities of the strategy. World Bank or other development actors, and focus on IFC's area of comparative advantage. In Response. Management agrees on the impor- all cases, however, the goal is to maximize our de- tance of a clear strategy and business framework velopment impact. IFC proposes to continue to for our AS business. As noted above, IFC has been emphasize programmatic approaches wherever working intensively over the past few years to feasible and appropriate. strengthen the strategic and operational effec- tiveness of our AS business. This has included a Recommendation 4. Improve execution of the raft of actions at the policy, system, process, prod- AS pricing policy. uct, and organizational levels. Each of these actions reflects the exercise of strategic judgment about Response. Management agrees that our pricing the kind of advisory services IFC intends to pro- policy is an important tool to strengthen the im- vide in order to fulfill our mission. We have cho- pact of our AS interventions, but differs with IEG sen to develop an integrated strategy for both on parts of their analysis and recommendations. our investment and advisory businesses, believ- As context, IFC has been charging clients for some ing that this is the most promising path to maxi- of its advisory products for many years. Since Jan- mize our development impact. At the corporate uary 2007 this approach was broadened to em- level, this is reflected in our overall strategy. In- brace the full range of our advisory services. dividual regional, country, and industry strate- Importantly, the policy is not intended to raise gies also reflect the complementary nature of our revenue per se, but rather, it aims to strengthen investment and advisory instruments, and are de- client commitment to implementation of our ad- veloped and honed through intensive annual vice, and to ensure any subsidy is justified by the bottom-up and top-down strategy exercises that balance of private and public benefits involved. Re- include full engagement of investment and advi- flecting these aims, the policy recognizes not only sory staff. As noted above, advisory services are client payments direct to IFC, but also in-kind also an integral part of our strategic response to contributions and payments to third parties (e.g, the unfolding financial and economic crisis. consultants). Moreover, since our AS is focused on addressing market failures, including the gen- Recommendation 3. Pursue more program- eration of public goods, pricing approaches based matic AS interventions. on the value or impact of our AS will often not be relevant or practicable. Indeed, if advisory prod- Response. Management agrees that program- ucts could be priced on a full commercial basis, matic AS interventions often promise more sub- questions might arise about why IFC, rather than stantial development impact than more limited a private consulting business, should be provid- interventions. In recent years this has become ing the advice. These considerations mean that the hallmark of our approach in areas such as fees received directly by IFC (the metric chosen corporate governance and small and medium en- by IEG) provide limited insights into the extent terprise financing, where interventions at the of compliance with the policy. IFC intends to level of individual firms are complemented by keep the operation of our AS pricing policy under measures that embrace a broader pool of firms and regular review, and will continue to refine the im- the overall enabling environment. In some cases plementation of the policy based on experience. liv I F C M A N AG E M E N T R E S P O N S E TO I E G - I F C Recommendation 5. Strengthen AS perfor - forward, Management would be very supportive mance measurement and internal knowledge of the development of an Expanded Project Com- management. pletion Report (XPCR) instrument, the criteria to determine projects that will be subjected to a Response. IFC agrees on the importance of ef- process, and a relevant guidance. fective performance management and internal knowledge management, and is committed to Research, development, and innovation in support improving its performance in both areas. IFC in- of IFC's strategic priorities are an integral part of troduced its monitoring and evaluation (M&E) our advisory services business. Management has system for advisory services in 2005. The M&E recently launched a major knowledge manage- function in IFC is decentralized, with every region ment initiative for IFC as a whole. It draws on les- staffed with one or more M&E officers. In addi- sons of experience with similar initiatives in the tion to regular monitoring on core indicators de- World Bank and elsewhere, and has a strong em- veloped for each advisory product, there have phasis on capturing lessons of experience from been a number of external in-depth project and our front-line staff, supported by a cadre of tech- program reviews to capture lessons and results. nical specialists for key products and M&E staff, As noted in the report, IFC's efforts in this area as well as active knowledge-sharing networks for compare very favorably with other MDBs. Cur- each business line. In addition, the joint World rently, about 60 percent of project approval doc- Bank/IFC Vice Presidency for Finance and Pri- uments apply lessons learned from evaluations or vate Sector Development engages in a substan- "smart lessons." Management's emphasis on the tial research program with internal and external application of lessons learned is strong and we partners. Against this background, Management expect to reach a 100 percent target in the next does not believe that a specialized IFC unit fo- two years. Formal portfolio review processes, in- cusing on private sector development knowl- cluding M&E data, started in 2007, and in 2009 will edge work is necessary over and above the incorporate standard corporate guidelines. Going current initiatives. lv Chairperson's Summary: Committee on Development Effectiveness (CODE) T he Committee on Development Effectiveness (CODE) considered the report, Independent Evaluation of IFC's Development Results 2009: Knowledge for Private Sector Development, prepared by the In- dependent Evaluation Group (IEG) of the International Finance Corporation (IFC), together with the draft Management Response (MR). The Advisory Panel Statement on the IEG report was circulated for information. Overall Conclusions. The Committee com- (IS); and on IFC's high reliance on newly hired mended IEG for a comprehensive evaluation and staff and outsourcing the provision of AS. There generally agreed with the main thrust of its rec- were varied views on the pricing policy and the ommendations. It was also pleased that overall, possible impact of changing this policy on IFC's IFC achieved high development results in most business model. For the next IEG report, there was of its investments and advisory services (AS) op- a suggestion that IEG evaluate AS embedded in erations. Some members asked management to investments in the financial sector and assess present an action plan for implementing these rec- how this knowledge is disseminated through im- ommendations. Members agreed on IEG's rec- plicit informal channels. ommendation for IFC to be prepared to address the many challenges ahead given the current glo- Next Steps. The Committee recommended that bal financial crisis, including balancing between the Board consider the IEG report and MR with- the need to protect the portfolio, the need to en- out a meeting, i.e., on an absence-of-objection hance the quality of AS, and the need for IFC to basis. Management will prepare a Supplemental play a counter-cyclical role. There were different Note to the upcoming IFC Road Map paper to opinions on whether there should be a separate expand on some of the comments and questions strategy for IFC AS or whether it should be em- raised at the meeting. bedded into IFC's corporate strategy. Related to this, members raised comments and questions on The following main issues were raised at the the integration of AS and Investment Services meeting: lvii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 IEG Report. Further elaboration was sought for importance of IEG's validating the effectiveness IEG to provide specific suggestions for improving of the IFC AS business model moving forward. IS and AS performances in East Asia and Pacific, Middle East and North Africa and Sub-Saharan Members stressed the importance of developing Africa, where progress lagged. IEG replied that IFC clear coordination mechanisms and incentives needed to improve appraisal and structure qual- between IFC and International Development ity in East Asia and Pacific and Sub-Saharan Africa, Association (IDA)/International Bank for Recon- and supervision quality in Middle East and North struction and Development (IBRD), within the Africa. IFC also commented that results in East Asia World Bank Group, and with other multilateral and Pacific were affected by downswings in the in- institutions based on their respective comparative formation technology industry, while business advantages. One member agreed on the need environment issues affected results in Africa, and for IFC to set out an overall strategy for AS, as rec- difficult country conditions in 2001 and 2002 af- ommended by IEG, and concurred with the Ad- fected results in Middle East and North Africa. visory Panel's recommendation to develop "a robust and integrated plan beyond just the strat- Crisis Context. Members encouraged IFC to egy" for AS. Another member felt the AS strategy protect its portfolio and the quality of its AS dur- should be reflected in the IFC's Road Map. Man- ing the current global financial crisis, and to in- agement noted that IFC has a corporate strategy, corporate lessons learned from past crises. They including both investments and AS. also encouraged IFC to play a counter-cyclical role and to be more innovative by including re- AS Pricing Policy. Some members disagreed structuring and financial engineering of enter- with IEG's recommendations to move toward prises, as well as designing innovative instruments value-based client contributions. They were con- such as guarantees. There were also comments on cerned that it may negatively affect the demand the impact of the current crisis on investments and for IFC's AS, especially in IDA and conflict-affected AS, given the IFC's increased decentralization. countries. They also cautioned against over- emphasizing fee-based services, especially be- Advisory Services. Some speakers commented cause part of the AS is to support governments on the bundling of AS with IS; the proportion of in providing public goods and improving invest- AS as an independent product line and comple- ment climate. Some agreed that there were pos- ment to IS and the possible impact of relying on itive aspects of pricing, such as demand discipline trust funds to finance AS. They queried how to bal- or revenue that might enhance the sustainability ance and improve IFC's organizational alignment, of AS. To this end, members asked management which currently relies on a dispersed set of new to consider such changes carefully. Management staff and short-term consultants to deliver global noted that the primary goal of the pricing policy knowledge. In this regard, one member felt that was to strengthen client commitment to imple- it was possible to use in-house knowledge to mentation, and to ensure that any subsidies were carry out IFC's core business, while outsourcing justified by the balance of public and private ben- the new areas of knowledge in which IFC did not efits. Management noted that it will clarify as- have enough skills. Management indicated that pects of the pricing policy in the revised version about 20 percent of AS is linked to investment of the Management Response. services. It also noted that intense participation of both staff and consultants was part of IFC's busi- ness model. Another member emphasized the Jiayi Zou, Acting Chairperson lviii Advisory Panel Statement Carl J. Dahlman, Professor of International Affairs at Georgetown University, former staff member of the World Bank, including Staff Director of the World Development Report: Knowledge for Development 1998/1999 Acha Leke, Partner, McKinsey & Company, Johannesburg, as leader of McKinsey's Sub-Saharan Africa Initiative Laurence Prusak, Co-Director of Working Knowledge, a knowledge research program at Babson College, and founder of IBM's Institute for Knowledge Management We found the IEG's Independent Evaluation analysis and recommendations, particularly re- of IFC's Development Results 2009: Knowl- garding AS because this is the main focus of the edge for Private Sector Development to be report. an excellent and timely report. The report suc- cessfully takes on the very challenging task of eval- Comments on the Report uating not only IFC's investment operations Overall, the report is very good, and is both detailed but also, for the first time, IFC's advisory services and fact-based. It contains concrete and actionable (AS). This task was made all the more challeng- recommendations. Here we comment on the con- ing by two unrelated aspects. The first is that ceptual framework and the methodology. IFC's AS has been growing very rapidly, and that it is only recently that IFC has begun to put in Conceptual framework: The report does a clearer objectives and procedures to approve, good job of outlining the importance of knowl- manage, monitor, and evaluate this line of bus- edge for development. It also emphasizes the iness. This has limited the coverage and the role of the private sector and of private sector quality of the data that can be used to assess the knowledge in development. IFC contributes to pri- wide range of AS. The second is the severe global vate sector development through its investment financial and economic crisis that has spread and operations, as well as through its advisory ser- deepened, while the evaluation was being un- vices. IFC transfers quite a lot of technical and or- dertaken, and which has not bottomed-out yet. ganizational knowledge to individual companies The authors are to be commended for begin- (firms, as well as financial and non-financial inter- ning to incorporate some of the implications of mediaries) as part of its regular investment op- the crisis for both IFC's investment operations erations. This is recognized in the report, but is and AS. not treated explicitly. Instead, the focus on knowl- edge transfer in the report is explicitly on AS. In this note, we will make some general com- This typically consists of broader policy knowl- ments on the report, and then complement the edge directed primarily to government (more lix I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 than 52 percent of IFC's AS by value), although a result of the growing global financial and eco- it also goes to financial and non-financial inter- nomic crisis. mediaries, large commercial firms, and small and medium enterprises. While the knowledge that In its recommendations, the report notes the im- IFC transfers through its investment operations portance of managing the tension between pro- may be hard to quantify, the important develop- tecting the existing portfolio and responding to ment impact of that knowledge should not be opportunities during the crisis. We fully agree and glossed over. It could be argued that this knowl- stress that this will require careful attention. We edge transfer is as important as that transferred would emphasize that as the depth and breadth through AS. In addition, there should be more of the crisis is expanding, IFC needs to do more evaluation of the quality of that knowledge trans- to manage the risk to its current portfolio, in- fer, as well as some analysis of how the value of cluding not only the balance between its current that transfer can be increased through more ex- portfolio and new investment opportunities, but plicit attention to management of that knowl- also mitigating conflict of interest that may impede edge. Addressing this may be an issue for the collaboration with the Bank and the IMF as noted report next year. in the report. Furthermore, we would add that IFC is likely to face a lot of demand for additional fi- Methodology: We found it very appropriate that nancial restructuring of existing operations, given the report complemented the database on proj- the global liquidity constraints and the drying up ect completion reports, particularly regarding AS, of credit markets. In addition, there will be in- with extensive internal and external interviews, creasing demand for restructuring specialists. IFC including with clients and other development should begin to staff up for the expansion of this organizations. This allowed it to compensate the type of work. Moreover, on the AS side, there is still relatively new and incomplete monitoring also likely to be increased demand for the design and assessment instruments for AS, and to over- of more appropriate regulatory structures, start- come some of the biases of self reporting by staff ing with finance (banking system, non-bank fi- involved in the projects being assessed. This al- nancial sector, stock markets), but extending to lowed the authors to provide some very impor- many sectors, as well as for financial restructuring tant and critical insights and to put their findings and consolidation of business. This has strong in perspective. A few additional analyses in criti- implications for the type of expertise and staffing cal areas (e.g., pricing) would have been helpful that will be necessary. It also gets at the issue of to understanding the crux of the issue better, how much of this is to be market driven and mar- and as a result, strengthen the overall recom- ket priced, vs. a public-good contribution to de- mendation. We would recommend drilling a bit velopment of better systems. deeper into critical areas in future reports. Performance of IFC's Advisory Services Performance of IFC's Investment The report examines in detail the performance of Operations IFC's AS, and shows how its rapid growth is cre- The report clearly demonstrates the significant im- ating challenges, which need to be addressed to provement of IFC's investment operations over ensure success and sustainability of this business the years and focuses on projects that reached line. It also outlines four concrete recommenda- maturity in 2006­08. It notes that part of the tions: develop a global strategy for the group, improvement has been due to the exit of a par- pursue more programmatic AS interventions, ticularly weak cohort of projects that matured in improve execution of the pricing policy, and 2005, more favorable conditions in the develop- strengthen performance measurement and in- ing world (until late 2008), improving IFC ap- ternal knowledge management. We generally agree praisal and structuring quality, and the move to with the thrust of the recommendations. However, larger projects. It appropriately notes that the we would like to make a general observation and performance of projects will likely deteriorate as then some specific suggestions. lx A D V I S O R Y PA N E L S TAT E M E N T We were surprised that the AS staff has grown by Therefore, our general comment is that in order a factor of seven since 2000, and that it now ac- to have a better evaluation of results of IFC's AS counts for roughly 45 percent of the total staff of it is necessary to have a clearer articulation of IFC. With AS expenditures of $245 million in fis- the strategy and plan for those services. The rea- cal year (FY) 2008 (roughly twice that in FY05 and son for this is that there are very strong inter- a tenfold increase since 2001), it certainly is an area dependencies between the objective and overall that needs explicit attention and appropriate man- strategy for IFC's AS--how IFC is to organize and agement. The relative size of this effort is quite im- operationalize that model, and how results are to pressive. According to the IFC FY08 financial be measured and evaluated. Therefore, we would statement, total IFC administrative expenses were like to reinforce and highlight the critical impor- $549 million. It is not clear whether the expen- tance of the first recommendation in the IEG re- diture for AS is included in this figure, since the port--that "IFC set out an overall strategy for IFC financial statement shows a separate expense line advisory services addressing the need for a clear of $123 for AS. In any event, the expenses for AS vision and business framework and link with IFC's are anywhere from one-third to one-half the total corporate strategy." administrative expenses for IFC, which is cer- tainly a large share, whichever way it is counted. There is a strong and urgent need to develop a IFC has de facto become a hybrid finance and con- robust and integrated plan beyond just the strat- sulting organization. This is a very substantial egy. This plan should cover five key areas: shift, and one that no organization that we know of has ever done before. The closest one we can a. The vision/mission: What does the IFC re- think of that has somewhat of a similar role is ally want to accomplish with this business? Goldman Sachs. To do this, IFC needs to sort out three dif- ferent objectives of advisory services: Many of the challenges described in the report i. supporting its investment operations1 are typical of rapidly growing organizations-- ii. providing public goods for the devel- balance between different operations, internal opment of the private sector in devel- and external alignment, organization and deliv- oping countries ery of services, staffing, quality, monitoring and iii. operating as a profitable fee-based con- evaluation (M&E), and results. What makes these sulting service at market prices particularly challenging in the case of IFC is the b. The strategy very rapid and seemingly uncontrolled growth of i. Which clients to focus on (e.g., gov- AS, complex interaction between AS and invest- ernments, investment operations clients, ment operations, and the broad and somewhat others unrelated to investments)? difficult measure of results. The last, as defined ii. What service lines to offer them and in the IEG report, includes relevance, develop- how to deliver them? Where is there a ment effectiveness, and additionality. All three true gap? Where can the IFC be distinc- of these measures go beyond a clear summary tive? And thus, how to streamline the indicator such as profitability, which is the typi- current offerings? cal performance indicator in commercial enter- iii. What geographical areas to prioritize? prises. IFC's three result indicators are difficult c. The operating model to quantify because they include a large element i. What key processes to put in place? of public goods and broader social and non- ii. How to leverage learning from other market objectives. An additional complication is knowledge organizations? that half of the funding for advisory services iii What partnership opportunities to pur- comes from donor funds, and that these are sue? Who is good at what? Whom to targeted at particular development objectives partner with and for what? (Here more and or regions. All of this makes evaluation of attention needs to be given to how to results very difficult. partner more effectively with the World lxi I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Bank, as well as with other multilateral tion to experienced consultants and help address development organizations that pro- some of the staff mix and experience issues. How- vide AS, as well as with think tanks, ac- ever, more work needs to be done to understand ademic institutions, consulting firms, how to transition from the current free model to and business associations.). the current cost-recovery pricing policy, and then d. The financing model to a value-based model. Quantifying and assess- i. What should be the combination of ing the risks to the business will be a critical com- donor and IFC funds for different prod- ponent of this work stream. ucts and programs? ii. What should be the pricing model for This is far from an easy task because an effective services to different clients? unit of analysis for working with knowledge would iii. What implications does the use of donor have to be developed. Few organizations have funds have for pricing of services tar- effectively de-coupled knowledge from other geted by donors? parts of a consulting assignment, such as a con- iv. How big are each of the bottlenecks in struction project, a finance system, or a market- applying the agreed pricing policy? ing strategy. By not doing this, they can charge for v. How big a risk does the IFC face in en- the knowledge they have developed as incorpo- forcing the pricing policy? rated in their overall charges. However, to charge vi. What do the financials look like under just for knowledge itself may prove a difficult a few different scenarios? thing to gain acceptance in the marketplace. All e. The organizational model of these things would need to be thought out and i. Organization structure: How to pull to- established before any sort of value charging gether the various units into a robust could occur. organizational structure? ii. Collaboration: internal (e.g., with the Charging for these knowledge services on a value WBG) and external organizations basis would involve the IFC in entering a mar- iii. Staff mix: What's the right model to ketplace that has some very established players. ensure sustainability in the long term? While many of these players may call their offer- (The current mix relies very heavily on ings in this area by different names, they all are short-term external consultants and ap- interested in this sort of work and they would pears to be unsustainable and incom- show up in any bidding situation. This would in- patible with high-quality services and clude the major management consulting firms, the effective knowledge management) large systems integrators, and many investment iv. Skills/experience of the internal staff. banks (when they get back on their feet) and even law firms, foundations, as well as many other Developing this plan quite urgently is critical. IFC nongovernmental organizations. This market is should dedicate the required resources as soon as very large. Depending on how it is measured, possible. Bringing in an experienced and objective there have been estimations of between $5 billion external firm to drive this should be considered. and $100 billion in expenditures per annum. The most-valued organizations are able to command We would also like to highlight four critical is- fees that are significantly higher than their costs, sues: pricing, knowledge management, staff mix and in exchange deliver multiple of these fees in and skills, and M&E. terms of value to their clients. For the IFC to ef- fectively capture part of this market, it will need Pricing is an important issue to address, and to: (a) clearly define its focus, strategy, and com- should be included in the plan as suggested. We petitive advantage, (b) better understand the real agree with the need to move to a more value- bottlenecks and risks in enforcing the current based pricing policy over time. This could also help pricing model, (c) quantify the true value of its AS attract and provide more attractive compensa- to its clients, (d) agree on how much of this value lxii A D V I S O R Y PA N E L S TAT E M E N T to charge as fees to clients, and (e) put in place knowledge and services provided by IFC? What a robust process to transition over time to a value- works? What lessons have they learned? These based pricing model. should be analyzed as part of the integrated plan. We recommend that IFC set up a small advisory Knowledge management: The IEG report notes board (with perhaps three members) who have ex- the very disorganized way in which AS are pro- tensive experience in knowledge-based organiza- vided with very little interaction and sharing of tions and also have some background in finance knowledge across the different regional offices, in- or with nongovernmental organizations. They sufficient blending of global best practice with could help keep IFC AS abreast of work processes local knowledge, and lack of coordination with and technology developments in knowledge man- other knowledge providers. The plan needs to agement, as well as theoretical developments in consider in greater detail the very different this area processes that are at the heart of any knowledge- based organization. These usually are understood Staffing and skill mix: That the current ratio as specifically focusing on knowledge develop- of external consultant to internal staff is roughly ment, knowledge retention, and knowledge trans- one to one, and three to one in the field vs. head- fer. Each of these has particular work routines and quarters, raises issues of how to ensure quality, practices that are well understood and pretty how to share relevant knowledge, and how to much universal among knowledge-intensive or- keep IFC expertise up to date. The current model, ganizations. In order for IFC AS to be effective in which relies extensively on short-term external these roles, it will have to institute these processes consultants and less-experienced internal staff, in a much more established and systematized is clearly not sustainable. There is a strong need way than currently exists. to upgrade skills internally; the best way to kick start the process is to hire experienced senior Strong advisory service organizations have de- consultants from other firms, who will help put veloped very robust knowledge management in place the required best practices and properly processes. As such, there is indeed a clear need train the junior staff. There will likely be a need for the IFC to strengthen its internal knowledge to complement them with external consultants in management. As recommended in the report, the short to medium term, but the medium- to we would encourage benchmarking of not only long-term aspiration should be to rely primarily other MDBs but also, and perhaps even more (and even almost exclusively) on experienced in- importantly, of world-class commercial knowl- ternal staff, while leveraging external consultants edge organizations--both McKinsey & Company for very specific in-depth expertise/knowledge in and Goldman Sachs come to mind. critical areas. To reduce the variability in the qual- ity of external consultants, IFC should consider en- Organizations like McKinsey, which are based on tering into partnerships with a few external firms/ these processes, have knowledge-intensive cul- individuals and work primarily (and if possible, ex- tures that are overtly managed. These processes clusively) with them. are well integrated with the overall work processes of the organization. There are many analyses and Monitoring and evaluation: We fully agree with descriptions of these types of cultures but they are the recommendation of the IEG report for the generally based on things like strong internal cul- need to strengthen performance measurement. tures, incentives, social norms, management sig- IFC introduced a new M&E system in 2006 "in- nals and symbols, and explicit and overt strategic cluding standardized project approval, supervi- directions. Again, these are all significantly differ- sion, and completion reports." However, the IEG ent from what one would find in organizations report states that actual staff compliance in prop- more focused on financial routines and opera- erly filling in the report was poor. This reflected poor tions. How do these organizations manage and training, as well as constant changes in the criteria share knowledge? What is relevant for the types of to be used in the evaluation and too much reliance lxiii I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 on self-assessments. The IEG report also notes all vision and plan for the role of advisory services that unlike the case for investment operations, the as emphasized above. M&E system for advisory services does not have targets on development impact or established M&E We would like to thank IEG for giving us the op- indicators of impact on a programmatic level, re- portunity to review the report and provide our flecting the immaturity of the system. Thus, it is clear perspectives. As mentioned at the beginning of that M&E needs to be strengthened and that it is this statement, we think overall the report is very necessary to go beyond project completion re- good. We would like to commend the IEG team ports to independent field assessments. We would for a job well done. We have attempted to com- also stress that developing an effective M&E system plement the report by highlighting some of the also depends on having a clear fix on the purpose issues that it has raised, and making some sug- and objectives of the advisory services. Hence the gestions for the consideration of IFC Manage- importance of the need to develop a clearer over- ment and of the Board as IFC moves forward. lxiv 1 Strategic Context T hroughout the developing world, the private sector has been a key con- tributor to growth and poverty reduction in recent years. The current global financial and economic crisis places these hard-won gains under severe threat--due to much tighter credit conditions, weaker capital inflows, and reduced developed country import demand. It has also revealed certain market and nonmarket failures and imperfections, including the heavy price of inadequate oversight, regulation, and risk management. Development institutions can play important fi- growth, typically accompanied by falling levels nancial and nonfinancial roles in response to the of poverty. The private sector has been a key crisis. These include providing finance to viable contributor to this growth through new capital in- enterprises where it is now lacking (thus sending vestment, but also through innovation and en- positive signals to other investors as well), acting trepreneurship, which has helped create jobs as an honest broker in financial restructurings, and and open up new markets. As a general rule of offering advice that helps address institutional thumb, countries with the highest levels of pri- weaknesses, for instance, with regard to effec- vate investment and those that have made the tive regulation and good governance. biggest strides in bridging the knowledge and technology gaps (and thus enhancing produc- This report examines IFC's experiences in fi- tivity and competitiveness)--particularly through nancing development (Part I) and in providing private initiative--have grown the fastest. (Figure knowledge for development (Part II), with a view 1.1 shows the relationship between private cap- to informing IFC's strategic and operational di- ital flows and economic growth, while figure 1.2 rections, including its part in responding to the shows the connection between knowledge ac- current global crisis. cumulation and future productivity.)1 This chapter sets the scene for the evaluation. It Across a broad range of developing countries, the considers the growing participation of the pri- private sector now plays a key role in economic vate sector in development in the last decade sectors previously under the domain of the pub- and the effects of the global financial crisis on the lic sector. In many countries, low- and middle- private sector in developing countries, and it out- income alike, the private sector now participates lines key implications for IFC. significantly in the delivery of transport (air, road, and rail), telecommunications, and health and ed- Growing Participation of the Private ucation infrastructure and services--all facilitators Sector in Development of growth. In 2007, commitments to infrastruc- In the last decade, many developing countries ture projects with private participation in devel- have experienced strong rates of economic oping countries amounted to $158 billion (1.1 1 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 pared with public fixed-capital investment at 6.4 Figure 1.1. Stronger Growth Has Generally Been percent.3 Associated with Increased Private Capital Flows The importance of the private sector to develop- ing country growth has been reflected in shifts 1990­2009 in the makeup of World Bank Group financing 6.0 Net private flows as a percentage of GDP R2 = 0.3981 and knowledge services. In 2000, the Bank Group's 5.0 share of financing to the private sector in devel- oping countries (through IFC and MIGA) amounted 4.0 to $4 billion, approximately a fifth of overall Bank Group financing. By 2008, driven in particular 3.0 by a fourfold increase in IFC's investment activi- 2009 ties, the private sector share accounted for about 2.0 $13 billion, or more than a third of Bank Group fi- nancing (figure 1.3). This figure does not include 1.0 2008 indirect support to the private sector through, for example, World Bank loans to governments de- 0.0 signed to improve industrial competitiveness. ­1.2 0.0 1.2 2.4 3.6 Thus, in effect, the focus on the private sector is Global growth even greater than this breakdown indicates. The Source: Institute of International Finance (IIF). share of private sector-oriented activity, including Note: Net private flows and global growth as defined by IIF. Bank lending to sovereign entities for "financial and private sector development," comes to $15.4 billion (or, 40 percent).4 The makeup of Bank Figure 1.2. Knowledge Accumulation Is Key for Group knowledge services follows a similar pat- Future Productivity tern, with just under a half of that now geared to benefit the private sector (figure 1.4).5 Armenia 5 Belarus Angola Other major development institutions, such as the Adjusted growth of real GDP China Latvia Estonia Russian Federation Asian Development Bank (ADB), African Devel- per worker 1996­2006 Botswana India Sudan 0 Mexico South Africa Korea, Rep. United Kingdom Japan United States opment Bank (AfDB), European Development Brazil Kuwait Italy Bank (EBRD), and Inter-American Development Honduras Saudi Arabia Paraguay Tajikistan Bank (IDB), have similarly recognized the im- Yemen, Rep. Mauritania portance of the private sector in generating jobs ­5 and growth, and have increased their financing Djibouti and advisory activities devoted to the private sec- tor. In EBRD's case, the private sector share of an- ­10 nual business volume in 2007 reached 86 percent, ­3 ­2 ­1 0 1 2 while for the first time, the majority (60 percent) Adjusted Knowledge Economy Index 1995 of AfDB operations in 2007 were directed at the private sector.6 Source: World Bank Institute 2008. Note: The Knowledge Economy Index is adjusted for differences in initial real GDP per capita and growth in capital per worker. Global Financial Crisis The current global financial crisis, coming soon after a food and energy crisis, places many of the percent of their GDP), about a half of which was hard-won gains of the last decade under severe in telecommunications.2 Overall private fixed- threat.7 The crisis began in the developed world, capital investment in developing countries, as a but it has spread rapidly to the developing world. share of GDP, was 17.2 percent in 2007, com- As a result, GDP growth in developing countries 2 S T R AT E G I C C O N T E X T Figure 1.3. Growing Share of Bank Figure 1.4. Bank Group Knowledge Group Financing to the Services Are Increasingly Private Sector Aimed at the Private Sector 45 600 40 500 35 Commitment (US$ billion) Commtment (US$ billion) Private Private 30 400 25 300 20 15 200 Public Public 10 100 5 0 0 2000 2008 2000 2008 Fiscal year Fiscal year IBRD IDA IFC MIGA IFC MIGA Bank-TE Bank-DA Source: World Bank, IFC, and MIGA databases. Bank-ESW Bank-RF Bank-TA Bank-WDR Source: IFC, World Bank, and MIGA databases. Note: TE = training external; DA = donor coordination; ESW = economic & sector work; RF = research services; TA = technical assistance; is expected to fall to 4.5 percent in 2009, from 7.9 WDR = World Development Report; MIGA Advisory Services (investment percent in 2007, driven largely by tighter credit promotion work) is now part of Foreign Investment Advisory Services/IFC. conditions, weaker capital inflows to middle- . income countries, and a sharp reduction in global import demand.8 Net private capital (debt and eq- advanced economies have exhibited inadequate uity) flows are projected to fall by about half, levels of risk management and governance, which dropping from $1 trillion in 2007 to $530 billion have put their balance sheets (and, inter alia, their in 2009 (from 7.7 percent to 3 percent of devel- financing activities in developing countries) at oping country GDP). At the same time, remit- risk. On the other hand, public sector institutions tances workers send to their home countries have also failed in their regulatory and oversight (another important source of capital inflow, which duties. reached an estimated $283 billion in 2007) are also projected to decline.9 Experience suggests that Aside from the financial crisis, which has major whether crises start in the real (nonfinancial) or economic and social ramifications, other sub- the financial sector, they have negative develop- stantial development challenges remain. They in- ment and welfare effects across the board because clude the perennial demand for basic needs of the concomitant drop in nutrition, education, infrastructure, such as hospitals and schools. An- health care, and social spending. other pressing need is to tackle climate change. Unless current trends are reversed with respect The crisis, while different in origin and scope from to carbon emissions and the underlying patterns prior developing country crises, has similarly ex- of resource use, scientists concur posed weaknesses in the functioning and effec- that prospects for sustaining any de- The current global tiveness of financial markets, as well as in the gree of economic growth will be se- financial crisis places various nonmarket institutions that oversee them.10 riously undermined. Yet, the crisis many hard won gains Many banks and nonbank financial institutions in of climate change is receiving less at- under threat. 3 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 tention at the present time, largely because of gation and support for environmental and social heightened concern about the current financial sustainability more generally--especially through crisis. Nonetheless, it presents a critical develop- transfer of knowledge about best practices and sus- ment issue and is the toughest challenge to con- tained capacity-building measures. IFC's role in en- tinued growth prospects, which will require the vironmental and social stewardship will need to dedicated attention of policy makers and business increasingly go beyond the specific performance leaders alike. of individual projects to cover the aggregate im- pacts its critical presence can bring in sectors, Implications for IFC ranging from agribusiness to infrastructure. Development institutions, such as IFC, exist to help tackle imperfections in the functioning of As the world reexamines the roles of govern- market and nonmarket institutions. Accordingly, ments and markets in the wake of the financial they are expected to play key roles in responding crisis, IFC has a vital part to play in supporting pri- to the financial crisis. vate sector development with sound regulatory frameworks. It would be valuable for IFC to First, IFC can help address funding gaps that demonstrate both the weight of market distor- have appeared with increased frequency due to tions and excessive regulations on the one hand, tighter credit conditions. In doing so, it is less the and the importance of value-adding means for actual amount of financing (the private sector prudential oversight, risk management, and so- operations of development institu- cial and environmental safeguards and safety In the wake of tions usually account for only a small nets, on the other hand. the crisis, IFC has a percentage of GDP), but more the sig- vital part to play. nal that such financing can send to With a view to informing IFC's future strategic other investors, which, in turn, can en- and operational directions, including its evolving hance their confidence in investing in a certain response to the crisis, this report examines IFC's country or sector. This effect is based primarily effectiveness in two areas: i) financing develop- on the long-term orientation and the track record ment, and ii) providing knowledge for develop- of an institution like IFC as a reputable investor ment. Part I of the report tackles the first theme, in emerging markets. focusing on the development results achieved among IFC investment operations that matured be- Second, IFC can play a number of nonfinancial tween 2006 and 2008, with a look back at IFC's ex- roles. At an individual project level, the Corpora- periences during previous crises. Part II deals with tion can serve as an honest broker between com- the report's main theme, the first examination of peting interests in a financial restructuring. More the Corporation's experience with its Advisory broadly, IFC can offer advice that helps tackle in- Services (AS) interventions--knowledge services stitutional shortcomings, including policies, laws, IFC provides to either private companies or gov- and regulations covering the financial and cor- ernments in support of sustainable private sector porate sectors (in partnership with the Bank and development, and which have grown tenfold since others), as well as governance and risk manage- 2001. This report thus considers, for the first time, ment by private sector entities. Of course, action the performance of both arms of IFC's business at each of these levels applies to basic needs in- (that is, investments and AS), including situations frastructure, in addition to climate change miti- where these instruments have been combined. 4 Part I Financing Development 2 Performance of IFC Investment Operations I FC's portfolio of investment operations (loans, equity, and other financial instruments) continued to expand in 2008, providing further opportuni- ties for IFC to extend its development reach. This chapter examines the nature of this portfolio growth, and then covers three main themes: i) project development results, through a review of the performance of IFC- supported projects that reached early operating maturity between 2006 and 2008; ii) a look at the impact of past crises on the performance of IFC-supported projects; and iii) a discussion of implications for IFC's response to the current crisis. Table 2.1 summarizes the evaluative tools and main data sources that IEG used in evaluating IFC investment operations. Project development results (along with IFC fi- terms. Such projects represent about 40 percent nancial returns) improved overall, including of IFC's outstanding portfolio (62 percent by vol- among most strategic sectors, between 2006 and ume), making the downside risk to IFC's devel- 2008. However, performance in Africa, Asia, and opment return substantial. Middle East and North Africa, as well as in non- telecommunications information technology (IT) Going forward, strong IFC work quality and ad- continued to lag. ditionality will be required (e.g., in making well- timed, catalytic, new investments; providing Stronger overall results reflected several factors: corporate finance; acting as an honest broker in i) the exit of a particularly weak performing group restructurings; and helping to improve gover- of projects that matured in 2005; ii) more favor- nance and regulation). able economic conditions in much of the devel- oping world; iii) improved IFC project appraisal Portfolio Pattern and structuring quality; iv) a conscious move by IFC's portfolio of investment operations (loans, eq- IFC toward larger projects; and v) especially strong uity, and other financial instruments) continued performance in Europe and Central Asia (ECA) and to grow in the last year. The cumulative volume in Latin America and the Caribbean (LAC), where of active, committed investments increased by the majority of mature operations are located. about a quarter from $32.7 billion in fiscal year (FY) 2007 to $40 billion in FY 2008, resulting in an in- Given the current global financial crisis, IFC- crease in the outstanding disbursed balance from supported projects in early implementation are $17 billion to $22 billion (see figure 2.1). The num- expected to be the hardest hit in development ber of projects in the portfolio rose by a lesser 7 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table 2.1. Methodologies Employed by IEG to Evaluate IFC Investment Operations Evaluation activity Focus Main data sources Meta-analysis of IFC investment portfolio and Results (relevance) IFC investment operations database, new business World Bank database Meta-analysis of secondary data on foreign direct Results (relevance) World Bank database, multilateral development investment, multilateral development bank bank annual reports Validations of mature IFC investment operations Results (outcomes) 178 IEG project validations, completed between IFC additionality 2006 and 2008; 178 IEG additionality reviews for IFC investments, completed between 2006 and 2008 Risk profiling of mature and new IFC investment Risk-adjusted expected 565 IEG risk-layering reviews, completed between operations development outcomes 2000 and 2008; Institutional Investor Country Credit Risk ratings Project & country case examples Results and IFC additionality IEG project validations, country & sector studies Source: IEG. Figure 2.1. IFC's per Client Exposure Has Doubled in the Last Ten Years 25 1,600 Outstanding balance-IFC (US$ billion) 1,400 20 1,200 Number of clients 15 1,000 800 10 600 400 5 200 0 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 Outstanding balance-IFC Number of clients Source: IFC database. order (8 percent), reflecting a general preference panies. Per client exposure also increased with for larger investment operations (increasingly in- the number of clients rising by only 5 percent.1 volving corporate finance rather than project finance) and a more wholesale approach to reach- How strategically consistent are IFC's operations? ing small- and medium-sized enterprises (SMEs) They are expected to meet one or more of these through financial intermediaries and larger com- corporate strategic priorities: focus on frontier 8 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S ments. Thus increasing IFC's presence in these Figure 2.2. IFC and World Bank IDA countries will of necessity be a gradual process. Operations Increased over FY06­FY08 The strategic priorities of IFC and the Bank Group broadly address key developing country needs, al- 100 though it is also useful to compare patterns in IFC's investment operations with the private sector lend- IDA share in lending (by number) 80 ing operations of other development institutions 71% 68% and patterns of foreign direct investment (FDI). This 63% 60 helps identify the extent to which IFC appears to 46% be addressing needs that others are not tackling. Fig- 44% ure 2.3 shows that by region, IFC has had a greater 40 35% share of multilateral development bank (MDB) investments in Asia, Middle East and North Africa, 20 and Latin America and the Caribbean (where total MDB presence tended to be smaller, which means 0 the field of multilateral lenders is more crowded in FY06 FY07 FY08 Europe and Central Asia and Sub-Saharan Africa). World Bank IFC Table 2.2, meanwhile, shows that IFC has been ori- Source: IFC and World Bank databases. ented more toward countries with lower levels of Note: Calculated as number of IDA and IDA-blend operations/total . FDI/GDP This indicates that IFC's resource alloca- operations. tion has generally been to developing countries that have been relatively lacking in external finance. However, in Europe and Central Asia, East Asia and markets (International Development Association, the Pacific, and Latin America and the Caribbean or IDA, countries and frontier regions of middle- especially, which account for around two-thirds of income countries, as well as SMEs and agribusi- IFC's investments, the Corporation needs to be ness); address constraints to growth in infrastruc- particularly selective in its investments, given the rel- ture, finance, or health and education; establish atively high flows of private capital to these regions. long-term partnerships with emerging players; support South-South investment; and address cli- mate change and environmental and social sus- Figure 2.3. IFC Has Made Up a Higher Share of tainability. Because some of these objectives are Multilateral Development Bank Finance relatively new or hard to measure (e.g., climate in Asia, MENA, and LAC change and sustainability), data are not yet avail- able to assess resource allocation patterns against 70 all of them. For those objectives with trackable data, most new commitments between 2006 and 60 MENA 2008 featured at least one strategic priority.2 Over- Asia all, this suggests at least a minimal level of pursuit 50 of key strategic objectives through individual in- 40 Share, % vestment operations--given that the objectives are LAC couched in such a way that it is difficult not to 30 achieve at least one objective. Since it became a Sub-Saharan Africa strategic priority, allocation of investment re- 20 sources to IDA countries has increased (figure Europe 10 2.2).3 The pace of growth in IFC's investments in IDA countries reflects the fact that as a minority 0 financier, IFC needs the support of commercial co- 2000 2001 2002 2003 2004 2005 2006 2007 financiers to pursue each new operation, which Source: Annual reports of other multilateral development banks. can be challenging in difficult market environ- Note: MENA = Middle East and North Africa, LAC = Latin America and the Caribbean. 9 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Project Development Results Table 2.2. IFC Tended to Invest in Countries with Lower Prior Overall Results Levels of Foreign Direct IEG's evaluations show that IFC-supported proj- Investment to GDP ect development results, along with the financial returns, improved overall. In the three-year period Average Share of IFC Share of 2006­08,4 72 percent of projects (85 percent by vol- FDI/GDP investments developing ume) achieved outcomes that, on balance, met or (2005­06) (2007­08) country FDI exceeded specified business, economic, environ- 0­1% 6% 0.2% mental, and social performance criteria, thus mak- 1­2% 21% 13% ing positive contributions to private 2­3% 29% 27% Project development sector development beyond the project 3­4% 17% 32% results and financial through, for example, demonstration ef- 4­5% 5% 4% returns improved overall. fects and linkages.5 This compares with 5­6% 4% 5% 63 percent of projects (75 percent by 6­7% 5% 5% volume) achieving high outcomes in 2005­07 (fig- 7­8% 6% 9% ure 2.4). On a cumulative basis, in the period since 8­9% 0% 0% independent evaluations started in 1996, up to and including 2008, 62 percent of projects (70 > 9% 8% 6% percent by volume) have achieved high develop- Source: World Bank Group databases, as of June 30, 2008. ment outcome ratings (figure 2.5). As in the past, larger operations were more likely to meet per- formance benchmarks than the smaller ones. cent of projects maturing in 2005 realized high development outcomes, compared with 75 Stronger overall results reflected several factors: percent of projects that entered the three-year i) the exit of a particularly weak performing group cohort in 2008;6 ii) more favorable economic of projects that matured in 2005. Fifty-one per- conditions in much of the developing world until Figure 2.4. Project Development Outcomes and IFC Investment Returns Improved in the Last Three Years 100 90 Percentage of projects rated high 80 70 60 50 40 30 20 10 0 1996­98 1997­99 1998­00 1999­01 2000­02 2001­03 2002­04 2003­05 2004­06 2005­07 2006­08 Three-year moving averages Development outcome Investment outcome Source: IEG. Note: A high rating means the project met or exceeded benchmarks. 10 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S Figure 2.5. Improvement in 2006­08 Followed Historically Weak Performance in 2004­05 100 Now outside 2006­08 = 72% 90 three-year cohort with high development rating 80 75% 70% 72% Percentage of projects 70 63% 66% 62 60% 60% 63% 60% 60 55% 56% 54% 51% 50 40 30 20 10 0 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 Year-on-year averages Development outcome 1996­2008 average Source: IEG. Note: A high rating means the project met or exceeded benchmarks. late 2008 (figure 2.6), by which time most eval- was still discernible in the latest evaluations, with uated projects had been substantially imple- negative implications for development outcomes mented;7 iii) improved IFC project appraisal and going forward. This is consistent with trends ob- structuring quality (figure 2.7), suggesting steps served in the context of past crises. taken by IFC--such as the establishment of credit training for all new investment officers in 2001 and Figure 2.6. Country Business Climate Risk organizational changes implemented between Improved in Most Regions 2001 and 2003--including a major departmental reorganization in 2002, are starting to have trac- 70 Country credit risk rating (out of 100) tion; iv) a conscious move by IFC toward larger projects, which have been more likely to achieve 60 high ratings than smaller projects, due in part to 50 greater internal scrutiny; and v) especially strong performance in Europe and Central Asia and in 40 Latin America and the Caribbean, where the ma- 30 jority of mature operations are located. The up- ward trend is consistent, to the extent data are 20 comparable, with the experience of ADB and 10 EBRD.8 (Boxes 2.1 and 2.2 describe the rating di- mensions that are used in project evaluations. Box 0 2.3 provides illustrations of projects with high and 2003 2004 2005 2006 2007 2008 low development outcome ratings). Central and Eastern Europe Latin America and the Caribbean East Asia and Pacific Middle East and North Africa The year 2008 presents a complex picture. The re- South Asia Southern Europe and Central Asia sults, for the most part, reflect the performance Sub-Saharan Africa of projects that matured well before the onset of Source: Institutional Investor. the crisis. Nevertheless, a decelerating trajectory Note: A higher rating equates to lower risk. 11 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 2.7. IFC Work Quality Improved Again 100 90 Percentage of projects with high 80 70 IFC work qualtiy 60 50 40 30 20 10 0 1996­98 1997­99 1998­00 1999­01 2000­02 2001­03 2002­04 2003­05 2004­06 2005­07 2006­08 Overall work quality Appraisal Supervision Role and contribution Source: IEG. Note: High work quality means that the project had satisfactory or better work quality. Box 2.1. How Are Project Development Outcomes Rated? Project development performance ratings are assigned projects, in terms of pollution loads, conservation of in the following dimensions: biodiversity and natural resources, and in a broader context, social, cultural, and community health aspects, Project business success: Returns relative to a com- as well as labor and working conditions and workers' pany's cost of capital (real sector); associated subport- health and safety. folios or asset growth contribution to an intermediary's profitability, financial condition, and business objec- Private sector development impacts (beyond the proj- tives (financial sector). ect): Demonstration effect in creating sustainable en- terprises capable of attracting finance, increasing Economic sustainability: Economic rate of return (real competition and linkages, and bringing about improve- sector). This indicator also takes into account job cre- ments in regulation. ation, net gains or losses by nonfinanciers, nonquan- tifiable indicators, and contributions to widely held These ratings are then synthesized (not averaged) into development objectives; economic viability of the fi- a single development outcome rating, on a six-point nancial institution and its sub-projects, and contribution scale from highly successful to highly unsuccessful. to improving living standards (financial sector). (The full rating criteria for each of the indicators are set out in appendix B). Environmental and social effects: i) Consistency with IFC requirements; and ii) net impact of the project or sub- 12 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S commitment to the project, and governance/ Box 2.2. IFC Investment Outcome business reputation), and product competi- Rating tiveness risk (which captures the project's un- derlying competitiveness in the market in which IFC investment return ratings are based on the gross it is operating, and any market distortions); profit contribution quality of an IFC loan and/or equity · Factors internal to IFC--the quality of IFC's investment (without taking into account transaction work in project appraisal and structuring, proj- costs or the cost of equity capital): ect supervision, and additionality. (See box 2.4 for details). Loans: Satisfactory provided they are expected to be repaid in full, with interest and fees as scheduled In general, external risks can be mitigated with (or are prepaid or rescheduled without loss). strong work quality, although project develop- ment outcomes still tend to be lower when proj- Equity: Satisfactory if they yield an appropriate pre- ect risk exposure is higher (figures 2.8 and 2.9). mium on the return of a loan to the same company (a nominal US$ internal rate of return greater than or Region and Sector Results equal to the fixed loan interest rate, plus an instrument IFC-supported projects in the predominantly risk premium). middle-income regions of Southern Europe and Central Asia, Central and Eastern Europe, and Latin America and the Caribbean again achieved More than a decade of evaluation and econo- the best development outcome ratings, followed metric testing shows that project development re- by South Asia, where development performance sults hinge significantly on two types of factors: has significantly improved in the last three years. However, performance continued to lag in East · Factors external to IFC--notably, changes in Asia and the Pacific, and in the mainly low-income country business-climate risk, sponsor risk IDA regions of Middle East and North Africa and (the sponsor's experience, financial capacity, Sub-Saharan Africa--with barely half of the proj- Box 2.3. Illustrations of High and Low Project Development Outcomes Below are illustrations of high and low project development out- Low--General manufacturing & services: The project involved come ratings: constructing and operating an industrial estate in the Middle East. Only one year after IFC's disbursement, the foreign spon- High--Infrastructure: The project was to upgrade, expand and op- sor suspended the project after construction delays and dis- erate an international airport in a country in Latin America and the putes with the local partner. In the following year, the project Caribbean under a concession granted by the government, following company shut down its operations after having only one short- a competitive bidding process. Although the revenues were lower term tenant, and laid off all of its nearly 150 employees. The proj- than projected at the approval, the project was successful in im- ect thus failed to achieve the expected job creation, promotion proving the airport facilities and creating nearly 100 new jobs (63 of foreign investment, and technology transfer. The company percent increase). The success of the airport has had a positive was diligent in meeting all the environment and social require- effect on business through increases in tourisma and improved per- ments during the construction phase, but the project stalled prior ception of investing in the country. The project meets its environ- to completion and never resumed at the time of IFC's exit. ment, health and safety, and social compliance obligations. a. Nearly 800 hotel rooms were added each year in the country between 2004 and 2007, partially as a result of the investment in the airport. 13 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Box 2.4. Measuring IFC Work Quality As much as possible, IFC's work quality is evaluated indepen- example, was IFC able to detect emerging problems in a com- dently of the project's outcome so as to avoid bias in the ratings. pany and respond expeditiously with appropriate and effective For example, 11 percent of projects with high development rat- interventions? ings were nevertheless judged to have had low overall IFC work quality; and 33 percent of projects with low development ratings IFC role and contribution: This indicator describes the extent to were still rated high for overall IFC work quality. Occasionally, which IFC played a catalytic role in an investment, and made a however, actual project results can influence work quality rat- special contribution. This aspect of work quality is analyzed in ings. Projects performing poorly can expose or exaggerate the greater detail in chapter 2, within the context of IFC's addition- materiality of weaknesses in IFC's structuring or supervision, ality (for which this indicator is currently the closest proxy). which in the absence of significantly negative project perfor- mance might have gone undetected. Conversely, a project that Each project evaluation contains lessons, which most often per- is performing very well may be doing so despite shortfalls in IFC's tain to IFC work quality. work quality, which might, under different circumstances, have been more evident. As a corollary exercise, IEG examined early review documents (PDS-ER) for 42 IFC investment projects approved in FY08, and Project evaluations cover three aspects of IFC work quality: selected based on a stratified random sample. In its PDS-ERs, IFC prompts investment officers to compare the new project with other Screening, appraisal and structuring: The extent to which IFC fol- IFC projects and to provide lessons learned. Ideally, there should lowed good practice standards, such as those identified in IFC be an undertaking to dig into the issues at appraisal, apply ap- Credit Notes. For example, with hindsight, did IFC identify key risk propriate lessons, and mitigate risks/issues going forward. IEG factors, mitigate them as much as possible, and arrive at realis- found that in each PDS-ER reviewed, IFC suggested a number of tic expectations for project and company performance? Actual re- lessons to be considered. However, in most cases, the sources sults are compared to expectations and the main reasons for of lessons were not provided and explicit comparisons to other variance are analyzed to assess whether IFC's assumptions were projects were not made. In 18 cases (43 percent) other projects well grounded in good practice, due diligence, and structuring, and were listed, but in only 12 were explicit comparisons made. In the extent to which differences in actual results were due to ex- many cases, the lessons listed were generic, and in a very few traneous effects, such as recognized but uncontrollable risks. cases unrelated to the project being reviewed. Overall, based on IEG's review, IFC was found to be inconsistent in its identification Supervision and administration: Following approval and com- of comparator projects and review of lessons. Lessons should mitment, and through to eventual closure, this indicator assesses come from projects with similar characteristics and be referenced how well IFC carried out its supervision of an investment. For accordingly. ects in these regions meeting or exceeding spec- uct market competitiveness. They also exhibited ified benchmarks and standards, although with strong IFC work quality. By contrast, projects in Sub- some slight improvement (figure 2.10). Saharan Africa, Middle East and North Africa, and East Asia and the Pacific featured relatively weak Performance continued Differences in project risk characteris- work quality (figure 2.11). Sub-Saharan Africa faced to lag in EAP, MENA, tics, notably a project's relative exposure the highest external risks (figure 2.12). Of par- and SSA. to country and sponsor risk, account for ticular concern is that over a third of operations some of these differences. However, in East Asia and the Pacific (38 percent), and 29 per- the quality of IFC's work in appraising, structuring, cent in Sub-Saharan Africa, featured low addition- and supervising its investments has played a major ality quality. In several cases in East Asia and the role. Projects in Europe and Central Asia and in Pacific, IFC's financial additionality was weak, while Latin America and the Caribbean were in Sub-Saharan Africa, client commitment to op- Over a third of operations generally carried out in better business erational and institutional changes that IFC sought in EAP featured low environments, and were also typically to bring about was a key constraint to realizing the quality of additionality. larger, with better sponsors, and prod- anticipated additionality. 14 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S Figure 2.8. Strong IFC Work Quality Figure 2.9. Strong Additionality Is Can Help Clients Important for Effective Overcome Risk Risk Mitigation 100 100 92% 91% 90 84% 90 Percentage of projects with high 79% Percentage of projects with high 80 78% 71% 80 development outcome 70 development outcome 70 60 60 50 50 40 30 29% 40 33% 21% 20 30 14% 12% 10 20 0 10 0 1 2 3 0% 0 Number of high-risk factors (sponsor, market, country risks) Low sponsor risk High sponsor risk Low work quality High work quality Low role High role Source: IEG. Source: IEG. Note: Econometric analysis shows that each of the risks cited above Note: "Role" refers to IFC's role and contribution to the project, in can have a significant effect on project development outcomes. terms of being a catalytic agent, and making a special contribution. Figure 2.10. Better Ratings Again in Europe and Central Asia and in Latin America and the Caribbean 100 90 with high development rating 80 Percentage of projects 70 60 50 40 30 20 10 0 Southern Central and Latin America South East Asia Sub-Saharan Middle East Europe and Eastern and the Caribbean Asia and Pacific Africa and Central Asia Europe North Africa 2003­05 2004­06 2005­07 2006­08 Source: IEG. Note: Based on 2003­08 evaluations. "Europe and Central Asia" refers to Southern Europe & Central Asia and Central & Eastern Europe. 15 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 health and education (mainly in hospitals and Figure 2.11. IFC Appraisal Quality and Realized tertiary/professional schools).9 However, it was Additionality Were Much Weaker in East Asia and the Pacific and in much weaker in nontelecommunications IT (In- Sub-Saharan Africa ternet and software). Eighty-six percent of telecom- munications projects achieved high development 100 outcome ratings, compared with 20 percent of Percentage of projects with high ratings 90 Internet and software projects. Across other sec- tors, Equity Funds Department projects again 80 achieved strong development outcome ratings, 70 while oil, gas, mining, and chemical operations per- 60 formance lagged (figure 2.13). 50 40 Sector variations, to some extent, reflect differ- 30 ences in project risk exposure, but also IFC work 20 quality and additionality. Risk exposure was a 10 clear factor in weak nontelecommunications IT 0 projects, most of which were small operations Appraisal Supervision Role and Overall involving inexperienced sponsors and unclear contribution work quality product competitiveness. However, work quality East Asia Middle East and Sub-Saharan Other regions was also well below par: high in just 40 percent and Pacific North Africa Africa of cases, compared with 91 percent for telecom- Source: IEG. munications. Strong IFC work quality was in evi- Note: Based on 2006­08 evaluations. dence in the health sector, where the Corporation showed it had learned lessons from past ex- Figure 2.12. External Risks Were Highest in perience, although the portfolio had had less di- Sub-Saharan Africa versity than envisaged.10 In oil, gas, mining, and chemicals, projects did not meet benchmarks for 100 a number of reasons: a sponsor without the nec- 90 essary technical expertise; a high-risk exploration venture that did not reach operational stage; and 80 one case of poor environmental compliance. Percentage of projects with characteristics 70 60 Factoring in Risk 50 Unlike in the world of finance, systematic risk- 40 adjusted performance measures have yet to be 30 established in the development arena.11 Factor- 20 ing in project risk exposure, and IFC work quality, 10 IEG is developing an initial Risk-Adjusted Expected 0 Development Outcome (RAEDO) framework. This Low sponsor risk Low market risk Country risk approach estimates the probability of achieving improvement high development outcomes, taking into account East Asia Middle East and Sub-Saharan Other regions project risk conditions (i.e., country, sponsor, prod- and Pacific North Africa Africa uct market, and project type risks), and in the ex- Source: IEG. pectation of satisfactory or better IFC work quality. Note: Based on 2006­08 evaluations. Project performance was generally The RAEDO approach can provide a new per- Performance was strong in IFC's strategic sectors of fi- spective on project performance. Risk factors always generally strong in IFC's nance, infrastructure (physical and have an impact on performance, and they are seen strategic sectors. telecommunications), agribusiness, and to be more pronounced in Sub-Saharan Africa and 16 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S in Middle East and North Africa. The effect of risk Figure 2.13. Performance Was Strong in IFC's factors is, however, less variable by industries than Strategic Sectors by regions. The Communications & Information Technologies and Global Financial Markets sec- IFC strategic sectors tors tend to have higher risk profiles than other sec- 100 tors. For most departments, IFC-controllable factors tend to dominate external risk factors in terms of 90 Percentage of projects rated high impact on development outcomes. The impact of 80 internal factors (i.e., IFC work quality) is particu- 70 larly pronounced in the case of East Asia and the 60 Pacific and of Communications & Information Technologies. It is evident that in all regions and 50 sectors, including Sub-Saharan Africa and Middle 40 East and North Africa, even if we account for risk, 30 the potential for success is high, but it is not 20 achieved largely because of shortcomings in work quality. It is worth noting that the current moni- 10 toring and evaluation (M&E) system is designed to 0 measure the level of effectiveness of the institution CHE CIN CGF CAG CIT CFN CGM COC at the project and aggregate levels, but does not 2003­05 2004­06 2005­07 2006­08 offer a single measure of the comparative magni- tude of development impacts across projects. Source: IEG. Notes: Based on 2003­08 evaluations. Therefore, since the RAEDO approach is also based CHE = Health & Education, CIN = Infrastructure, CGF = Global Financial Markets, CAG = on projects' development success rates, it still can- Food & Agribusiness, CIT = Communications & Information Technologies, CFN = Private not capture the differences that may exist with re- Equity & Investment Funds, CGM = Global Manufacturing Service, COC = Oil, Gas, Mining, & spect to these magnitudes. This is an interesting but Chemicals. complex area for future work. (Appendix D contains further discussion of these preliminary results). percent in 2006, to 62 percent in 2008. Few FI operations At the same time, IFC's role and con- achieved high Environmental and Social Performance tribution in building client commit- environmental and Most project development indicators improved, ment, skills, and capacity has not social ratings. but environmental and social effects ratings show improved, and remains low for FIs (56 a slight decline (figure 2.14). As figure 2.15 shows, percent in 2008), compared with the real sector this was due to the relatively low number (49 (83 percent).12 This level of performance is far percent) of financial intermediary (FI) operations from optimal and has been an important factor in evaluated between 2006 and 2008 achieving high low FI environmental and social effects ratings. This environmental and social ratings. Real sector op- is because FI environment and safeguards (E&S) erations, on the other hand, achieved a much performance can be largely attributed to the ex- higher environmental and social effects rating tent of client commitment and capacity. Some ef- (71 percent). Low performance was most appar- forts to build FI capacity in partnership with local ent among FI projects in Sub-Saharan Africa, banks and training organizations have been en- mainly related to weak FI environmental and so- couraging, for example in China. However, such cial commitment and management capacity, and efforts have been much less successful in other poor reporting of the environmental and social ef- parts of the world, particularly in Africa. fects of subprojects. Weak regulatory frameworks also contributed to low results. IFC has increased the number of FI environmen- tal specialists since 2004--from one to four full- IFC's environmental and social supervision qual- time specialists (three in the field), and two part- ity of FI projects has improved from a low of 47 time consultants. They are collectively tasked 17 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 2.14. Environmental and Social Effects Performance Weakened in 2008 100 90 Percentage of projects rated high 80 70 60 50 40 30 20 10 0 1996­98 1997­99 1998­00 1999­01 2000­02 2001­03 2002­04 2003­05 2004­06 2005­07 2006­08 Three-year moving average Business success Economic sustainability Environment and social Private sector development Source: IEG. Figure 2.15. Environmental and Social Effects Performance Has Declined Sharply for Financial Intermediary Operations 100 90 Percentage of projects rated high 80 70 60 50 40 30 20 10 0 1996­98 1997­99 1998­00 1999­01 2000­02 2001­03 2002­04 2003­05 2004­06 2005­07 2006­08 Three-year moving average Financial markets Real sector Source: IEG. with improving FI supervision and client capacity- Relatedly, the internal communication links be- building. However, during this period the FI- tween IFC's E&S specialists, investment officers, committed portfolio grew sevenfold, from $1.7 bil- and the client's environmental staff could be fur- lion to $12.3 billion, and the number of projects ther strengthened to ensure timely client follow- doubled. IFC's FI E&S management capacity and up. A process has been initiated for joint quarterly approach has not kept pace with the increase. portfolio review meetings between the Environ- 18 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S Figure 2.16. Project Development Outcomes and IFC Profitability Were Strongly Correlated By number of projects By commitment volume 72% 85% 1 1 5% 2 66% 4% 2 81% HIGH HIGH High development High development High development outcome High development outcome Development rating Development rating outcome High IFC return outcome High IFC return Low IFC return Low IFC return 80% 90% 4 3 4 3 17% 11% 6% 8% Low development Low development Low development Low development outcome outcome outcome outcome LOW LOW Low IFC return High IFC return Low IFC return High IFC return LOW HIGH LOW HIGH IFC investment return IFC investment return Source: IEG. Source: IEG. Note: Based on 2006­08 evaluations. Note: Based on 2006­08 evaluations. Volume is by IFC investment size. mental and Social Department and the Global Fi- 16 percent of projects. In most of these cases nancial Markets Department for client follow-up. (11 percent), IFC still achieved an acceptable in- It is worth noting that following earlier IEG feed- vestment return, reflective of IFC's ranking claim back, IFC has recently selectively started visiting on company cash flow for loan service, as well as FI's subprojects during supervision missions, as the collateral security package (most of these op- a means to validate the FI's reported E&S per- erations were loans), which together provided fomance but also to train the FI's staff in con- some downside protection.13 (Appendix C pro- ducting appraisals and monitoring E&S effects. vides further details on the characteristics of dif- Meanwhile, IFC's on-line training program for ferent result combinations.) E&S appraisal and monitoring has remained under development for several years. The share of operations in the high/high quadrant has increased substantially in the last few years, Relationship between Project Development from 47 percent in 2003­05 to 66 percent in Outcomes and IFC Profitability 2006­08 (or from 59 percent to 81 percent, by vol- As in previous years, IEG found a strong con- ume). Conducive business environments in many nection between project development outcome developing countries up until late 2008, as well and IFC profitability. Combined high/high out- as clear improvements in IFC work quality (in comes (high development outcome and high IFC appraisal and structuring) have been key factors investment return) were achieved in 66 percent in increasing the share of operations in the of projects (81 percent by volume), while 17 per- high/high quadrant. The exit of a low-performing cent of projects (6 percent by volume) achieved year, 2005, also had a significant effect. Again, low/low outcomes (see figure 2.16). There was a larger operations, typically with stronger sponsors difference between project development out- and exhibiting better IFC work quality, were more comes and IFC investment performance in only likely to achieve high/high outcomes. The rela- 19 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 2.17. On a Cumulative Basis, High/High Outcomes Were Achieved Half the Time By number of projects By commitment volume 62% 70% 1 1 12% 2 50% 11% 2 59% HIGH HIGH High development High development High development outcome High development outcome Development rating Development rating outcome High IFC return outcome High IFC return Low IFC return Low IFC return 61% 68% 4 3 4 3 27% 11% 21% 9% Low development Low development Low development Low development outcome outcome outcome outcome LOW LOW Low IFC return High IFC return Low IFC return High IFC return LOW HIGH LOW HIGH IFC investment return IFC investment return Source: IEG. Source: IEG. Note: Based on 1996­2008 evaluations. Note: Based on 1996­2008 evaluations. Volume is by IFC investment size. tionship between project size and performance These findings reflect several factors: i) IFC op- can also be seen over the longer term, with per- erations approved before a crisis, like other pri- formance by volume of commitments being bet- vate sector activities, were not immune to the ter than performance by number of operations sharp deterioration in the investment climate (see figure 2.17). caused by the crisis.14 Clients tend to approach IFC to increase their output capacity when economic Impact and Implications of the Global conditions are buoyant and prices are high in the Financial Crisis market cycle. However, by the time the projects come on-stream, the market has often peaked and Performance during Past Crises prices are in the down-cycle. Recently committed Given the current global financial crisis, it is im- and disbursed projects thus tend to suffer most. portant to examine IFC's experiences in past crises. ii) The better results of postcrisis projects are Evaluations of projects affected by 27 crises in the consistent with the finding that the improvement last 15 years show a common characteristic: par- in the business environment (represented by ticularly low development outcomes for projects beneficial changes in country credit ratings be- in implementation at the time of the crisis, with tween approval and evaluation) was a significant less than half achieving high ratings. Operations determinant of better development outcomes.15 that were maturing, or were approved postcrisis, iii) Given that IFC's additionality, particularly fi- fared much better. (Box 2.5 and figure 2.18 illus- nancial additionality, should be stronger follow- trate these patterns in a general sense, while table ing a crisis, the finding supports the thesis that 2.3 shows the severe effects of crisis on project higher IFC additionality is associated with better performance in a single country--Argentina). development results. 20 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S Box 2.5. Projects under Implementation in the Downturn Are Most Vulnerable to the Crisis The crisis is expected to have very different effects, depending condition as the starting point, and the crisis may erode justifi- on the stage of the project in its lifecycle: cations for business expansion, while financial losses of sponsor business(es) may weaken the sponsor's ability to carry out fur- Mature projects: Already operational before the crisis hit; the cri- ther expansion. Sponsors may need to reconsider investment sis may influence future earnings but they have probably already plans and may shift their emphasis toward restructuring/reorga- benefited from the precrisis boom period. Lower ratios of finan- nization, rather than expansion (or even consider project termi- cial rate of return (FRR) to economic rate of return (ERR) are nation), with consequent effects on development outcomes. possible, due to lower cashflow projections postcrisis and lower valuation of terminal value, but the nature of discounted cash flow Projects approved in the wake of the crisis: Not approved when puts an emphasis on earlier years' cash flows (and in this case, the crisis hit; the sponsor can accordingly take into account slow- realized cash flow, vis-à-vis future cash flow, which are dis- ing growth and reduced product or service demand in its plans for counted to obtain FRR/ERR). business development and expansion. As the business cycle im- proves, the project can be well placed to take advantage of in- Projects approved just prior to the crisis (in implementation dur- creased demand and grow the business, thereby creating increased ing the downturn): Not operational when the crisis hit; the proj- revenues and new jobs, and contributing to economic growth. ect financing plan was typically based on a boom period market Evaluations also indicate that visible, timely in- IFC's catalytic role can be found with respect to terventions can have a strong signaling effect. Turkey. In addition to restructuring major com- Key interventions, such as visible restructurings panies, IFC mobilized $100 million of its own and of major industrial clients, fast recapitalizations commercial bank funds in the wake of a major fi- of major banks, and large loan syndications have nancial crisis, which was an important signal to the had strong demonstration effects and positive markets during the recovery of the financial cri- impacts on market confidence (Republic of Korea, 1997; Russia, 1998; Turkey, 2001). This effect is based primarily on IFC's long-term orientation, Figure 2.18. Best Results When IFC Investments track record as a reputable and successful in- Were Made in the Wake of a Crisis vestor in emerging markets, and ability to support key restructurings through honest-broker lead- 100 ership in steering committees of creditors and 90 bondholders, which can signal turnaround for the entire sector and economy (as in the case of 80 development outcome (%) Projects rated with high 70 68% a major bank in Argentina). 60 58% The size of the effect depends on the visibility-- 49% 50 investments in large key flagship companies of sys- 40 temic importance for a country, such as banks, 30 manufacturing, or infrastructure companies are 20 likely to send a strong signal. The timing of the in- tervention is also important--announcement at 10 the peak of market uncertainty can have pro- 0 found effects, as in Korea during the Asian crisis, Approved before Approved in Approved after crisis year crisis year crisis year where IFC investment increased dramatically after a period of low involvement. Another example of Source: IEG. 21 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table 2.3. In Argentina, Performance Fell Dramatically as the Business Environment Deteriorated Early Mature implementation Difference Mature at the time at the time (c­a) precrisis (a) of crisis (b) of crisis (c) (%) Average change, Country Credit Risk Rating +7.5 (+21%) ­9.3 (­24%) ­21 (­50%) ­71% Share of IFC projects with high development ratings 77% 44% 11% ­66% Source: IEG and Institutional Investor. Note: Country Credit Risk Rating is measured on a scale of 100, with lower ratings equating to higher country risk. sis. However, difficult or badly implemented re- standards, promote better corporate governance structuring of IFC's own problem projects has practices, enhance risk management practices in negatively affected its ability to play a signaling financial institutions, help build financial infra- role. Some of the difficult restructuring cases ab- structure, including credit rating agencies and sorbed significant IFC resources, attracted nega- credit bureaus, and enhance regulatory capacity tive publicity, and inhibited its ability to be more relating to new financial instruments and institu- effective during the crisis (e.g., Thailand, 1997). tions. These activities grew initially in response to structural weaknesses made apparent by crisis In past crises, services demanded by the private (particularly during the Asian crisis) and have be- sector included: balance sheet restructuring, come an important part of IFC's AS operations. instead of financing new productive assets; cor- Some of IFC's postcrisis interventions combined porate financing, instead of project financing; investments and AS. These experiences are dis- short-term liquidity and trade finance, instead of cussed in Part II of this report. medium- and long-term financing; and local cur- rency financing, instead of dollar financing. Given Evaluation suggests close attention is needed in IFC's historic focus on project financing, its re- four general areas when responding to a crisis: sponse to these needs was often slow and inade- quate. The case of trade finance illustrates the ·the nature and timing of investments; point. From FY98 to FY03, IFC committed 21 trade · opportunities and constraints for bigger finance facilities amounting to $542 million. Of impact; these facilities, 11 were never used, and of the 10 · IFC's own internal practices, notably arrange- that were used, the average utilization rate was just ments for organizing and conducting its work; 27 percent. Over time, motivated initially by the and need to respond to crisis, IFC built up the capac- ·good IFC-Bank collaboration. ity to provide these services. Corporate finance now dominates IFC's business. Within a short pe- Nature and timing of IFC investments. IFC's riod of time, the Global Trade Finance Program has additionality and project development outcomes, become a significant part of IFC's business. Some as discussed above, have been stronger following capabilities have been developed for local cur- a crisis. Key IFC interventions--investment in rency finance, but IFC's capacity in this area is flagship companies, visible restructurings of major still weak relative to private sector demand. IFC has industrial clients, or large syndications of com- also increased its field presence significantly. mercial bank loans, for instance--that capitalize on its reputation as an investor and honest bro- The speed of response is Crises have also expanded demand ker can have a strong signaling effect that helps crucial. for IFC's AS, for instance, to improve restore market confidence, particularly if an- corporate transparency through com- nounced at the peak of market uncertainty. Con- pliance with international accounting versely, failure to deal decisively and expeditiously 22 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S with its own problem projects can undermine needs were distinctly better when IFC had: (i) IFC's effectiveness in responding to crisis. recognized signs of deterioration in economic conditions; (ii) adapted country strategies to Opportunities and constraints for bigger changing circumstances; (iii) adjusted investment impact. Crises can present opportunities to reach approaches by becoming more selective and new clients and result in rewards for risk-taking. worked--including through advisory services-- Often, however, such opportunities are missed with companies less vulnerable to currency fluc- owing to the diversion of staff attention and effort tuations or with familiar sponsors; and (iv) taken to restructuring extant projects, thereby under- measures to alleviate exposure constraints (Brazil, mining IFC's ability to function as a counter- 2002; Turkey, 2001). Conversely, IFC's effectiveness cyclical financier. For example, in Argentina, In- during a crisis was impaired when it had not ad- donesia, and Thailand, IFC restructured invest- justed the project mix to economic deterioration ments and injected liquidity. However, difficulties (Argentina, 2001). in restructuring absorbed significant resources, and negatively affected IFC's ability to play a coun- The speed of response is also crucial. IFC made tercyclical role. Separating restructuring from significant efforts to mobilize large amounts of cap- new business teams may help in facilitating col- ital through trade facilities, liquidity facilities, and laboration among Bank and IMF teams. equity funds, but slow decision making prevented timely response to opportunities (Thailand, In- In addition, the quality of the bankruptcy regime donesia). For instance, IFC was slow to respond and its legal enforcement can have a major impact to the opportunities in the earlier crisis in Russia. on operations after the crisis. A working bank- It had fewer staff working on Russia following ruptcy regime, by encouraging cooperative out-of- the 1998 crisis than before, and did not have the court restructuring efforts among investors, has resources to work with potential Russian sponsors. helped speed recovery. Conversely, weak bank- On the other hand, in Korea, where IFC had lit- ruptcy regimes have been used by unscrupulous tle activity prior to the crisis, quick mobilization shareholders to frustrate recovery efforts and max- of resources led to an effective IFC response to imize private gains. In restructuring portfolio com- the 1997 crisis. IFC has experienced strong de- panies, IFC has on occasion tested the bankruptcy mand for local currency financing during past regimes of some crises-affected countries (Thailand, crises (East Asia, Pakistan), but its capacity to re- Indonesia). In doing so, IFC has raised awareness spond quickly, including by borrowing locally and of structural issues affecting corporate restructur- using the proceeds for on-lending to clients, has ing and has helped strengthen investors' rights. been limited. An important element of IFC's restructuring strat- While forecasting crises is inherently difficult, egy was cooperation with the Bank to focus the good quality of work helps project outcomes. government's attention on such systemic re- Prediction of the gravity of a crisis is by nature a structuring issues faced by the private sector very imprecise exercise and IFC is subject to many (Indonesia and Thailand, 1997). Unfortunately, in of the same difficulties in forecasting crises as the end, bankruptcy regimes did not improve other investors. IFC teams often discussed the pos- much, which limited general investor's interest sibility of crises (in Turkey, for example, where the and limited the effectiveness of IFC's interven- economic environment was considered a key risk tions predicated on the existence of restructur- in IFC projects), but full-fledged scenarios were ing opportunities. not typically developed. IFC's internal practices. In many cases, the ef- Given the inherent difficulties in forecasting crises, fectiveness of response depends on it being pre- good quality of work contributes to the resilience ceded by a progressive sequence of steps to of projects. For instance, there were significant dif- adapt to the outbreak and spread of crisis. Time- ferences in quality among projects in Argentina liness, size, and relevance to country and business that broadly mirrored differences in ratings of 23 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 IFC's upstream preparation activity among these downside development and investment risk. Op- projects. Conservative assessment of the avail- erations that are most likely to fail to achieve de- ability of complementary sources of finance, velopment and financial benchmarks--those in which often dries up in crises, was also important. early operating maturity--currently make up 40 Projects that were clearly and adequately docu- percent of IFC's active portfolio of operations mented--a sign of good supervision--were more (62 percent by volume).16 likely to be successfully restructured (Argentina, 2001). Realistic, cautious, and timely loan and Careful stewardship of the portfolio will clearly be equity loss provisions that more accurately re- paramount, both from a development, as well as flected the larger risks to IFC's investment port- a financial perspective. But new investment op- folio in crisis countries also helped restructuring portunities must also be seized. Factoring in the by focusing staff attention on improving the port- lessons set out above from past crises, for exam- folio quality and, to some extent, understanding ple, in effective restructuring and working in negotiation room with clients. collaboration with the Bank, will be important. Getting the balance right between portfolio pro- IFC collaboration with the Bank. Finally, when tection and new opportunity maximization will be managed well, such collaboration has enhanced a key challenge. the effectiveness of IFC's interventions by sup- porting private sector responses to policy mea- IFC's crisis response, which is part of a broader sures (Korea). Bank advice and other Bank Group response, is still evolving. It contains When managed well, interventions have on occasion been a mixture of portfolio management and short- collaboration between IFC informed by IFC's knowledge of the term capital injections: supporting the portfolio and the World Bank helps corporate and financial sectors in a of existing clients; a broadening of the trade finance results. crisis-affected country. IFC's signaling program to $18 billion, including guarantees that role can be an important complement would cover the payment risk in trade transactions to public sector interventions. At the same time, with local banks in emerging markets; a bank re- the Corporation's role as a creditor and share- capitalization fund (a global equity and subordi- holder in key financial institutions or corpora- nated debt fund managed by IFC, with a minimum tions can be a powerful tool in corporate and endowment of $5 billion, which aims to help re- industry restructuring. capitalize banks in smaller emerging markets); distressed asset management, with a first stage While IFC crisis interventions could have con- worth about $500 million; an infrastructure crisis tributed to the preservation of jobs, IEG could not facility, a joint loan financing trust, equity facility, find evidence of joint efforts by the Bank and IFC and advisory facility to which IFC is initially pro- on employment and poverty during crises. Bank- viding $300 million, aimed at stabilizing existing, IFC collaboration had been modest, in general, viable infrastructure projects facing temporary and not any better--and sometimes worse--dur- liquidity problems due to limited private partici- ing past crises. On occasion, IFC cooperation pation, and enabling some continuation of new with the Bank and the IMF was impaired by per- project development in private infrastructure; a ceived conflicts of interest on the part of IFC, es- microfinance liquidity facility of $500 million (in pecially in highly publicized commercial disputes cooperation with Germany's KfW and the Nether- involving IFC's clients. Large-scale, wholesale in- lands Development Finance Company); and an ob- terventions through funds or facilities gave IFC a jective to continue efforts aimed at climate change seat at the table and facilitated IFC-Bank dialogue mitigation. For the first time, IFC's response in- (trade finance facilities in Korea, Argentina). tegrates investment operations and advisory serv- ices, for example, in using advisory services to Implications for the Current Global build company restructuring skills. Challenges in Financial Crisis implementing this response include: lower than In the first instance, given rapid commitment anticipated funds mobilization from third parties; growth in recent years, IFC is exposed to a large complex structures (bank recapitalization fund 24 P E R F O R M A N C E O F I F C I N V E S T M E N T O P E R AT I O N S as a wholly owned subsidiary); and adaptation to Figure 2.19. Global and Regional Investments specific country circumstances and needs. Tend to Perform Less Well Than Single-Country Investments Looking beyond the immediate term, if the cri- sis is longer and deeper than expected, IFC may 100 need to take certain contingency measures to tackle risks to sustainable economic, social, and 90 Percentage of projects rated high environmental development. Such measures 80 might include pro-poor interventions and new 70 global or regional development platforms. In the 62% past, global and regional investments have tended 60 50% to achieve weaker development outcomes than 50 single-country investments (figure 2.19). This im- 40 plies that any such efforts may need to be re- shaped, and emphasizes the importance of 30 factoring in lessons from experience. (Table 2.4 20 provides a summary of lessons from such past 10 global and regional investments). 0 Global or regional Country-specific Finally, IFC, at present, does not systematically as- investment investment sess risks to development, as it does to financial risks. This might include the risks to achievement Source: IEG. of SME development, climate change, and rural poverty reduction goals. While there is a close as- sociation between financial and development "re- project development expectations, but also to the turns," it is not sufficient to assume that the latter sector, country, and region levels. Tracking devel- will be ensured only through attention to the for- opment risks more systematically, and undertaking mer. This applies not just to the project level, some sensitivity testing through scenario devel- which IFC's Development Outcome Tracking Sys- opment, may help guide future resource allocation tem partly addresses in monitoring changes in so that it enhances IFC's development impact. Table 2.4. Selected Lessons from Regional and Global Investments Lesson Scope The original project scope (30 businesses in over 100 countries) was too ambitious. Both the sponsor and IFC underestimated the time, difficulty, and cost of setting up enterprises in multiple countries simultaneously. The concept of setting up regional hubs also proved to be an expensive and time consuming proposition. IFC should invest in projects that have an achievable scope, and test the concept before expanding. Country tailoring Multicountry lending facilities can be difficult to implement when the project requires security or other documentation to be adjusted to the specifics of each locality in the facility for disbursements to be possible. A global security framework, if possible, could ease the documentation and implementation burden. Source: Project evaluations. 25 Part II Knowledge for Development 3 Performance of IFC Advisory Services T his is the first global assessment of IFC AS, thus we have adopted a broad, holistic approach to the subject. The chapter begins with a dis- cussion of the connections between knowledge and private sector de- velopment. It then traces the growth of IFC AS, and its strategic implications (for IFC and the Bank Group more generally), followed by an examination of three themes: i) the organizational alignment of AS; ii) the delivery of AS; and iii) the results and additionality of IFC in these operations. In line with good evaluation practice, we triangulate evidence using multiple sources where possible (table 3.1). IFC Advisory Services have been growing rap- Additionality is fundamental for better perfor- idly--tenfold in the last seven years--and AS teams mance, and may be enhanced by some--though dominate IFC's presence in the field. This raises not all--combinations with IFC investments (e.g., key strategic questions, including resource balance better ratings when combined with loans, and and possible quality trade-offs. IFC has taken for SME linkage projects in agribusiness and man- steps to improve the organizational alignment of ufacturing). More benchmarking, against both its AS, but more needs to be done to improve in- other MDBs and commercial knowledge provi- ternal focus and accountability, and better com- ders, may be helpful. plement the efforts of others. Knowledge, Development, and the Available results data suggest better performance Private Sector in Southern Europe and Central Asia, weaker per- The accumulation and effective deployment of formance in Latin America and the Caribbean (prior financial and physical resources are indispensable to a recent reorganization) and for global proj- conditions for development, but they are not suf- ects; and strong associations between country con- ficient. Advances in knowledge and technology are ditions, client commitment, the degree to which fundamental components of almost any coun- AS is programmatic, local presence, IFC addition- try's growth story--from the Industrial Revolution ality, and results. IFC's delivery approach appears in the nineteenth century, to today's developed to compare well with that of other development economies, to the economic success stories of the institutions, but is far from optimal. likes of Korea, India, and the Baltic States in the 29 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table 3.1. Methodologies Employed by IEG to Evaluate IFC Advisory Services Evaluation Activity Focus Main Data Sources Literature review on knowledge and private sector Context/concept Various development Meta Analysis of IFC AS portfolio, staffing, and new Evolution and relevance/ IFC AS database business additionality World Bank development database Meta Analysis of IFC and Bank Group strategies Strategic alignment Annual corporate and business line strategies Facility strategies 33 Country Assistance Strategies, completed between 2007­08 Meta Analysis of AS project approval documents Strategic alignment/ 248 AS approval documents, for projects approved additionality between 2006­08a Structured Interviews with IFC clients, donors, other Delivery, results, and About 150 interviews multilateral development banks, etc., in seven regions additionality Interviews with IFC AS managers & staff, in the Alignment and delivery About 150 interviews regions and headquarters Survey of IFC and Bank managers & staff Alignment and delivery 1,025 survey responsesb Meta Analysis of external reviews of AS (including Delivery, results, and 51 external program, product, and project reviews impact evaluations) additionality Validations/Quality Reviews of completed AS Results and additionality 458 IEG Project Completion Report (PCR) Reviewsc operations Project & Country Case Examples Results and additionality IEG PCR evaluations & country studies High-Level Comparison of IFC AS activities, processes, Delivery, results, and Interviews with eight development institutions and results of other providers of knowledge services additionality Document and data review (including annual corporate, and independent evaluation, reports) Source: IEG. a. Selected by stratified random sample, from a population of 692 projects. b. Out of a population of 1,920 managers and staff, covering IFC investment operations, IFC Advisory Services, as well as World Bank country directors, managers, and private sector development specialists. c. Out of a population of 707 project completion reports, a coverage rate of 65 percent. See appendix G for further details on sample representativeness. Advances in knowledge last 20 to 30 years.1 This is due prima- notably through investments in education and and technology are rily to the beneficial effects of knowl- major research and development programs, but fundamental components edge and technology progress on also in protecting intellectual property rights and of growth. productivity.2 Conversely, those coun- providing communication arteries through which tries that have failed to make advances knowledge can travel. However, it is the private in these areas, particularly in Africa, sector that translates this knowledge into pro- have typically fallen behind. ductivity, profits, and job creation (thereby con- tributing to poverty reduction) through innovation The public sector is the main provider of the and investment.3 At the same time, for sustainable knowledge infrastructure in many countries-- long-run results, as the current global financial 30 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S crisis has highlighted, appropriate standards, reg- maintaining a strong country knowledge base, as ulation, and governance surrounding private en- well as recipient developing country government terprise are also required. buy-in.6 In this context, development institutions, such Growth of IFC Advisory Services and as the Bank Group, have key roles to play--notably, Strategic Considerations in promoting improvements in standards, regu- With donor support, IFC's role as a knowledge lation, and governance of private sector enter- provider emerged on a relatively small scale in the prise, and in facilitating knowledge advancement 1980s. At that time, IFC's advisory activities had two that contributes to sustainable private sector de- main objectives: i) to improve the enabling en- velopment in the developing world.4 Just as im- vironment for private investment; and ii) to build portant as tangible changes in regulation and the capacity of small- and medium-size enterprises governance, are the less tangible shifts in atti- (SMEs). The main delivery vehicles for these tudes and behaviors that can help underpin ef- services were, respectively, the Foreign Invest- fective business practices. ment Advisory Services (FIAS), the regional SME development facilities, the Africa Project Devel- In facilitating beneficial change through knowl- opment Facility (APDF), the Africa Management edge transfer, experience suggests several fac- Services Company (AMSCO), and the Caribbean tors that could affect the chances of success: i) the Project Development Facility. (See appendix E for absorptive capacity of the recipient and the ca- more details on the early development of IFC AS). pacity gap between provider and recipient--the bigger the capacity gap, the more difficult the IFC AS have grown rapidly since 2001. AS expen- transfer; ii) the level of overall development of the ditures increased tenfold, from $24 million in host country--the bigger the development gap be- 2001, to $245 million in 2008. Meanwhile, staffing tween the source and the recipient country, typ- has risen sevenfold over the same period, from 168 ically the more difficult the transfer; iii) the level to 1,262 (or 36 percent of all IFC staff). As of June of commitment of both supplier and recipient-- 2008, IFC was managing a portfolio of 839 AS the greater the provider's stake in the process, in- projects, with a total approved value of $908 mil- volvement over time, and the level of supporting lion (figure 3.1). These data do not include certain assistance, the greater the value (but also the advice that is embedded in IFC investment op- cost) to the recipient (there is no substitute for erations, for instance ad-hoc assistance with fi- the active role of the recipient in absorbing the nancial structuring, company strategy, knowledge and the information); iv) comple- and new business development. So in IFC's Advisory Services mentarity with other relationships between the effect, the extent of IFC's efforts to have grown rapidly. provider and the recipient (if the exchange of provide knowledge to clients is even knowledge and know-how is supported by ex- greater than the AS numbers alone suggest. Based change of other services, the effectiveness of the on published data, it is estimated that IFC's share transfer is likely to be higher); v) complexity of the of MDB AS to the private sector is about a quar- knowledge being transferred--the more codi- ter.7 This share appears relatively stable, as other fied and explicit the knowledge is, the easier (and MDBs have also increased their AS operations-- less costly) its transfer.5 The recent IEG evaluation generally reflective of a growing need for this kind of the effectiveness of Bank economic and sector of knowledge as the private sector has taken on work and technical assistance confirmed some of a greater role in development, and greater avail- these factors empirically, notably the absorptive ability of donor funding for private sector devel- capacity of recipient governments (economic & opment (PSD) related assistance to developing sector work and technical assistance were less countries. The fact that most IFC AS and that of effective where government capacity was lower); other MDBs are provided free of charge (at best and commitment of the provider (Bank), in terms subsidized) has also fuelled this growth, since a of resource allocation to IDA countries and in free good always has excess demand. 31 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 3.1. Rapid Growth in Advisory Services Operations and Staff 1,262 250 1,300 1,126 225 1,100 200 Expenditure (US$ million) 863 175 900 Number of staff 150 686 700 576 125 497 100 500 342 75 300 168 50 100 25 0 0 2001 2002 2003 2004 2005 2006 2007 2008 Expenditure (US$ million) Advisory Services staff Fiscal year Source: IFC Human Resources; donor-funded operations, quarterly AS financial reports. Note: Includes FIAS activities, which are partly funded by the World Bank and MIGA (IFC is the main funds provider and manager). In FY08, IFC provided $11.8 million, MIGA provided $4 million (for investment policy and promotion activities), and the World Bank provided $2 million. Recent IFC corporate strategies have indicated Figure 3.2. Seventy-Seven Percent three main objectives for IFC AS: first, to improve of Advisory Services Staff the overall enabling environment for private in- Are Based in the Field vestment, particularly where investment oppor- tunities are limited; second, to integrate AS with 100 Investment Services (IS), as a means to improve 90 IFC additionality and development impact; and 80 77% Percentage of staff third, to pursue objectives common with those for 70 66% IFC investments, such as focusing on frontier 60 markets (including IDA countries and frontier 50 regions of non-IDA countries, as well as SMEs 40 34% and agribusiness), the strategic sectors of finance, 30 23% infrastructure, health and education, and sup- 20 port for environmental and social sustainability (in- cluding climate change mitigation in middle- 10 income countries and fast-growing IDA-blend 0 Advisory services Investment services countries, such as India).8 Field Headquarters Just over three-quarters of IFC's 1,262 AS staff Source: IFC Human Resources. are based in field offices, typically in one of 18 re- Note: Includes all AS and IS staff (analyst and above), as of end 2008. gional facilities. This compares with a roughly 1:2 split of IS staff between field offices and head- quarters (figure 3.2). Accordingly, there are more · Central and Eastern Europe--Private Enter- AS staff than IS staff in the field in developing prise Partnership (PEP) countries. By region, the 18 facilities are distrib- · East Asia and the Pacific--PEP-China; Mekong uted as follows: Project Development Facility; PEP-Pacific; PEP- 32 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Philippines; Program for Eastern Indonesia edge and share it globally across the IFC uses as many short- SME Assistance AS program. Some good practices term consultants as staff · Latin America and the Caribbean--LAC Program are emerging, such as SmartLessons, to deliver its Advisory · Middle East and North Africa--PEP-MENA; Iraq BEENet, and `Deep Dive' training ses- Services. Small Business Facility sions. Special efforts to retain and · South Asia--South Asia Enterprise Develop- spread knowledge may include: field-based train- ment Facility (SEDF); SME Development Pro- ing, practice groups, exchange and codification of gram; SEDF--Sri Lanka and Maldives (SLDF); tacit knowledge, creation and maintenance of Bangladesh Investment Climate Facility relevant databases, and possibly a dedicated global · Southern Europe and Central Asia--PEP-SE; research department/center of excellence to com- PEP-SEI--Balkan Infrastructure Facility plete the knowledge value chain. Recognized · Sub-Saharan Africa--PEP-Africa; Mozambique leaders in this sense include the McKinsey Global SME Initiative; SME Solutions Centers Institute and the Harvard Business Review.10 Mech- anisms of this kind, some of which IFC is pursu- The remainder of AS staff work at the headquar- ing, such as M&E network and conferences, are ters, in Washington DC, either in the Advisory fundamental if IFC is to consider global knowledge Services Vice Presidential Unit (established in as one of its comparative advantages. early 2008)--in portfolio management, results measurement, training, or partnerships manage- Since early 2006, AS operations have been arranged ment--or work for one of 13 global business into five business lines:11 units, such as the FIAS and the Global Environ- ment Facility, some of which have staff in the · Access to Finance (A2F)--Assistance that seeks field. to expand the availability of financial services to micro and small businesses and low-income IFC uses a considerable number of external, short- households. term consultants to deliver its AS; there are as · Business Enabling Environment (BEE)--Ac- many consultants as staff. In FY08, the cost for em- tivities geared toward improving the business ploying those consultants (some 1,332) was $72.3 environment to allow private sector projects to million, only slightly less than IFC staff costs (1,262 be viable. staff, at $82.7 million). This is a pattern consistent · Corporate Advice (CA)--Activities aimed at im- with previous years, and reflects a much greater proving the business capability of companies. tendency to use short-term consultants for AS · Environmental and Social Sustainability (ESS)-- than for IS (where staff outnumber short-term Advice and market transformation activities consultants by around 2:1).9 Consultants can of that enable the private sector to deliver envi- course bring skills and knowledge that the in- ronmental and social benefits in developing house staff do not have but, putting aside judg- countries. ment of the ratio between staff and short-term · Infrastructure (INF)--Advice on improving ac- consultants, such substantial use of consultants on cess to basic services such as roads, telecom- short-term contracts does raise service continu- munications, water and energy utilities, and ity and quality challenges--both in meeting client health and education. needs, and with regard to IFC additionality and knowledge retention (where the same consultants BEE- and SME-directed activities--provided mainly are not reemployed by IFC). through the A2F and CA business lines--remain key elements of IFC's advisory offerings, collec- Knowledge management is a significant challenge tively accounting for about 70 percent of opera- with such a wide dispersion of staff across the tions (figure 3.3). By region, Sub-Saharan Africa world, and especially given that 60 percent of AS remains the main locus of IFC advisory activity, fol- staff have been with IFC less than three years. Man- lowed by East Asia and the Pacific (figure 3.4). A2F agement is increasing efforts to capture knowl- is the lead business line in four regions (Sub- 33 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 3.3. Access to Finance Is the Figure 3.4. Highest Share of Largest Business Line Operations Is in Sub-Saharan Africa Infrastructure 12% Central and Eastern Europe Finance 5% Africa 27% South Asia 22% 10% ESS Southern 17% Europe and Central Asia 10% East Asia and Pacific Corporate Middle East 16% BEE advice and North Africa 20% 23% 11% Latin America World Source: IFC database. and the Caribbean 16% Note: By number of operations, as of June 2008. 11% Source: IFC database. Note: By number of operations, as of June 2008. Saharan Africa, Middle East and North Africa, Cen- tral and Eastern Europe, and East Asia and the capacity-building efforts, for instance, related to en- Pacific), CA in two regions (Southern Europe and vironmental and social systems improvements. Central Asia and South Asia), while ESS is the IFC has standard output achievement indicators, most active business line in Latin America and the but does not currently classify output types sys- Caribbean. The top five countries, by outstanding tematically, an effort that could enhance under- portfolio value, are: China; Russian Federation; In- standing about relative strengths and weaknesses donesia; Ukraine; and Bangladesh. of different outputs (e.g., surveys vs. diagnostic re- ports), and ultimately improve resource allocation. What does a typical AS project look like? Projects completed since 2005 have taken an average of 18 IFC works with five main client groups: govern- months to complete, although INF operations ments, financial and nonfinancial intermediaries, have tended to be shorter (14 months) and ESS SMEs, and large enterprises. Of these, govern- and multiregion operations significantly longer ments are the single largest client group, involved (25 and 27 months). Average project size has been in nearly half of AS operations (table 3.2). Strong about $350,000, although INF, multiregion, and ESS strategic coordination and operational collab- operations have tended to be larger (average be- oration with the Bank and other donors are tween $400,000 and $600,000), and BEE operations therefore important, particularly where recipient smaller (average of $220,000). Project outputs in- government capacity is weak, and for BEE and INF clude: diagnostic reports, feasibility studies, sur- work, where government clients predominate veys, transaction designs, draft legal and financial (table 3.3). This issue is discussed in the follow- frameworks, advice on institutional de- ing sections. Governments are IFC's velopment and capacity building, best single largest client group. practice guidance, training, and one-off The rapid, largely unchecked growth of AS events--such as conferences, work- raises a number of key strategic questions for shops, and seminars. Project duration is generally IFC. First, in changing the nature and face of related to the complexity of the output with, for the Corporation, has IFC struck the appropriate example, more codified diagnostic reports, such balance between its traditional core business-- as those related to Bank Group Doing Business in- investments--and its new business of Advisory dicators. They generally take less time to com- Services? Knowledge delivery is inherently more plete than broader institutional development and labor intensive than is providing financial services, 34 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S which makes direct comparisons between the Table 3.2. IFC's Main Advisory Services Client two businesses difficult. However, a clear under- Is Government standing of how the two businesses relate to one another in enhancing development effectiveness Share of Share of across different contexts is paramount. The Client AS operations AS expenditures broader issue of Bank Group resource use for Government 43% 52% maximum impact, particularly at the country level, Financial intermediaries 35% 49% also needs to be addressed. SMEs 33% 38% Nonfinancial intermediaries 33% 31% Second, while IFC has lately sought to bring Large enterprises 21% 26% some structure to the growth of AS, for exam- ple, through the creation of business lines, these Source: IFC Advisory Services Portfolio Management. Note: Portfolio as of June 2008. A single project may have multiple clients; about one-fifth changes will take time to embed. This would of government-directed AS is accounted for by FIAS. seem to imply a focus on consolidation rather than further growth. Evaluation shows that during pe- riods of major organizational change in IFC In- Table 3.3. Government Clients Predominate for vestment Services, work quality has suffered.12 Business Enabling Environment and Tensions between growth, change, and quality Infrastructure Work are common among organizations, and will need to be managed carefully. Of related import is Main client share the need to establish effective quality baselines of business line Business line Main client expenditures through sound M&E. A2F Financial intermediaries 83% Third, the increased availability of free (or BEE Government 89% subsidized) AS in support of private sector de- CA Large companies 37% velopment--from IFC and other development ESS Other intermediary/SMEs 43%/41% institutions--makes it impossible to assess true INF Government 74% client demand, and can be market distorting. Free Source: IFC Advisory Services Portfolio Management. or subsidized AS is likely to have excess demand, Note: Portfolio data, as of June 2008. Population of 839 operations. and does not screen out clients that do not really need them, and/or are not committed to effective global oversight and direction/control Rapid, unchecked growth implementation of the AS, as would be the case of AS. The business lines, meanwhile, of Advisory Services has with market pricing. It also does not send a sig- through global business line leaders raised a number of key nal as to whether a service is valued relative to an- (and regional business line heads), are strategic questions. other service (i.e., whether it is additional). Such tasked with leading business line and submarket pricing also has consequences for ex- product strategy development, pro- isting commercial providers of AS, or possible viding technical direction and quality control over new entrants to the market. products and projects, overseeing knowledge management, and managing central funding al- Organizational Alignment of Advisory location activities. Finally, the regional facilities Services and global business units are expected to develop delivery strategies and manage regional funding Internal Alignment allocation activities, and execute AS projects on the The structure of IFC AS, from direction to deliv- ground, in line with business line priorities and in ery, is a matrix that has three essential components: alignment with regional and country needs. i) the Advisory Services Vice Presidency, estab- lished in 2008; ii) business lines; and iii) regional The strategy process varies, depending on whether facilities and global business units, such as FIAS.13 it applies to a business line, global business unit, The vice presidency is charged with providing or to an AS facility, which presents some alignment 35 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 challenges. The strategy for each business line Infrastructure Advisory Department, whereby the and global business unit is revisited and updated portfolio in the region is managed by a regional/ annually by IFC as part of the corporate strategy joint appointment. However, overlapping and review. Strategies for each of the facilities are usu- parallel structures still persist, notably in Sub- ally approved at the time of donor and IFC fund- Saharan Africa, and South Asia. (Figure 3.5 il- ing renewal, which is typically every five years. lustrates the structures that exist for projects de- Since facilities were created at different times, livered in Sub-Saharan Africa). they tend not to have coterminous strategies. Since there are 18 such facilities, the potential is IFC has been seeking greater alignment at the high for inconsistent, or superseded approaches product level--product offerings within each (or, alternatively, strategic adaptations that do not business line. Since late 2008, IFC is seeking to dis- align with original commitments to donors). At tinguish its products as follows:15 present, there is no overarching strategy for AS, beyond the key principles outlined above, which (i) Entry­-A new product/approach, with as yet could help weave these various approaches to- limited or no results information gether. The survey of, and interviews with, IFC staff (ii) In development ­-Product with growing de- reveal some frustration with low interaction among mand/potential for scaling up and replication facilities (and across business lines), as well as across markets, and some supportive results change fatigue. A global AS strategy may help (iii) Developed--Scaled up and replicated across tackle some of this unease by bringing greater at least three regions, with supportive clarity to the overall direction AS is results Staff reveal frustration heading in, and identifying and foster- (iv) Exit­-Product with low demand/other supply, with low interaction ing greater synergies among facilities and with weak results among facilities (and and across business lines. (v) Other­-Idiosyncratic products, suitable for a across business lines). particular country/market segment, and not In principle, for each AS operation, expected to reach scale or be replicated there is a dual reporting structure--to AS business broadly. line leaders and to regional directors, the latter of whom are responsible for both AS and invest- Of 55 AS product types that were proposed by ments in a region. In practice, however, organi- business line leaders in December 2008, only 12 zational reporting lines and accountabilities can products were categorized as "developed" (31 be complicated. This is largely because of the percent of the project portfolio). This reflects the donor-influenced, "ground up" nature of the evo- somewhat heterogeneous, experimental growth lution of IFC AS, which has left a legacy of nu- of AS products in the past, and the "catch up" ef- merous facilities in many regions--as referred fort to bring some structure to these offerings.16 to above (also see appendix F). Staff can find By business line, A2F and INF had the greatest themselves seeking internal approval to proceed share of operations in the "developed" product with a project from many sources: the general category, with CA and ESS, which has emerged manager/manager of a facility, a regional director, more recently, the least (no "developed" prod- a business line head (potentially both in the re- ucts in each case). The lack of developed CA gion and in headquarters), and a global business products is surprising, given the fact that IFC unit head.14 Feedback from IFC managers and has been involved with SME development since staff is that these overlapping organizational struc- the 1980s. This may be due in part to the fact that tures can be substantially improved upon. IFC some classifications are not reflecting what is management has recognized these alignment happening in practice. For example, SME Toolkit challenges and has begun taking steps to con- and Business Edge have already been scaled up solidate coordination in the field. In the Latin and replicated (and even outsourced), which America and the Caribbean region, there has been would imply a more mature classification than a move toward joint ventures with FIAS and the their current "in development." The evaluated re- 36 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Figure 3.5. Delivery Structure for IFC Advisory Services Projects in Africa Field HQ Business lines Africa regional office A2F Two regional directors Joint project sign off BEE VAF (CA) ESS INF CIA dept. CIC dept. SME initiatives PEP Africa facility SME solution centres A2F 0% A2F 0% A2F 86% BEE 0% BEE 65% (Kenya & Madagascar) BEE 35% SME initiative ESS 0% ESS 0% ESS 0% INF 76% INF 0% (Mozambique) INF 15% VAF 0% VAF 0% VAF 65% CES dept. Other units A2F 10% CGF (A2F & ESS 4% BEE 3% VAF 2%) ESS 84% COC (ESS 12%) INF 0% CCG (VAF 9%) VAF 2% CGB (VAF 15%) CSM (VAF 4%) Notation: Delivery units/departments Coordination level Coordination lines Source: IEG. Notes: The percentages are based on the number of projects in each business line delivered in the region, given the portfolio as of June 2008. VAF(CA) = Value Addition to Firms (Corporate Advice), CIA = Infrastructure Advisory, CIC = Investment Climate, CES = Environmental & Social, CGF = Global Financial Markets, COC = Oil, Gas, Mining, & Chemicals, CCG = Global Corporate Governance, CGB = Grassroots Business Initiative, CSM = Small & Medium Enterprise. sults that come later also suggest other possible skills, rather than the achieved performance of a reclassifications. particular product. One problem that IFC has faced in determining IFC expects to have an 80:20 split between core whether to expand or contract product offerings (in development and developed) products and has historically been the lack of robust M&E data noncore (entry and other) products.19 Is this the to inform what works well, what does not, and what right balance between product expansion and should be changed in order to make products innovation/adaptation? Where should innovation work more effectively.17 A new project-level M&E originate--at headquarters, or in the field? In any system was introduced in 2006, together with 150 organization, there is always tension between standardized output, outcome, and impact indi- product standardization--for market consolidation cators.18 However, reliable self-reported results and efficiency purposes--and product differenti- data have so far been inadequate. In the absence ation--for the exploration and exploitation of of good results data, product classifications may-- new market opportunities. This tension needs to to some extent--reflect the quality of a product or be managed carefully. A review of the business lit- business line leader's negotiation and persuasion erature suggests that the 80:20 ratio IFC is choos- 37 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 ing to pursue, between core and noncore prod- strategy might help to bring some much-needed ucts, is broadly in line with the practice of other direction, as well as improved in-the-field incen- organizations. It could be argued that given the tives. The issue of in-the-field collaboration be- Corporation's mission to be a catalytic agent, more tween AS and IS operations is picked up again in rather than less innovation is required. However, the section of this report on delivery of AS. so long as product development is based on: i) client demand, ii) results achievement; and iii) Alignment with Other Knowledge Providers IFC capability, which in principle the approach is Aside from ensuring internal strategic and orga- trying to achieve, then the classification system nizational coherence, it is important for IFC to would seem appropriate, that is, if products are align effectively with other development actors well defined from the outset. For example, it is not providing knowledge services. This will ensure that clear to stakeholders how some products mate- IFC does not duplicate, but rather complements rially differ, for example, Subnational Advisory their approaches and thereby contributes to (exit), as opposed to Advisory Mandates (devel- greater development impact. The philosophy oped) in the INF business line. Data on new and underlies the 2005 Paris Declaration on Aid Ef- other product origination are limited, but sug- fectiveness, which, inter alia, called for greater gest a relatively even balance between complementarity among donors through a more IFC has yet to elucidate a headquarters and local offices. It is, effective division of labor.20 model for integrating however, not clear what balance IFC is advice and finance. aiming for in this regard. One important lens through which to examine alignment with others is at the country level, specif- Another major alignment question is how, and to ically in Bank Group joint Country Assistance what degree, AS integrate with investments. IFC Strategies. Country-level coordination is highly has yet to elucidate an overall model for inte- relevant, given that governments are involved in grating the two businesses. Given the large AS about half of IFC AS clients. A review of 33 joint presence in the field, and decentralization of IS Country Assistance Strategies--produced in 2007 operations, there are increased opportunities for and 2008--reveals that alignment of IFC AS with coordination between the two businesses in ad- Bank operations is often considered, though gen- dressing client needs. Beyond certain products, erally only in part, and there is typically limited such as SME linkages operations and in Access to reference to non-Bank actors, and to IFC addi- Finance, evidence of cooperation is limited. Since tionality (see figure 3.6).21 Country strategic co- AS is generally more programmatic in its makeup-- ordination is, however, not restricted to Country funding is agreed several years out, thus it gen- Assistance Strategies, and other mechanisms, such erally does not need to find investors/providers as private sector forums, have been tried suc- to cooperate on a single project, unlike with IS. cessfully in some countries. There is the potential for AS to serve as the an- chor business in the field. That is, if various chal- At the project level, it appears there is substantial lenges can be overcome. These include: different room for better up-front coordination with other program cycles; project timetables; processes, players. The majority of FY07 and FY08 IFC AS proj- and clients (IFC does not invest with state enti- ect approval documents, for instance, contained ties, the main client of AS); lack of personal in- no mention of the activities of, and complemen- centives to cooperate (especially for AS staff, a tarities with, other actors--even going so far as to majority, whose future is tied to the continuation say that no other donor or commercial provider of a particular program); and the possibility for in a country, region or sector does or could pro- conflict of interest (COI). Surveyed and inter- vide the service that IFC is proposing (figure 3.7). viewed staff expressed wide-ranging views about This gap in coverage of other players in the mar- AS/IS integration (from support for full integration ket is recognized by seasoned IFC managers and to rejection of any integration), although they staff as an area for improvement. As one manager usually voiced concern about the lack of clarity sur- put it, "At the project level, there is often very lit- rounding integration. Again, an umbrella AS global tle analysis of what others in the market are doing 38 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Figure 3.6. Country Assistance Strategies Provide Limited Coverage of Other Knowledge Providers Specific AS objectives Alignment with Bank Alignment with region M&E indicators Alignment with other institutions Alignment with IFC investments IFC additionality 0 20 40 60 80 100 Percentage of Country Assistance Strategies Detailed coverage Partial coverage No coverage Source: IEG. Note: Covers 33 joint Country Assistance Strategies completed in 2007 and 2008. (which should only take two or three meetings). Figure 3.7. Discussion of Activities and IFC has no business doing anything on the ground Complementarities with Others in without mapping what others are doing." On the Project Approvals Is Weak other hand, the strategies and project approval documents of other development institutions 90 typically do not explain how their knowledge serv- 81% ices exhibit uniqueness and align with those of 80 74% other providers. It shows that the development community, as a whole, has room for improvement 70 Percentage of project approvals in this respect.22 However, as discussed later, IFC's 59% 60 collaboration with donors during program and project implementation appears relatively strong. 50 41% Strategic coordination can also happen globally, 40 regionally, and by theme/sector. A good example 30 26% is the creation of a Bank Group unit, such as FIAS, 19% which accounts for around 20 percent of AS ac- 20 tivities with government clients. A more recent example is the development of a joint Bank Group 10 response to the global financial crisis, including new 0 resources, and some reallocation of funds for IFC Any other institution World Bank Non-World Bank AS. Going forward, IFC plans to place particular Mentions No mentions focus on its AS to the financial sector (especially Source: IEG. financial regulation matters) and infrastructure, Note: Covers 248 projects approved in 2007 and 2008. The 248 projects were selected for restructure existing business lines and products, review by stratified random sampling, excluded from the population were approval and introduce new efforts to help clients with risk documents for which the "IFC role" section was not completed (typically small, one-off management and workouts/restructuring.23 These workshops and seminars, or internally oriented projects). 39 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 efforts are still evolving, and their effectiveness of which was closely associated with availability will take time to determine. Past evaluation data on of donor funding. The heterogeneous nature of AS crisis responses are generally lacking because donor funding, and resultant programs, raised IFC's M&E system has only existed since 2006 concerns within IFC about the efficiency of this (after any major developing-country crisis). Also, model of funding for AS (not least because new ini- IFC AS was relatively small in scope during past tiatives required donor approval before being ini- crises. This evaluation provides some illustrative ev- tiated, which could lead to delays in addressing idence of how relevant and useful IFC AS was dur- client needs). Thus, in 2004 IFC established the ing prior crises (see the section on results), but it Funding Mechanism for Technical Assistance and focuses more on general insights that can be fac- Advisory Services (FMTAAS). At the same time, tored in as the Bank Group continues to adjust to IFC began seeking donor funding across longer the crisis, and to help inform the overall align- horizons, and on a more pooled basis (i.e., for a ment and delivery of AS. At the regional level, a cer- range of projects in a particular region, all regions, tain degree of donor activity mapping has occurred, or within a certain business line). IFC also looked for instance, with the development of the South to new, nongovernmental sources of funding, such Asia Enterprise Development Facilities. But inter- as institutional and private partners/foundations, views with staff and donors suggest more could be which provided 20 percent and 3 percent of do- done in this area. nor funds respectively, between 2004 and 2008. FMTAAS involves taking a portion of IFC's retained Delivery of IFC Advisory Services earnings and allocating it to the FMTAAS Trust This section examines four issues that are central Fund (using a sliding scale formula). Since 2004, to the delivery of IFC AS: IFC has made $715 million worth of FMTAAS con- tributions. This compares with $739 million of · Funding donor commitments, a leverage ratio of approxi- ·Project design and implementation mately 1:1 (table 3.4).24 Total donor commitments ·M&E systems have been highest for global programs, and low- ·Internal and external collaboration. est for Latin America and the Caribbean (figure 3.8), while donor leverage has been the greatest in The section concludes by comparing IFC's deliv- South Asia, where donors cover all of the costs of ery mechanisms with those of other knowledge the Bangladesh Investment Climate Facility, and service providers across these same dimensions. most of the costs of SEDF, and lowest in Latin America and the Caribbean, where IFC is expect- Funding ing to cover most of the Latin America and the A key factor in the delivery of any service is its fund- Caribbean Program, the only facility in the region ing. This is especially true of IFC AS, the emergence (table 3.5). Table 3.4. Two Main Funding Sources: Donors and IFC Donor IFC FMTAAS Total commitments commitments commitments Leverage Year ($ million) ($million) ($ million) (donor $ / IFC $) FY04 142.8 36.0 178.8 4.0 FY05 99.8 222.8 322.6 0.4 FY06 172.9 93.3 266.2 1.9 FY07 112.4 184.6 297.0 0.6 FY08 210.7 178.2 388.9 1.2 Total, FY04­FY08 738.6 714.9 1,453.5 1.03 Source: IFC Financial Controller reports. Note: Donor funding comes from governments (77 percent of the total), institutions (20 percent), and private partners/foundations (3 percent). 40 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Figure 3.8. Donor Commitments, Table 3.5. Donor Funding Leverage Has Been by Region, FY05­FY08 Highest in South Asia Latin America Leverage Europe and and the Caribbean Rank Region (Donor $ / IFC $) Central Asia 3% 9% 1 South Asia 7.9 South Asia 2 Middle East and North Africa 3.3 10% 3 Southern Europe and Central Asia 2.4 Global 40% 4 East Asia and Pacific 2.4 Africa 5 Africa 1.2 10% 6 Central and Eastern Europe 1.2 7 Latin America and Caribbean 0.5 Middle East Source: IFC financial controller reports. East Asia and North Africa Note: Includes funding cycles that were current in June 2008. 17% 11% Source: IFC. Since donor commitments are now typically pooled case of a public good, for instance, advice to a gov- for multiyear programs, and FMTAAS is designed ernment on business regulation. IFC management as long-term pot of funds, IFC's AS programs are, allows a certain degree of flexibility in applying the in effect, funded several years out. About half of pricing policy, depending on the project context. FMTAAS funds committed, $332 million, has been However, staff are expected to start from the as- spent to date. At the same time, the financial cri- sumption of 100 percent client cost contribution, sis is starting to affect commitments, in that no and justify any contribution less than that as a FMTAAS contributions are anticipated in FY09, special case. and donor contributions for new programs may be adversely affected. Donor funding is sometimes still In the two years since it was introduced, IFC's pricing policy raised on a project-by-project basis, as in Central the pricing policy has yet to have sig- implies 100 percent client and Eastern Europe, which not only raises sus- nificant traction. The vast majority of cost contribution in the tainability concerns, but also is not a cost-effective projects, before and after introduction case of a private good. approach to fund raising.25 of this policy, have received no or lim- ited contributions by clients (figure 3.9). Realized In the last few years, IFC had been seeking a third client contributions for approvals in calendar year source of funding--client contributions--which 2007 (following the introduction of the pricing become even more relevant in the event that FM- policy) amounted to $13 million, or 5 percent of TAAS and donor funding falls substantially. The total expenses--implying, improbably, that 95 per- thinking behind this, set out in a pricing policy cent of services were a public good in nature. By introduced in January 2007, is as follows: first, to region, Middle East and North Africa has achieved obtain client commitment to a project or pro- the highest level of client cost contri- gram; second, to avoid market distortion (com- bution and Central and Eastern Europe, Realized client petition with other knowledge providers and/or the least. By business line, client con- contributions amounted cross-subsidy of an IFC investment) by asking tributions were generally higher than to 5 percent of total clients to pay toward the cost of private goods; the private good component (the share expenses. third, to target any cost subsidies at public goods. of excludable benefits the client re- The policy implies full cost contribution by clients ceives): hence, INF and CA had higher cost re- in the case of a private good, such as corporate gov- covery than BEE (table 3.6). Nonetheless, whichever ernance advice to a company, and some (though region and business line, client cost contribution less than 100 percent) cost contribution in the fell well short of 100 percent. 41 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 ity (the issue is magnified for short-term con- Figure 3.9. Most Advisory Operations Featured No sultants, who will likely lose a future income flow Client Contributions in the event the project does not proceed). In other cases, clients who did not previously pay 100 for IFC services have shown reluctance to com- 90 12% 12% mit to cost-sharing. Clients know that IFC is using 80 Percentage of projects other people's money (donor funds), which also 70 perpetuates expectations that it should be free. 60 Meanwhile, government clients often face fiscal, 50 policy, and procedural constraints in providing 85% 84% 40 contributions.26 Cost contribution by clients is 30 generally higher in non-IDA countries than IDA 20 countries, though not always. For example, con- 10 tributions have been considerably higher in Mada- 0 gascar (an IDA country) than Indonesia (a non-IDA Pre-January 2007 Jan­Dec 2007 country).27 (n = 1,253) (n = 356) Approval year Advisory Client The slow implementation of the pricing policy Contribution: 0 1­50% 51­100% >100% raises several concerns. First, the willingness Source: IFC database. of clients to contribute toward the cost of a Note: Calculated as actual client cost contribution/actual expenses. Does not include in-kind service (where they are able to pay) provides contributions (e.g., use of office space), n = number of projects. some feedback about the value they place on the service--including for nominally public goods. Table 3.6. Middle East and North Africa Region and Indeed, the IMF is also seeking to introduce charg- Infrastructure Have the Most, Though ing for its Advisory Services,28 while the Interna- Still Limited, Cost Contributions by tional Bank for Reconstruction and Development Clients (IBRD) charges on a full fee-for-service basis for some advisory work in upper middle-income Average cost recovery (%) countries (about $15 million per year). In general, Rank Region Business line the higher the level of client contribution, the 1 Middle East and North Africa 18% INF 13% higher the value they assign to the project. In the 2 Latin America and the Caribbean 8% CA 7% absence of a contribution, there is no "market 3 Sub-Saharan Africa 4% A2F 5% test" of the project's value. Indeed, the provision 4 South Asia 3% ESS 3% of free or near-free AS could be market distort- 5 Southern Europe and Central Asia 3% BEE 1% ing, because: i) the project may directly com- 6 East Asia and Pacific 2% pete with projects offered by private providers of knowledge services; and ii) IFC may be indirectly 7 Central and Eastern Europe 1% Overall 5% competing with other financiers by effectively Source: IFC database. Note: Covers the period since January 2007 (when the Pricing Policy was introduced); cal- cross-subsidizing an investment it has with the culated as actual client cost contribution/actual expenses. Does not include in-kind contri- same client. The risk is that a company agrees to butions (e.g., use of office space). a loan it could have obtained in effect more cheaply from other sources, removing IFC's fi- In general, AS teams have found full execution of nancial additionality in the deal. There appears the pricing policy to be challenging. Staff report to be some limited evidence of cross-subsidy that they sometimes fear losing projects to other (as shown in table 3.7), which will need to be donors who are providing similar ad- addressed going forward. IFC additionality with In the absence of a vice for free, or losing them altogether, respect to AS alone may also be in question, es- contribution there is no if they ask clients to pay. This reflects pecially where the company is already being pro- "market test" of value. somewhat of a supply-driven mental- vided similar services. 42 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Table 3.7. Some Free or Near-Free Advisory Services Operations Cross-Subsidize IFC Investments Number of AS projects Level of cost recovery Type of IFC investment (Jan. 2007­Jan. 2008) Zero 1­50% 51­100% >100% Loan 41 66% 24% 5% 5% Equity 17 88% 8% 0% 0% Loan & equity 13 77% 12% 15% 0% IS in prospect 18 83% 17% 0% 0% All AS linked with investment (excluding BEE) 89 75% 18% 4% 2% AS not linked with investment (excluding BEE) 200 85% 12% 2% 2% Source: IFC database. Note: As of November 2008. Numbers of projects by business line are too small for comparisons between business lines. The pricing policy is cost-based, rather than mar- sign procedures were generally stronger, with INF ket value-based. As a result, a 100 percent cost con- having established quality-at-entry (QAE) compo- tribution by a client could still be market distorting, nents that mirror those used for new investment since it does not include a premium that would operations (i.e., with concept notes, clear risk be normal for a commercial provider (which is the identification, lessons from past operations, peer basis on which IFC investments are priced). Rec- review, etc). However, quality-at-entry efforts of this ognizing the inherent difficulties in pricing advice, depth were rare among other business lines, be- value could also be determined in terms of client yond the creation of standardized approval doc- success or impacts, as in the case of INF advisory uments--with, as discussed earlier, often weak mandates, which charge fees on the basis of a rationales for IFC embarking on a new project, and, transaction going ahead, or with energy efficiency as discussed later, limited use of appropriate base- audits. Alternatively, value could be linked to the line data.30 Key design flaws identified in this and future market value of the company, which is es- other reviews included: insufficient tailoring to sentially the venture capital model of reward for local conditions (particularly when delivered from up-front investment in a company (i.e., linking AS afar), and lack of realistic timetables. "payment" to a proportion of future equity value). The evaluation system does not currently track Project Design and Implementation project implementation quality on a systematic Past IEG evaluations and external reviews of specific basis, but past evaluation work shows that strong AS programs have repeatedly stressed the impor- implementation can compensate for weaknesses tance of good project design and implementation in project design. Interviews with, and IEG's sur- for stronger impact, both for beneficial outcomes vey of, managers and staff confirmed that project and the avoidance of adverse outcomes.29 IFC has implementation quality has been highly incon- responded to the need for sound design and im- sistent. The influences on project implementation plementation by introducing standardized proce- quality that emerged in interviews with managers dures for approval and supervision in 2005 and and staff, and in PCRs, included: level of staff 2006 respectively. This review found some evi- experience, degree of staff continuity, balance dence of good practice design and implementation, between local and global, and in-house and ex- for projects approved since then, but overall room ternal expertise, quality of short-term consul- for improvement remains. The INF and BEE busi- tants, internal procedures which have made for ness lines stood out as an area in which project de- a slow disbursement of funds, level of client com- 43 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table 3.8. Ratio of Staff to Consultant Expenses Is Roughly 1:1 Average, FY05 FY06 FY07 FY08 FY05­FY08 Expense type $ million % $ million % $ million % $ million % $ million % Staff 38.5 32% 51.0 34% 63.9 34% 82.7 34% 59.0 34% Consultants 34.7 29% 37.1 25% 56.1 29% 72.3 30% 50.1 28% Travel 13.0 11% 18.5 12% 23.4 12% 35.4 14% 22.6 13% Other (e.g., office rent & equipment) 32.5 27% 42.5 29% 47.1 25% 54.3 22% 44.1 25% Total 118.7 100% 149.1 100% 190.5 100% 244.7 100% 175.8 100% Source: IFC Financial Controller reports. mitment, lack of clearly defined exit strategies, ten- tice to a local setting, and retains global knowledge sion between operational growth and portfolio as a comparative institutional advantage over other quality, and lack of robust mechanisms to hold knowledge service providers. As IEG's evaluation individuals accountable for poor deliverables. of the PEP-ECA illustrated, getting the right Client commitment, tailored knowledge, and local/global mix of staff is fundamental to suc- strong performance M&E were especially im- cess.31 Second, while contracts of shorter duration portant, as reflected in their effects on project per- have provided IFC management with increased formance ratings (see section on results). flexibility, they have also meant a less well-defined career path for AS staff, with career progression The staffing model for AS differs from that of IS. dependent on the continuity of program funding First, AS staff have been based more in the field, rather than one's professional potential. While as illustrated earlier. Second, they tend to be competencies for AS staff have recently been de- newer to IFC than those involved with invest- veloped, there are no explicit incentives for them, ment operations (60 percent of AS staff have been as there are for investment staff, either in the with IFC less than three years). Third, over two- form of volume or locus of activities, or develop- thirds of AS staff are on coterminous contracts ment impact. These factors help explain why the (linking the staff contract to program funding). vast majority of IFC staff--whether or not they are Fourth, there has also been a much higher propen- employed in AS--believe that AS staff are less val- sity to use short-term consultants in project im- ued than their counterparts on the investment plementation on the AS side--AS employs as side.32 Going forward, given the greater presence many consultants as staff each year, while IS of AS teams in the field, there may be avenues for employs twice as many staff as consultants. In more long-term arrangements, with AS staff driv- general, reflective of the labor-intensive nature ing greater synergies between the two arms of of knowledge provision, staffing and consultant IFC's business. However, there would need to be costs have made up a higher share of total proj- appropriate training, including management for ect costs--about a third each (table 3.8). possible COI risk. Several opportunities and challenges have Third, the extensive use of short-term consult- emerged with this staffing model. First, since staff ants in project delivery affords IFC the opportu- are based predominantly in the field, IFC should, nity to buy-in expertise for a specific purpose, but in principle, be able to better appreciate client it does also presents continuity and quality is- needs and tailor project design accordingly. On the sues, with ramifications for IFC additionality. In other hand, as field staff tend to have stronger local FY08, the cost for employing consultants was than global expertise, it is a constant effort for IFC $72.3 million, only slightly less than IFC staff costs to ensure that it transmits international best prac- ($82.7 million), with a consultants to staff ratio of 44 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S about 1:1. Continuity issues arise since the con- Notwithstanding these steps toward Extensive use of short- tract, by its nature, is a one-off arrangement, and improved performance measurement, term consultants in IFC cannot promise clients any long-term imple- as with the introduction of most new Advisory Services has mentation support if it does not reemploy the systems, there have been "growing ramifications for IFC same consultant and needs to return to the mar- pains." IEG has assessed PCR quality additionality. ket to recruit in the skills. Quality issues arise in across the following dimensions: use of the need to: i) train new consultants in IFC meth- measurable indicators, appropriate baseline data, ods and procedures (not least M&E); and ii) offer soundness of logic model (differentiation between unique knowledge, as IFC is effectively only func- outputs, outcomes, and impacts), comprehen- tioning as a sourcing and funding agency if the con- siveness (discusses results of all components), sultant is already available in the market. Feedback concurrence with supervision reports, and in- from clients confirms these service continuity and corporation of useful lessons.34 Based on 458 re- quality concerns, suggesting in several cases that views carried out by IEG between 2006 and 2008, a more "hands-on" approach to oversight of short- there remains considerable scope for improve- term consultants might be required by IFC, and ment, and the approval and supervision docu- a general preference for IFC staff rather than con- ments that precede the PCRs (e.g., in setting sultant support. performance baselines and tracking performance against them). The CA business line and the Mid- M&E Systems dle East and North Africa region show better qual- As mentioned earlier, effective M&E is essential ity than others, but the general picture is of low for learning what works well, what does not, and PCR quality (figures 3.10 and 3.11).35 how strategy and operations should be redirected going forward. IFC management understands the IFC-commissioned reviews of AS facilities, prod- importance of M&E and, in 2006, introduced a ucts and projects, while offering some insights on new M&E system for AS, including standardized the organization and delivery of AS, have exhib- project approval, supervision, and completion ited some issues with independence. An evalua- reports. At completion, the AS team provides a tion is independent when it is "carried out by self-assessment of performance in a Project Com- entities and persons free of the control of those pletion Report (PCR), followed by independent review and validation by IEG (EvNote). The PCR and EvNote are completed following project Figure 3.10. Project Completion Report Quality, closure, as opposed to early operating maturity by Business Line in the case of investment operations, which tend to have longer project lifecycles. PCRs are com- 100 with minimum acceptable PCR quality Percentage of projects (on average) pleted for all AS projects, unless they were 90 dropped or terminated (which may be a lost op- 80 portunity for learning and bias results). IFC has 70 complemented the introduction of the PCR sys- 60 54% tem with the completion of some 51 program, 50 47% product, and project reviews by commissioned 40% 39% 40 33% consultants (including a handful of impact eval- 30 uations),33 the establishment of an IFC/Bank 20 Group project lessons awards program, portfo- 10 lio review meetings, as well as experimenting 0 with cost-benefit analysis. Finally, IFC has intro- CA ESS A2F BEE INF duced activity-based costing, although managers Source: IEG PCR Reviews, FY06­FY08. and staff report limitations with the IT platform. Note: Minimum acceptable quality is defined as, on balance, incorporating the following Taken together, these efforts put IFC at the fore- dimensions "to some degree": use of measurable indicators, appropriate baseline data, front of results measurement among MDBs and soundness of logic model, comprehensiveness (discusses results of all components), major donor organizations. concurrence with supervision reports, and incorporation of useful lessons. 45 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 culated and, in the case of regional reviews, has, Figure 3.11. Project Completion Report Quality, to a large extent, depended on interview evi- by Region dence. In one case, the program team requested that the consultant focus the review on just four Central and Eastern Europe 25% cases, all success stories. Also, the product and Latin America 33% project evaluations have been highly clustered in and the Caribbean the CA business line (see table 3.9), suggesting Multiregion 33% the need for more systematic selection of evalu- Regions South Asia 34% ation topics.37 Also, the reviews to date have East Asia placed limited emphasis on results and more 44% and Pacific on delivery (table 3.10 and appendix I). Such a Southern Europe 44% focus clearly limits the generalizations that can be and Central Asia made about the performance of a facility, prod- Sub-Saharan Africa 48% uct or project, and ultimately weakens the basis Middle East and North Africa 62% on which decisions can be made about future 0 20 40 60 80 100 funding. Percentage of projects (on average) with minimum acceptable PCF quality A good results measurement system should per- vade an organization. On this basis, there are some Source: IEG PCR Reviews, FY06­FY08. other gaps in terms of the M&E of AS. At the cor- Note: Minimum acceptable quality is defined as, on balance, incorporating the following dimensions "to some degree": use of measurable indicators, appropriate baseline data, porate level, IFC's scorecard, albeit with some soundness of logic model, comprehensiveness (discusses results of all components), limitations, includes targets for IFC development concurrence with supervision reports, and incorporation of useful lessons. impact and reach largely through its investment operations. Indicators for AS are very limited, IFC-commissioned reviews responsible for the design and im- which to some extent reflects the relative imma- have offered insights, but plementation of the development turity of the project M&E system, but also the exhibited issues with intervention." This indicates that in- absence of established M&E indicators for IFC's independence . . . and dependent evaluation presumes "free- impact at a programmatic level.38 The targets that methodological quality dom from political influence and are included for AS pertain to the number of has been inadequate. organizational pressure," "full access public-private partnership advisory mandates and to information," and "full autonomy the level of overall AS expenditures, neither of in carrying out investigations and reporting find- which captures IFC's development impact.39 ings."36 By contrast, the facility and product re- views that have been conducted to date have Collaboration with Others often been commissioned, overseen, and ap- Strategic cooperation--both internally and ex- proved by the responsible facility and product ternally--is critical for IFC if it is going to maxi- managers. Project reviews have been carried out mize its additionality and play its development role in something of a more detached way, under the to the fullest. The need for good cooperation purview of the Results Measurement Unit. While also applies to service delivery (collaboration). IFC-commissioned reviews can never be truly in- This section looks at three important types of serv- dependent, the degree of freedom from political ice delivery collaboration: i) with IFC investment influence and organizational pressure can be en- operations, ii) across the Bank Group, and iii) with hanced, for example, through a different part of donors. the organization from that being reviewed initi- ating and managing the review. Between 2006 and 2008, some 30 percent of AS projects were with existing or potential IFC in- Corporate performance Methodological quality has also been vestment clients (figure 3.12). This contrasts with indicators for Advisory inadequate. The methodological ap- EBRD, where 88 percent of AS activities support Services are very limited. proach has often not been well arti- EBRD investment projects, and the European In- 46 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Table 3.9. Few External Reviews Have Thus Far Focused on Impact Level Coverage Focus of Review Number Business Line Region Program Regional facility Delivery 6 All EAP, LAC, MENA, SA, SSA A2F business line Delivery 1 A2F na Product Single product Delivery 16 A2F: 3 na BEE: 2 CA: 10 ESS: 1 Project: Single project For all projects reviews: Ex post Outcomes vs. objectives 10 A2F: 2 LAC: 9 Ex ante Performance baseline 8 CA: 13 SSA: 6 During Progress report 3 BEE: 6 EAP: 5 With/without Impact vs. alternative: ESS: 2 SECA: 3 Pre/post 3 INF: 5 MENA: 2 Quasi-experimental 3 CEE: 2 Experimental 1 SA: 1 TOTAL 51 Source: IEG, based on IFC Results Measurement Unit report database. Note: Based on reviews that had been completed by December, 2008. vestment Bank (EIB) where virtually all AS is tied Figure 3.12. About One-Fifth of Advisory Services to existing or potential investments. As figure Operations Are Linked to IFC 3.13 shows, links are especially strong for A2F Investment Services Operations projects (46 percent), and most limited for BEE projects (2 percent, reflecting their predomi- 50 nantly public good nature). The section on results examines whether stronger additionality and de- 45 velopment impact seem to have been realized as 40 a consequence of various ties between AS and in- Percentage of AS projects 35 vestment operations. linked to IFC IS 30 Beyond the project level, there has been limited 25 programmatic integration between AS and IS to date, reflecting some of the alignment challenges 20 cited earlier. IEG's survey of, and interviews with, 15 IFC managers and staff, found repeated refer- 10 ence to integration between AS and IS as some- thing IFC did least well in delivering its AS. They 5 also pointed to disincentives to align, discussed 0 earlier, such as different program and personal ob- 2005 2006 2007 2008 jectives (the latter particularly important for AS Fiscal year staff whose future depends on the continuation Link to existing IFC investment Link to IFC investment in prospect of a program), as well as practical constraints, such as unclear understanding about the intended Source: IFC database. 47 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 on which IFC is advising. For example, an IFC in- Figure 3.13. Advisory and Investment Services vestee company may express interest in bidding Linkages Are Greatest for Access to for a privatization deal on which IFC is acting as Finance Operations an advisor. In this case, the AS infrastructure ad- visory team's independence and objectivity could 50 46% be compromised by a perception of favoritism, 45 or if public confidence in their independence 40 Percentage of AS projects and handling of confidential information is 35 eroded. COI risk can arise when IFC, on one linked to IFC IS 30 hand, gives regulatory advice to government 25 clients, and, on the other, has investment or fi- 22% 20% nancial interest in private sector entities whose 20 business prospects are materially affected by the 15 10% regulatory advice. For instance, an AS project in- 10 volving assistance to a central bank to develop 5 2% banking supervision modalities raises significant 0 COI concerns, if IFC has investment interests BEE Infrastructure Sustain- Corporate Finance in the regulated banks in the country. Such cases ability advice typically exhibit greater COI risk than single- Source: IFC database. borrower AS projects delivered to IFC investee companies.40 model for AS/IS integration, and different pro- gram timetables and cultural differences between IFC's COI guidelines stipulate that business line the two businesses. As IFC continues to decen- leaders and regional directors are primarily re- tralize its IS, there appears to be strong potential sponsible for identifying actual, potential, or per- for AS to serve as the anchor for linkages be- ceived COI with respect to operations in their tween the two, because it is a generally more respective departments, and managing these programmatic business, with greater field pres- cases--with or without the assistance of the ence. But a clear integration paradigm and oper- COI office. Staff are expected to inform the busi- ational incentives to integrate are not yet apparent. ness line leaders and/or regional directors in a timely fashion about any issues relating to COI, Whatever the model, closer integration brings and leaders/directors determine whether any COI with it the possibility of COI, which is fundamen- exists, and whether assignments should be re- tal for any business, especially an advisory business, ferred to the COI office for clearance (as well as and needs to be carefully managed. Objectivity of whether directors outside of the joint depart- advice is key for maintaining a good business rep- ments are likely to be affected and should be no- utation, and can be impaired in situations where tified). Handling of COI is also the responsibility AS is perceived to be unduly influenced by the pres- of leaders and directors, as is ensuring that staff ence of an IFC investment or financial interest, are adequately trained. The COI guidelines lay out or is motivated primarily by a desire to help IFC several mechanisms that should be considered generate new business in the form of new invest- in effective handling of COI: i) providing full dis- ments. It is important, therefore, for IFC to main- closure to the affected parties; ii) obtaining client tain its independence in offering knowledge consent to multiple roles to be played by Bank services to its clients, and to have pro- Group entities; iii) instituting separate project Conflicts of interest needs cedures in place to manage potential, teams as appropriate; iv) sequencing assignments; to be carefully managed. actual, and perceived COI. v) reducing the scope of an assignment; vi) trans- ferring the assignment to a unit outside the jointly Conflicts between IFC AS and IS relate to IFC hav- managed department; and vii) establishing mech- ing an actual, apparent, or possible financial anisms to protect the flow of confidential and interest (e.g., loan or equity interest) in an issue other sensitive information. 48 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S In the past, there had been no systematic data on providers and also government as a client (lower the extent to which COI cases were identified transaction costs). More importantly, combining and settled outside of those referred to the COI approaches has the potential to contribute to office, so it was not possible to determine how greater development impact, through identifica- comprehensively those cases had been identi- tion and exploitation of respective comparative fied and managed. AS/IS conflict situations ac- advantages and synergies, avoiding service dupli- counted for 151 (51 percent) of 298 total referrals cation, and learning from one another. This po- to the Bank Group COI office between FY06 and tential has been recognized by IFC and Bank FY08. By business line, most Advisory/Investment management, and various steps have been taken COI cases relate to the INF and A2F business to align service delivery, including: establishment lines (39 and 29 percent of cases, respectively).41 of joint departments and teams; transfer of MIGA Investment Promotion Agency technical assis- The fact that only a quarter of AS projects with IS tance work to FIAS/IFC; joint IDA/IFC Secretariat; connections that were approved in the last three joint strategy sessions; and guidelines for IFC ad- years were referred to the COI office could al- visory staff on cooperation with the Bank. It should ternatively be a sign of weak identification of COI be noted that the costs and risks of cooperation or strong local management and resolution (with- may sometimes exceed the benefits of coopera- out the need for intervention from the COI office). tion, and thus the appropriate level of cooperation However, IEG's survey of IFC managers and staff needs to be judiciously determined.42 does suggest some scope for improvement. In the survey, respondents reported that nearly 40 per- In principle, there is most fertile ground for co- cent of the time, when a conflict did arise, they operation on BEE and INF work, where the client did not feel that it had been resolved effectively. is typically government. Opportunities for coop- IFC's new COI guidelines should, if applied cor- eration do also arise in relation to ESS, A2F, and CA, rectly, help improve conflict resolution, in that they although generally to a lesser extent (client is usu- call for identification of actual, potential, or per- ally not government). In practice, cooperation ap- ceived COI in each new project approval docu- pears to have followed this pattern. Of the 26 ment, which did not happen before. However, the percent of new project approvals in FY07 and FY08 guidelines do not call for ongoing tracking of COI that refer to Bank activities, nearly two-thirds were cases in project supervision and completion doc- BEE operations and one-sixth were INF operations. uments, which could be a useful complement. The While documentary reference to an- There is most fertile INF business line seems to stand out as an area other institution's activities does not ground for cooperation of relatively good practice--with well-established necessarily mean that there was actual on BEE and INF work, procedures for transparently disclosing informa- cooperation or that it was of a good where the client is the tion to affected parties, protecting the flow of quality, these data are consistent with government. confidential information through the establish- feedback received from interviewed ment of "firewalls" between AS and IS teams, and IFC and Bank staff about areas in which coopera- sequencing assignments. Beyond guidelines and tion is taking place.43 Recent examples of Bank procedures, experience suggests that the com- Group cooperation include: a joint Doing Business mitment and leadership of managers (business Reform Advisory unit; BEE programs in Bangladesh, line leaders and regional directors) plays an im- Kenya, and Yemen; joint infrastructure projects portant role in effective COI management. in Kenya, Senegal, and Uganda; and IFC advisory staff providing diagnostic and implementation Collaboration in knowledge service provision support to IBRD loans in Georgia and Tajikistan. across the Bank Group is important in at least two respects. From a purely practical perspective, At the same time, staff also pointed to a lack of or- IFC shares the same primary client as the Bank in ganizational and personal incentives to cooper- about half of its AS operations: government. Close ate, and even to compete with one another,44 as well coordination of efforts can provide for delivery ef- as a lack of clarity about the other institution's ficiencies, on the part of both the Bank Group as products, delivery mechanisms, respective roles 49 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 and comparative advantages. In Bangladesh, for ex- impacts). IEG's survey of IFC managers and staff, ample, Bank staff had little knowledge of the na- as well as an IFC-commissioned survey of donors, ture of IFC's AS activities in the country, and vice broadly concur with the view that IFC's relation- versa. In general, staff felt that opportunities to ex- ship with donors in the field is generally sound. ploit synergies were not being maximized, with po- tential in some cases for service duplication. The Some donor representatives, however, felt that IFC fact that about half of new IFC AS project approvals should be more active in its outreach and knowl- with government clients do not even mention the edge dissemination. The desire for more IFC Bank provides broad corroboration of less than op- outreach was also raised by other stakeholders timal engagement across the Bank Group. Issues (e.g., United Nations Development Programme, of competition, or overlap, came up most in rela- U.S. Agency for International Development) dur- tion to BEE work, where the line between one in- ing IEG field visits. IFC was frequently compared stitution's activities and those of the other is to the Bank, and several times stakeholders felt blurred. Client governments can potentially be that "IFC is not at the table." In other words, dealing with four different units of the Bank Group IFC's presence in the field does not appear to have (Financial and Private Sector Development, Poverty translated into visible outreach for some stake- Reduction and Economic Management Network, holders. "We know IFC is there, but we do not feel FIAS, and an IFC regional facility). At present, joint them" was another comment that was made. This BEE teams that bring these units together in one view was shared by a number of IFC managers and delivery platform are the exception, not the norm. staff interviewed by IEG, who also felt that it was, The relative growth of IFC/FIAS activities provides to some extent, a trade-off of rapid growth (i.e., an impetus for renewed focus on alignment of lack of time to do outreach). Donors generally BEE services across the Bank Group, although had the most favorable view of outreach efforts there does seem to be resistance from some indi- in the Europe and Central Asia region. However, viduals, who fear a loss of established "turf." There the approach in this region does rely on a differ- is no system in place to systematically measure ent funding structure and engagement with and monitor the results of such efforts, which is an donors--project by project--which is not the issue for the Bank Group as a whole to address, and case in other regions, so it may not be replicable. which has been identified in previous IEG evalua- In which case, an alternative approach may be re- tions (in addition to general incentive issues).45 quired, such as more dedicated donor/partnership relations in the field. At present, the outreach During the course of its regional visits, IEG met with task often falls to managers and staff, who are about 30 representatives of donor organizations, otherwise engaged in program delivery. who provided valuable feedback on IFC's delivery performance. Donors included the Canadian In- Comparing IFC AS: How Others Deliver ternational Development Agency, U.K. Depart- Knowledge Services ment for International Development (DFID), It is instructive to compare the way IFC delivers its Netherlands Development Finance Company, AS with other development institutions that have Swedish International Development Agency, and private sector-oriented knowledge services pro- U.S. Agency for International Development--all grams. IEG's comparator review included a com- major contributors of funds to IFC AS programs. parison of the funding/pricing, delivery mechanisms, On the whole, donors reported a high level of sat- and M&E systems of each of the major multilateral isfaction with IFC, offering favorable views on the donors--EBRD, ADB, IDB, AfDB, European Com- technical quality, relevance, and timeliness of IFC's mission, and European Investment Bank--and two work, as well as the pricing policy (as a means to bilateral donors, DFID and the Danish Interna- reduce subsidies for the supply of private goods, tional Development Agency.46 The review also and target donor funds purely at public goods, if looked at their knowledge service strategies, ac- implemented effectively), and the relative sophis- tivities, comparative advantages, and evaluated re- tication of IFC's M&E framework (although they sults, which are covered elsewhere in the chapter. had yet to see much reporting on outcomes and It should be noted that some benchmarking of 50 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Box 3.1. How Does IFC's Delivery Approach Compare with That of Other Development Institutions? The high-level benchmarking exercise found a number ance between local and regional/global staffing and of common delivery issues among institutions, and some between using in-house staff and consultants; lack of relative strengths of IFC in terms of its delivery ap- output/product standardization (which IFC is moving proach. Highlights of the review include: toward). Funding/pricing: Most organizations rely heavily on M&E: Most development banks do a poor job of sepa- donor funding (more efficient if pooled), and provide rating knowledge services from other activities for the knowledge services free of charge. Although some purposes of monitoring, defining performance indicators, MDBs (notably EBRD and ADB) have made progress in and conducting ex-post evaluations. Although some defining cost recovery policies, IFC is relatively ad- development banks (EBRD and ADB) have begun to vanced in its thinking in this area (at least in principle, adopt better M&E systems, IFC appears to be ahead in with a private/public good-based pricing policy). its approach. Project design and implementation: In general, rela- Internal coordination: For MDBs that provide advice and tively ad-hoc project selection (more than strategies lend directly to private firms, COI procedures do not would suggest), and weak quality-at-entry; striking a bal- appear to be well advanced. service provision with other institutions has taken It concludes by examining the performance ex- place in IFC, but on a fairly limited scale.47 IFC may periences of other MDBs that provide knowledge accordingly be missing out on opportunities for services. Currently, M&E systems and standards learning from others, and adjusting its services for are too immature across the various institutions maximum comparative advantage and impact. to enable direct performance comparisons. The comparator review found a number of com- Relevance of Resource Allocation mon delivery issues among institutions, with IFC's A necessary (but not sufficient) condition for de- delivery approach generally comparing favorably velopment effectiveness is relevance: the extent with that of other development institutions. Com- to which resources were allocated where the mon delivery issues included: improving donor need was greatest, and consistent with corpo- coordination through pooled funding approaches; rate strategic priorities. IFC appears to be target- relative ad-hoc project design and weak quality at ing its AS resources toward high-need destinations entry; striking a balance between local and non- (and Bank Group strategic priorities), i.e., Sub-Sa- local staff; and between in-house and outsourced haran Africa as a region and low-income IDA coun- expertise. IFC exhibited relative strengths in terms tries more generally. This allocation pattern is of its approach to funding (pricing policy), M&E, broadly in line with the general pattern of official procedures for handling COI, and steps toward overseas development aid--grants, loans, and greater product standardization--assuming these technical assistance provided by official agencies measures are implemented effectively. (See box to developing countries (figure 3.14). It, thus, re- 3.1 for a summary of findings of the review of com- inforces the need for IFC to carefully map its ac- parator MDBs). tivities against those of other aid organizations, particularly where the client is a government Results of IFC Advisory Services agency and recipient absorptive capacity is weak, This section examines the following dimensions to avoid overlap.48 of IFC AS results: By business line, resource allocation has focused ·Relevance of resource allocation on countries that would appear to be most in ·Project development effectiveness need, prima facie, of knowledge services (see ·IFC additionality. table 3.10). Individual project evaluations sup- 51 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 port this conclusion, with strategic relevance rated Figure 3.14. Majority of Operations in IDA high in the vast majority of cases. Instances of Countries Are Similar to Official Aid low strategic relevance, however, even if relatively Pattern small in number, do tie up resources that could be used on other, more relevant endeavors (i.e., 70 Share of funding/disbursements (percent) they impose an opportunity cost on IFC). There 60 may also be implications for IFC additionality, if the service IFC is providing could have been obtained 50 from another source--as with a small number of operations in high-income countries (nominally 40 intended to support "South-South" investments). 30 For some business lines, there is no direct com- parator with the pattern of IFC investments, but 20 where there is (e.g., infrastructure), AS appear to be somewhat more oriented to high-need coun- 10 tries. This reflects the demand for appropriate 0 enabling environments for investments to take IDA/Non-IDA Africa place (e.g., an appropriate legal framework for Official development assistance IFC AS public-private partnerships). It also highlights the potential for AS to serve as an anchor for closer Source: IFC and OECD databases. Note: Shares are based on 2006­07 calendar year AS funding and net official development synergies between AS and IS teams--rather than assistance disbursements. the alternative of AS feeding off investment client needs--in that the AS intervention in the sector would precede, and help set up the conditions for, the investment intervention (so long as the in- Table 3.10. By Business Line, Resources Have Tended vestment takes place on a level playing field, avoid- to Be Allocated to Countries in Greatest ing any COI, as discussed above). Need Share of Share of Project Development Effectiveness Business line: AS IFC IS The real test of effective resource allocation is IS sector Country indicator operations operations whether the project actually delivered beneficial A2F: < 1/2 of population has access to 87% 84% impact in the field. The PCR system, introduced Finance (Inv) a formal account in 2006, seeks to capture such results. This sys- tem, as well as assessing strategic relevance, in- BEE: High riska and/or in the bottom 80% 62% cludes measures of output achievement, outcome All (Inv) half of Doing Business rankings achievement, and impact achievement. Taken to- gether, considering also the project's efficiency, CA: Informality > 30% 63% 69% an overall synthesis rating (not an average) is as- All (Inv) signed for the project's development effectiveness. (See box 3.2 for definitions and criteria for each INF: < 10 prior PPI projects 66% 43% of these evaluative terms). Infra (Inv) Considering the relative immaturity of the PCR ESS: Bottom half of Environment 74% 52% system, IEG has focused much of its effort to date All (Inv) Protection Index rankings on the evaluative substance of the PCRs, assessing the sufficiency of evidence and correct application Source: IFC and Doing Business databases; Institutional Investor; World Bank A2F and PPI databases, Environmental Performance Index 2008. of the guidance in assigning ratings--supple- a. With an Institutional Investor country credit risk rating of less than 30 (out of 100). Com- mented with selective field validation. In 2008, puted by $ volume, as of June 2008. IEG undertook field verification of performance in 52 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Box 3.2. How Is Development Effectiveness Rated? PCR performance ratings, which IEG verifies through Impact achievement--Intended longer-term effects of desk and field validation, are assigned in the following the advisory intervention. dimensions: Efficiency--Ratio of costs to benefits; economy in the Strategic relevance--Importance to achieving country use of resources; cost in relation to alternatives strategic objectives, appropriateness at initiation and completion, including whether AS was the appropriate These ratings are then synthesized (not averaged) into instrument. a single development effectiveness rating, on a six- point scale from highly successful (overwhelmingly Output achievement--Immediate project deliverables positive development results and virtually no flaws) to (products, capital goods, services, or advice). highly unsuccessful (negative developments and no Outcome achievement--Short- or medium-term be- positive aspects to compensate). The full rating crite- havioral changes resulting from the advisory project ria for each of the indicators are set out in appendix H. (positive and negative, intended or unintended). about one-third of cases.49 IEG has reviewed 458 ness ratings in 38 percent of reviewed opera- out of 707 PCRs completed by IFC up to June tions, and impact ratings in 72 percent of cases.50 2008--a coverage rate of 65 percent, and repre- Of the 38 percent of cases, some 25 percent were sentative across multiple dimensions. (See ap- rated too soon to tell at the time of completion. pendix G for further details). Weak M&E quality meant that development ef- fectiveness was not discernable in approximately Evidence of achieved results from AS can be hard 10 percent of cases.51 More consistent M&E qual- to discern for two reasons related to the nature of ity, as well as after-project-completion follow-up, knowledge transmission. First, knowledge, in many would enable greater understanding of develop- senses, is intangible. New methods of thinking ment effectiveness. and work habits, and their effects, can not easily be measured. Second, even when knowledge is Of the 285 projects for which development ef- tangible, such as with the specific diagnosis of a fectiveness ratings could be assigned, some 70 gap in business procedures, the response (im- percent were rated high for development effec- proved procedures) may take some time after tiveness. Among the individual indicators, there project completion to have an impact (because was considerable divergence. As figure 3.15 shows, those affected by the new procedures take time projects were rated strongest on strategic rele- to adapt). Thus, some knowledge impacts will vance (90 percent high), and weakest on impact never be captured, and others not at project com- achievement (52 percent high). Illustrations of pletion, when results evaluation is currently car- high development effectiveness are provided in ried out. These constraints are compounded by box 3.3. The impact rating is a particular concern the relatively weak application of M&E guidelines because IFC is ultimately in the business of pro- to date by IFC staff. Having more consistent M&E moting development impact. However, impact is quality, where development effectiveness is dis- less within IFC's control than relevance, since it cernable, as well as after-project-completion, M&E takes time to achieve and in the process can be follow-up, would enable greater understanding of influenced by exogenous factors, notably the level development effectiveness. of client commitment to the project. Together, these factors have contributed to IEG The INF, BEE, and CA business lines exhibited the being unable to assign development effective- highest development effectiveness ratings (be- 53 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 3.15. Strategic Relevance Was Often Rated High, Impact Achievement Much Less So 100 90% High development effectiveness rating 90 85% 80 71% 70% 70 60 52% 50 40 30 20 10 0 Relevance Output Outcome Impact Development (n = 440) (n = 439) (n = 301) (n = 131) effectiveness (n = 285) Source: IEG PCR Review data. Note: Excludes cases where it was too early, or data were insufficient, to discern performance. The justifications for the "efficiency" rating have been particularly weak given that the use of cost-benefit analysis has been introduced only recently. Staff also failed to provide information on cost effectiveness (other potentially less costly ways to achieve the objectives) and the comparison to other, similar projects to assess whether resources were spent economically. As a result of these weak justifications and missing analysis, IEG was unable in a majority of projects to validate the self-rating for efficiency. Also, in cases where it is still too early to observe and measure the outcomes and impacts of a project, it is similarly difficult to assess efficiency in the absence of knowledge about the quality of results. Box 3.3. What Does Strong Advisory Services Development Performance Look Like? Development results span a range of different social, Corporate Advice: An IFC-designed linkages project di- economic, and financial indicators, depending on the rectly helped 200 small businesses win contracts with business line and product type. Thus it is not possible an IFC client worth approximately $40 million per year. to compare directly the realized impacts across all proj- ects, but rather the extent to which each project met its Environmental and Social Sustainability: IFC project impact objectives. Below are illustrations of different helped improve the labor conditions for over 50,000 kinds of project development results: workers in a country's apparel industry. Access to Finance: IFC's training program paved the way Infrastructure: IFC assisted a government in tendering for $32 million of new trade finance to four client banks. for a Public-Private Partnership arrangement, covering dialysis services for eight public hospitals, which led Business Enabling Environment: Implementation of IFC to higher-quality dialysis treatment for over 200,000 report recommendations led to average number of days people. to obtain a business license in the country, a major bar- rier to business establishment (and thus job creation), to be reduced by 93 percent. tween 71 and 77 percent high). However, per- This was mainly associated with weak performance formance lagged in ESS, which had a significantly in Latin America and the Caribbean and Sub- lower proportion of projects with high ratings, 58 Saharan Africa (42 percent and 17 percent of proj- percent, than other business lines (figure 3.16). ects, respectively, achieved high ratings). The ESS 54 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S ratings are a matter of concern, for a number Figure 3.16. Environmental and Social of reasons. First, as evaluation of IS has shown, Sustainability Project Ratings Sub-Saharan Africa has historically exhibited the Have Lagged Behind Those of Other weakest standards of, and commitment to, envi- Business Lines ronmental and social performance, both at the country and company levels.52 Second, ESS is one 100 High development effectiveness rating of the main business lines in the Latin America and 90 the Caribbean region, accounting for about a quar- 80 77% 74% ter of projects. Third, attention to environmental 71% 70 65% and social issues tends to weaken when companies 58% 60 are in financial distress, which is a growing phe- 50 nomenon in light of the current global financial cri- sis. It should be noted that products in the ESS 40 business line are generally younger and less often 30 replicated than those of other business lines. It is 20 therefore doubly important that IFC learns les- 10 sons from these experiences, including through 0 more robust lesson capture in PCRs, to improve INF BEE CA A2F ESS its contribution to sustainable development. Business line Source: IEG PCR Review data. In one case for an African FI in which IFC had an Note: Excludes cases where it was too early, or data were insufficient, to discern performance. existing investment, IFC designed a project to im- prove the environmental due diligence capacity of its 50 loan officers, and to help improve oversight lenges encountered when IFC tries to The INF, BEE and of the company's subprojects. The project had to persuade clients that environmental CA business lines be cancelled due to lack of client buy-in (the client and social sustainability is a worthwhile exhibited highest did not see the fit between the project and their pursuit. Lack of local IFC E&S presence effectiveness ratings. bottom-line/competitive advantage), and funds seems to have been a limiting factor.53 were returned to the donor. This shows the im- (Box 3.4 provides other examples of low devel- portance of client commitment, but also the chal- opment impact across different business lines.) Box 3.4. Illustrations of Low Development Impact Below are examples of intended impacts that were not Corporate Advice: A training program designed to en- achieved in IFC projects: hance the capacity of local consultants, who were to train 60 micro, small, and medium enterprises in good Access to Finance: A project to train an IFC investment management practices, was managed poorly and ter- client bank's 50 loan officers in environmental due dili- minated early, without the desired capacity-building gence (and thereby improve oversight over the FI's sub- effect. projects) was cancelled due to lack of client buy-in (the client did not see the fit between the project and Environmental and Social Sustainability: An experi- their bottom-line/competitive advantage), and funds re- mental project to promote sustainable cultivation in the turned to the donor. rainforest, so locals could earn a better living from con- serving the forest (rather than cutting it down), led to only Business Enabling Environment: A project to improve 17 out of the anticipated 250 farmers reaching mini- the ease of business registration did not have the de- mum wage; problems between the sponsors ultimately sired effect of reducing informality. led to the cancellation of the project. 55 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table 3.11. Selected Ratings, by Product Rank Product Classification Business line % High 1 SME linkages (with IFC inv) In development CA 100% 2 INF--other Other INF 90% 3 Industry-specific BEE Entry BEE 82% 8 Non-(IFC) investment linked value chain/sector work Exit CA 64% 9 Investment policy & promotion Developed BEE 61% 10 CA--other Other CA 59% Source: IEG PCR Review data. Note: Product lines with 10 or more project ratings. For those product lines with 10 or more ratings, the "exit" classification of non-IFC investment- performance was highest for SME linkages work linked value chain work, and imply that the "de- in the agribusiness, extractive, and manufactur- veloped" classification of Investment Policy and ing sectors (100 percent, much higher than for Promotion projects may need to be reconsidered noninvestment-linked value chain work) and (or execution improved). lowest for "CA--other" (59 percent) (table 3.11). Since product maturity is based partly on achieved By region, Southern Europe and Central Asia op- results, this may imply that linkages projects should erations were rated significantly higher than other graduate from "in development" to "developed" regions (figure 3.17), while those in Latin Amer- product status. The ratings also seem to endorse ica and the Caribbean (and a small number of global operations) lagged significantly. All business lines in Southern Europe and Central Asia other Figure 3.17. Ratings of Operations, by Region than A2F were rated high, with performance es- pecially strong in Serbia and Macedonia. Mean- while, multiregion operations related to A2F Latin America and 64% projects were mostly rated low, while relatively low the Caribbean ratings in the ESS business line (where projects East Asia and Pacific 70% were spread thinly across 10 countries) pulled down Latin America and the Caribbean's overall Central and Eastern Europe 70% performance. Sub-Saharan Africa 71% What explains high and low ratings for develop- ment effectiveness? Given that the M&E system South Asia 72% is still evolving, it is premature to construct an econometric model of project performance, as Middle East and North Africa 73% IEG has done for IFC investment operations. That said, IEG's validation work does suggest a num- Southern Europe and 80% ber of possible project success factors: Central Asia 0% 20% 40% 60% 80% 100% (i) Country conditions: AS performance is High development effectiveness rating stronger in high-risk environments (figure Source: IEG PCR Review data. 3.18), driven by strong performance in BEE Note: Excludes cases where it was too early, or data were insufficient, to discern performance. operations.54 56 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Figure 3.18. Country Conditions: Figure 3.19. The Overall Relationship between Better Ratings Where Advisory and Investment Services Country Business Performance Is Moderate Climate Risk Is High 83% 80 High development effectiveness rating 77% 1 2 IFC investment development outcome HIGH 75 28% 55% High IFC AS 70 68% High IFC Inv High IFC Inv Low IFC AS 65 65% 4 3 60 7% 10% 55 Low IFC AS High IFC AS Low IFC Inv Low IFC Inv LOW 50 Non-high risk High risk LOW HIGH Country risk at approval IFC Advisory Services development effectiveness Source: IEG PCR Review data. Source: IEG PCR Review data. Note: Excludes multiregion projects, and projects with no country Note: Based on 30 instances of AS and IS with the same client, for which performance data business climate risk data; where no development effectiveness for both types of services were available. rating was possible. of the costs of the project, which is an in- (ii) Certain programmatic interventions: dication of commitment (figure 3.22). This · Some (though not all) AS and IS com- effect is particularly pronounced for ESS binations, such as linkage operations in operations, where projects with no client agribusiness, manufacturing, oil, gas, contributions achieved high ratings in mining, and chemicals (table 3.11), and only 44 percent of cases, compared with 70 ESS interventions with investee clients. percent of cases where there was a client The overall relationship between the contribution. performance of AS and IS provided to (iv) Sound project design: the same client is moderate (figure 3.19). · Realistic objectives and timetable; · Where AS was combined with an IFC · Tailored to local conditions (a possible loan rather than equity, AS development problem with multiregion offerings and performance seems to have been ESS operations, many of which are man- stronger (figure 3.20), which may reflect aged from headquarters); greater interaction with, and leverage · Clearly defined exit strategy. over clients to implement changes rec- (v) Effective project implementation: ommended by AS in the case of loans. · Good mix of global and local expertise, On the investment side, more than with locally based task leader (figure 3.23); half of equity investments (57 percent) · Good quality consultants (where used); achieved high development outcome · Effective cooperation with Bank and ratings, in spite of low AS ratings;55 and others; · Where AS operations were sequenced, · Implemented on schedule (i.e., without rather than one-off (figure 3.21). delay); (iii) Client commitment/buy-in: Better ratings · Flexibility to respond to country and where the client contributed some or all market needs. 57 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 3.20. Better Advisory Services Figure 3.21. Stronger Performance in Ratings When Combined Repeat or Combined with IFC Loans Rather Advisory Services Than Equity Operations 80 80 77% 76% High development effectiveness rating 75 75 High development effectiveness rating 70% 70 70 66% 65 64% 65 60 60 55 55 50 50 Equity Not linked Loan No Yes Linked with an IFC investment Combination or continuation of an AS project Source: IEG PCR Review data. Note: Excludes projects where no development effectiveness rating Source: IEG PCR Review data. was possible, and five cases where there were both equity and loan Note: Excludes projects where no development effectiveness rating investments. was possible. (vi) Strong IFC role and contribution (figure Figure 3.22. Client Commitment: 3.24), which was especially noticeable for Better Ratings When BEE projects in high-risk IDA countries;56 Client Contributed and (vii) High M&E quality, from approval to com- 80 pletion (figure 3.25). High development effectiveness rating 75% 75 These success factors are broadly consistent with feedback provided in IEG's survey of IFC staff. The 70 top 10 success factors that IFC staff cited were: 64% strong client commitment, fit with client needs, 65 good project design, in-house expertise, deep understanding of the issue for which the advice 60 is provided, local knowledge and presence, strong task leader commitment, tangible target out- 55 comes, cooperation with the Bank and other part- ners, and strong project management. 50 No Yes As the M&E system evolves in the coming years, Client contributed and more data become available, IEG will be seek- Source: IEG PCR Review data. ing to confirm the statistical significance and Note: Excludes projects where no development effectiveness rating relative influence of various drivers which might was possible, and where contribution data were not available. emerge. The associations presented in the fig- 58 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Figure 3.23 Project Implementation: Figure 3.24. Higher Role & Contribution, Better Local Presence Is Ratings Associated with Better Results Excellent 100% Role and contribution quality 80 78% high development effectiveness rating 75 Satisfactory 81% Percentage of operations with 70 Partly 18% Unsatisfactory 65% 65 Unsatisfactory 0% 60 0 20 40 60 80 100 55 Percentage of operations with high development effectiveness rating 50 Source: IEG PCR Review data. Headquarters Local Note: Excludes projects where no development effectiveness rating was possible. Task leader location Source: IEG PCR Review data. for IFC's future strategy and service delivery. First, Note: Excludes projects where no development effectiveness rating it may be more effective for IFC to focus its AS on was possible, and where task leader location was not available. high-investment-risk countries (not just IDA), preparing the ground for private investment. Second, in the longer term, programmatic ap- ures above are significant in binary terms,57 but proaches have potential for greater impact, but further analysis is needed as the quality of the data- equity/AS combinations, as currently formulated, base improves, to continue testing the robustness seem to lack leverage (IFC can impose more of these relationships. IEG did not find significant associations between Figure 3.25. Better M&E Quality, development effectiveness ratings and a number Better Ratings of other variables: IDA vs. non-IDA operations (al- though performance was better in those that 80 79% Percentage of operations with high exhibited high country risk), separating out oper- development effectiveness rating 75 ations with government clients (suggesting that capacity constraints were not a limiting factors), 70 frontier vs. nonfrontier countries; conflict-affected vs. non-conflict-affected countries; the level of ma- 65 turity of a product (i.e., whether it was entry, in de- 61% velopment, developed, exit, or other); and project 60 size.58 Importantly, the review found no significant difference in the performance of AS projects started 55 before or after the organizational changes that were initiated in 2005/06 (figure 3.26). By com- 50 pletion year, ratings were slightly lower for opera- Low High tions completed in 2007 and 2008 (figure 3.27). M&E quality Source: IEG PCR Review data. If future data should hold up the above associa- Note: Excludes projects where no development effectiveness rating tions, there could be several general implications was possible. 59 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Figure 3.26. Similar Performance for Figure 3.27. Slightly Lower Projects That Began Performance for Recently between Periods Completed Operations 2003­05 and 2006­08 100 100 high development effectiveness rating 90 high development effectiveness rating 90 Percentage of operations with 80 73% Percentage of operations with 80 74% 70 67% 70 69% 60 60 50 50 40 40 30 30 20 20 10 10 0 0 2005­06 2007­08 2003­05 2006­08 Advisory Services projects, Advisory Services projects, by end fiscal year by start fiscal year Source: IEG. Source: IEG. Note: Excludes projects completed prior to 2004. Note: Excludes projects that began prior to 2003. conditions in the case of loans). Third, IFC addi- plemented by 2008 and were not affected by a tionality is paramount for development effective- major developing-country crisis. Also, IFC AS was ness. Fourth, effective pricing can enhance results. much more limited in scope at the time of previ- Fifth, strong project design, with local implemen- ous crises. However, we do have evidence from tation, is fundamental. Sixth, M&E is not an after- such episodes. In some prior crises, IFC AS was thought; it matters in enhancing results prospects. paired with investments. IFC's banking invest- ments, for example, were often accompanied by At the same time, where associations are absent, extensive AS programs. Their goal was to help implications would seem to include: better defi- the banks implement a reengineering and cor- nition of product maturity; there is no inherent rective action program, upgrade their practices, trade-off in increasing operations in IDA countries systems, and technologies to international stan- and development effectiveness; recent organiza- dards, and improve their internal audit functions tional change has not adversely affected results in and management information systems. One les- the short term, but does not appear to have im- son learned in that experience was the impor- proved them either. Further evaluation will de- tance of determining the true level of client termine whether the benefits of these changes commitment to improving corporate practices, take longer to accrue. In any event, IFC will need although this may be difficult to assess in a crisis to carefully manage the tension between any fur- situation. In Russia, for example, an IFC AS pro- ther organizational changes and more business gram was implemented under the auspices of the growth. Bank's Financial Sector Development Project. The program was expected to result in considerable Are there any implications for IFC's crisis re- transfer of technology and international best prac- sponse? The 458 reviewed operations were im- tices to a Russian-owned operation, aimed at in- 60 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S Box 3.5. Illustrations of IFC Additionality Below are some illustrations of different kinds of IFC fits . . . Their evidence convinced skeptics that better additionality: companies lead to better societies." Global knowledge/best practice: IFC shared its cor- Combined services: IFC initially provided the client com- porate governance expertise by advising on the for- pany with a study on how it could develop its operations mation of an Institute of Directors in a southern African in a new marketplace. IFC followed up this advice with country. In its first year, the institute attracted over 100 a loan, an environmental and social assessment, as members, and is now operating on a sustainable basis. well as specific assistance for managing HIV/AIDS in According to the client, IFC "provided examples illus- the workplace. Since the initial intervention, the com- trating how countries that adopted strong corporate pany has more than doubled the number of people it em- governance laws and supported companies' efforts to ploys, has enhanced environmental and social practices, implement these reforms resulted in economic bene- and is looking at the possibility of further expansion. creasing its efficiency, improving service to clients, IFC Additionality and helping to develop local managers and staff. Various evaluative sources (client interviews, client The advisory services program was not success- and donor surveys, and project evaluations) point ful, however, because the Russian bank lacked to the following as possible IFC additionalities vis- true commitment. It undertook the program more à-vis most commercial knowledge service providers so to give IFC the assurances required to obtain in developing countries: i) global knowledge/ best loan financing from the Corporation. practice awareness; ii) technical expertise in a certain business or product line (e.g., INF public- We can also observe the level of performance of private partnerships, ESS, and corporate gover- AS projects in IFC's crisis priority areas relative to nance); iii) neutral broker/convener/advocate role; other areas. IFC has so far outlined three AS pri- iv) combination of AS and IS; v) having an invest- orities in tackling the current crisis: i) in the fi- ment perspective; vi) ties to the Bank, in particu- nancial sector, helping financial institutions assess lar for macroeconomic policy capacity; and vii) and quantify critical risks, and taking action to mit- IFC's brand, or reputation. (Box 3.5 offers some igate crisis impact, while scaling up programs to illustration of IFC additionality.) strengthen financial infrastructure and disseminate good practice; ii) for the business enabling envi- The global 2008 survey of IFC clients showed that ronment, expanding advice on regulatory sim- the Corporation tends to face its greatest "com- plification, including assistance on the Doing petition" from other development institutions. Business reform agenda, trade logistics, and busi- The survey found that IFC would be the service ness tax reform, insolvency, and investor aftercare; provider that clients would turn to about half the and iii) vis-à-vis corporate governance, improving time, when compared with other options, such as the competence of the boards of directors of cor- a domestic development institution/government porations in emerging markets through targeted program, international consulting firm, internal re- training. When comparing the ratings of projects sources, private equity investor/commercial bank, in these areas--where data are available--with a university, domestic consulting firm, or other (all projects in other areas, there is no statistical dif- less than 10 percent). This stresses the impor- ference in performance between the two groups. tance of IFC understanding the approaches and This may suggest that IFC needs to do a better job activities of other organizations, and its own rel- delivering these products going forward. ative strengths and weaknesses, so that IFC AS is 61 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Box 3.6. How Is IFC Additionality Rated? The PCR system captures IFC additionality with the Satisfactory: IFC's role and contribution was consistent Role and Contribution indicator, which considers to what with its operating principles, making contributions that extent IFC was additional or provided a special contri- may otherwise not have been readily delivered. bution to the project. As with development effectiveness ratings, IFC staff first provide their own self-assessment, Partly unsatisfactory: IFC's role and contribution was not which IEG verifies through desk and field validation. The significant or fell short in one important area. rating criteria are as follows: Unsatisfactory: IFC's role was not plausibly additional, Excellent: IFC played an essential role and made major and IFC's expected contribution was not delivered. contributions to make the project particularly catalytic, innovative, or developmental. most complementary with that of other develop- choose a cheaper source of finance), how can we ment institutions across different environments. determine whether IFC delivered additionality in its AS? And with what impact? At the project level, In most cases, other development institutions can IEG has found, ex post, IFC's role and contribu- offer similar additionalities, and in some cases, tion to be satisfactory in most cases--making con- commercial knowledge service providers can too, tributions that may otherwise not have been such as best practice awareness and certain tech- delivered. IFC was judged to have played an es- nical skills. Relative to other development insti- sential role in only 14 percent of cases, and either tutions, IFC does appear to have an edge in terms played an insignificant role or was not plausibly ad- of diagnostics. The IFC, along with other Bank ditional in 17 percent of cases. Most of the time Group members, has been a leader in developing (69 percent of cases), IFC role and contribution quantitative indicators of the quality of the in- were rated as satisfactory (see box 3.6 for rating vestment climate, and the ability of firms to access criteria). Achieving the highest level of IFC addi- finance. These efforts have been appreciated by tionality is crucial, not only in ensuring that IFC client countries and are used by other donors as does not crowd out commercial providers but well. At the same time, IFC does not appear to have also in enhancing impact. As figure 3.24 showed, a comparative advantage in macroeconomic pol- the effect on development effectiveness of excel- icy (IBRD/IDA, IMF, and some of the regional de- lent role and contribution, rather than satisfactory, velopment banks have greater analytical capacity and especially unsatisfactory role and contribution, and more appropriate instruments); some meso- is significant. By country type, additionality was level interventions, in particular, institutional de- rated higher for projects carried out in frontier velopment, for which the regional development countries, which probably goes some way toward banks may have a greater understanding of coun- explaining better project results in high-risk coun- try context and better partnerships with clients; and tries, but was not any stronger for "developed" longer-term capacity building, which many bilat- products (e.g., business simplification; PPI advisory eral donors are better able to provide. Combining mandates), providing further evidence that prod- AS with lending operations is also an advantage uct classifications might need to be revisited. shared with the EBRD and European Investment Bank (in Europe), and the IDB's Inter-American Evaluation highlights the value of IFC taking a Investment Corporation (in Latin America and the programmatic approach to its AS interventions. Caribbean). In the Middle East and North Africa region, for ex- ample, IFC worked with a number of countries in Since there is no market test of value, as there is developing their national corporate governance with most investments (where the client can codes. IFC ran workshops, covering all aspects of 62 P E R F O R M A N C E O F I F C A DV I S O RY S E RV I C E S code preparation from content development and emerge from independent evaluations that have implementation to monitoring adoption, and also been carried out of EBRD, ADB, the European provided postworkshop advice to assist in draft- Commission and IBRD/IDA AS activities. The find- ing the codes. The program began with the in- ings are broadly consistent with IFC experiences tention of contributing to the drafting of three discussed in this report: i) broader and more national codes. Ultimately, nine codes were drafted sustainable results are obtained from interven- and passed in six countries, with another five in tions at the macro and meso level rather than the the process of being adopted. The program took micro level (firm-level support is low in outreach, on its own momentum, driven by a reputation for which makes it difficult to achieve broader PSD technical expertise and professionalism, as well as impacts beyond the beneficiary firms); ii) inter- sensitivity to local needs and conditions. In an- ventions at all levels need to be targeted at local other case, well-sequenced AS and investment market deficiencies, identified by an assessment activities helped develop the housing mortgage of the actual conditions in the field (some progress market in Russia.59 has been made in developing tools for assessing the business environment, but more needs to Comparing IFC AS: Results of Others Delivering be done to develop methodologies for assessing Knowledge Services the quality of institutions and the functioning of At present, there are no international good prac- markets); iii) interventions to improve the busi- tice standards for the evaluation of PSD-related AS, ness environment should be encouraged, as long and M&E systems are generally not as advanced as as there is sufficient government commitment the one in IFC (notwithstanding the implemen- (support to intermediary organizations can be a tation issues discussed above). As a consequence, way of influencing public policy for the private it is currently not possible to directly compare sector); iv) long- or short-term support within IFC's AS performance with that of other organi- broader programs, leads to better and more sus- zations. Given that IFC's M&E system is generally tainable outcomes; v) despite the fact that there more advanced than that of other development in- is no one-size-fits-all approach to PSD interven- stitutions, at least in principle, IFC is well placed tions, it is important to adopt a methodical pro- to lead efforts to improve and harmonize M&E cedure for selecting areas of intervention in a standards--for instance, through further elabo- country, including: a critical assessment of the ration of the Common Performance Assessment priority areas of interventions, selecting an area System, established in 2005 by six MDBs (includ- in which the donor has a comparative advan- ing IFC) to report performance on a range of key tage, and an assessment of whether the pre- performance indicators. And this year, it is con- conditions for intervening in a given area have sidering PSD AS indicators for the first time.60 been met; vi) client ownership, involvement of Meanwhile, IEG is working with other MDBs to de- local actors, and building of institutions in re- velop good practice standards for PSD AS evalua- cipient countries on the basis of the transfer of tion in the PSD Evaluation Cooperation Group. regulatory, facilitation, and intermediation com- petencies is a necessary condition for sustain- While direct comparisons of performance are ability. (Further details about the findings of these not yet possible, some common lessons do evaluations is provided in appendix J). 63 4 Conclusions and Recommendations T hroughout the developing world, the private sector has been a key con- tributor to growth and poverty reduction in recent years. The current global financial and economic crisis places some of these hard-won gains under threat because of much tighter credit conditions, weaker capital inflows, and reduced developed-country import demand. It has also revealed certain market and nonmarket failures and imperfections. In response to the crisis, development institutions fects ratings remained weak, reflecting contin- can play important financial and nonfinancial ued client and IFC weaknesses. Bank Group im- roles. These include providing finance to viable pact in these regions will be vital in the coming enterprises where it is now lacking (sending pos- years. Environmental and social impact will be itive signals to other investors), acting as an hon- critical in view of the mounting difficulties in est broker in financial restructurings, and offering these areas. advice that helps address institutional weaknesses, for instance, with regard to effective regulation and Stronger overall results reflected several factors: good governance. This report examined IFC's i) the exit of a particularly weak performing co- experiences in financing development (Part I) hort of projects that matured in 2005 (51 percent and providing knowledge for development (Part of projects maturing in 2005 realized high devel- II), with a view to informing IFC's future strate- opment outcomes, compared with 75 percent gic and operational directions, including its re- maturing in 2008); ii) more favorable economic sponse to the current global crisis. conditions in much of the developing world (until late 2008, by which time most evaluated projects Conclusions had been substantially implemented); iii) im- proving IFC project appraisal and structuring Financing Development quality; iv) the conscious move by IFC toward Concerning IFC's efforts to finance development, larger projects, which have been more likely to the review found that project development re- achieve higher ratings than smaller projects, due sults (along with IFC financial returns) improved in part to greater internal scrutiny; and v) espe- overall between 2006 and 2008. However, per- cially strong performance in Europe and Central formance in Africa, Asia, and Middle East and Asia and in Latin America and the Caribbean, North Africa, and in nontelecommunications IT, where the majority of mature operations are lo- continued to lag. FI environmental and social ef- cated. In these regions, business conditions were 65 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 most supportive and IFC work quality strongest. development terms. Such projects represent South Asia exhibited improving performance, about 40 percent of IFC's outstanding portfolio with higher IFC work quality than in the past. (62 percent by volume), thus downside risk to IFC's development "return" is substantial. Going Performance lagged considerably in East Asia and forward, strong IFC work quality and additional- the Pacific, Middle East and North Africa, and Sub- ity will be required (e.g., in making well-timed, cat- Saharan Africa--with barely half of projects in alytic, new investments; providing corporate these regions meeting or exceeding specified finance; acting as an honest broker in restruc- benchmarks and standards. External conditions turings; and helping to improve governance and played some role--projects in Sub-Saharan Africa regulation). and Middle East and North Africa generally fea- tured high levels of country, sponsor, and prod- Knowledge for Development uct competitiveness risks--but the quality of IFC's IFC AS have been growing rapidly--so much so work and contribution to the project tended to that AS teams now dominate IFC's presence on the have a larger impact. This was especially the case ground. This rapid growth has happened in largely , in EAP where nearly 40 percent of projects ex- uncontrolled manner, and raises some important hibited low IFC additionality. strategic questions. These include whether, in grafting a consulting business onto a bank, IFC has Among IFC's strategic sectors, project perfor- the right balance between AS and investment op- mance showed continued improvement in health erations; possible quality trade-offs, given sub- and education, better performance in agribusiness, stantial organizational change and a high reliance continued strong performance in infrastructure on short-term consultants; and an increased pos- and financial markets, and lagged performance in sibility of COI and market distortion (where AS is nontelecommunications IT (software and Inter- offered together with financing, and is provided net).1 In other sectors, such as oil, gas, mining, and at less than market value). IFC has taken steps to chemicals, projects achieved relatively poor rat- improve the organizational alignment and deliv- ings. Risk exposure was a clear factor in weak ery of its AS, but more needs to be done to im- nontelecommunications IT projects, most of prove internal focus and accountability, and to which were small operations involving inexperi- complement better the efforts of others. enced sponsors and unclear product competi- tiveness. However, work quality was also well AS delivery quality reflects client commitment, below par--rated high in just 40 percent of cases. effective project design, and implementation (in- Strong IFC work quality was in evidence in the cluding getting the right global/local and in-house/ health sector, where the Corporation showed consultant staffing mix), M&E, and collaboration that it had learned lessons from past experience, with others. While IFC's approach to delivery although the portfolio had less diversity than en- compares well to that of other MDBs, there are visaged. In oil, gas, mining, and chemicals, proj- also substantial gaps that need to be addressed-- ects did not meet benchmarks for a number of particularly in matching corporate intent with reasons: technical weaknesses of the sponsor; consistent implementation in the field. This also higher than expected asset acquisition cost, and applies with respect to the execution of the pric- in one case, poor environmental compliance. En- ing policy and provision of reliable M&E data, as vironmental and social effects ratings were stable well as ensuring good quality project design and for real-sector projects, but remained weak in FI implementation, and effective collaboration with operations, reflecting a need to focus on strength- other actors, including the Bank. Getting the right ening client capacity and securing commitment, staffing mix has been a particular challenge, with while addressing shortfalls in IFC additionality. a heavy reliance on relatively new staff and ex- ternal, short-term consultants. Such dependence Given the current global financial crisis (an ex- has considerable implications for the quality and treme exogenous risk), projects in early imple- continuity of IFC AS, and preservation of global mentation are expected to be hardest hit, in knowledge leadership. 66 C O N C L U S I O N S A N D R E C O M M E N D AT I O N S Available results data suggest better performance critical importance of sustained development of in Southern Europe and Central Asia, weaker per- the private sector, for which regulatory frame- formance in Latin America and the Caribbean works are important and excessive deregulation (prior to a recent reorganization) and for global costly. In these circumstances, this review pro- projects; and a strong association between coun- vides further findings on what IFC might do to en- try conditions (including the pursuit of AS activ- hance development effectiveness and additionality: ities in high-risk countries), delivery quality, and results. Additionality is fundamental for better Operations during the Crisis: performance, and may be enhanced by some-- Effectively manage the tension between pro- though not all--combinations with IFC invest- tecting the portfolio and responding to op- ments (e.g., better ratings when combined with portunities during crisis. In the past, this loans, and for second generation linkage proj- tension has not always been managed adequately ects in agribusiness and manufacturing). More and IFC has missed opportunities to have a deeper benchmarking may be helpful. At all stages of de- impact. Experience suggests the importance of livery, M&E data provided by staff and consultants arrangements to isolate portfolio problems from (in particular) has remained unreliable. Relatedly, new business development, mitigating conflicts of IFC-commissioned reviews of AS facilities, prod- interest that may impede effective collaboration ucts, and projects, while offering insights on the with the Bank and the IMF, and of clear rules of organization and delivery of AS, have exhibited engagement for crisis response, particularly for shortcomings in independence and design. staff in the field. Experience also indicates the important role IFC and the Bank Group must Charging effectively for IFC's advisory services is play in promoting sound frameworks for prudent perhaps the most important step going forward. financial risk management and safeguards for sus- Effectively charging clients for service will introduce tainable private sector development. This is es- a market test for AS, and is likely to have a posi- pecially relevant today, as the world reexamines tive impact on all aspects of the AS business: in cre- the roles of governments and markets in the wake ating incentives for greater client buy-in, stronger of the financial crisis. project design and implementation, stronger M&E, development of products that best meet demand, IFC Advisory Services: and ensuring IFC additionality. In the immediate Set out an overall strategy for IFC advisory term, IFC would need to strictly implement the cur- services, addressing the need for a clear rent pricing policy, which is largely cost-based vision and business framework that is more (i.e., the price the client is expected to pay is a pro- closely linked with IFC's global corporate portion of the cost of the project, rather than its strategy. Following years of unchecked growth value per se). Over time, efforts should be made and recent organizational changes, the role of to move to a market value-based approach for AS in IFC's business model needs to be addressed. pricing, to make sure that IFC does not run the risk The strategy would need to better articulate of crowding out other knowledge providers. IFC IFC comparative advantages, objectives, and goals investments are priced according to this principle for AS in different contexts (a source of confu- for the same reason. The current economic crisis, sion among staff), and to consider the best staff- and its likely effects on donor and IFC funding, is ing combinations (with respect to internal or an opportunity for the Corporation to push harder external, as well as global or local staff), delivery in the direction of value-based pricing, and to en- unit organization, incentives, and performance courage other development institutions to do measures to help realize these objectives and likewise. goals. Recommendations Pursue more programmatic AS interven- This review comes at a time of deep distress in fi- tions. Evaluation shows that IFC achieved better nancial markets and of severe downsizing in pri- results in AS projects that were carried out in vate economic activities. This reminds us of the conjunction with other AS interventions. One- 67 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 off activities have been less effective. However, project-completion follow-up, better lessons- programmatic efforts of this kind have been in the capture (including from dropped or terminated minority (about a fifth of all AS projects), and IFC projects), and more arms-length facility, product, should accordingly seek to expand this type of and project reviews. In the medium term, it would intervention. pay to introduce an Expanded Project Completion Report system (akin to the Expanded Project Improve execution of the AS pricing policy Supervision Report system for investment oper- through greater client contributions. Over the ations, and carried out later than the PCR to bet- longer term, it would be important to seek client ter capture impacts), more programmatic impact contributions that reflect value and impact (i.e., evaluation and impact research, setting results- not just cost, to create a true test of client demand, based targets for AS in its corporate scorecard, and incentives for better AS delivery, and to ensure IFC regular benchmarking of IFC AS activities and is being additional). systems with other providers of knowledge serv- ices, including other MDBs and commercial Strengthen AS performance measurement providers. In the longer term, the aim could be and internal knowledge management. In to establish a specialized research unit focusing the short term, it would be important to have on generating and bringing together private sec- more hands-on M&E support in the field, after- tor development knowledge work. 68 APPENDIXES APPENDIX A: PROJECT SAMPLE REPRESENTATIVENESS-- INVESTMENT OPERATIONS Table A.1. Representativeness of the 2006­08 XPSR Sample (compared with 2001­03 net approvals population) Number of Investments Value of Investments ($ million) CY2006­08 XPSRs CY2001­03 NAP (c) = CY2006­08 XPSRs CY2001­03 NAP (c) = (a) (b) (a)/(b) (a) (b) (a)/(b) No. % No. % % Amt. % Amt. % % 178 100 349 100 51 3679 100 7340 100 50 Net IFC: Mean ­ ­ ­ ­ ­ 21 ­ 21 ­ ­ Median ­ ­ ­ ­ ­ 12 ­ 12 ­ ­ Investment size: X = <4.04 36 20 72 21 50 70 2 156 2 45 4.04 < X= < 38.02 114 64 220 63 52 1700 46 3399 46 50 X > 38.02 28 16 57 16 49 1909 52 3785 52 50 178 100 349 100 51 3679 100 7340 100 50 Instruments: Equity only 42 24 83 24 51 568 15 982 13 58 Other 136 76 266 76 51 3111 85 6358 87 49 178 100 349 100 51 3679 100 7340 100 50 Sectors: Financial markets 73 41 144 41 51 1760 48 3072 42 57 Nonfinancial markets 105 59 205 59 51 1919 52 4268 58 45 178 100 349 100 51 3679 100 7340 100 50 Departments: Agribusiness 11 6 22 6 50 244 7 458 6 53 Global Financial Markets Group 60 34 119 34 50 1420 39 2438 33 58 Global Inform. & Comm. Tech. 15 8 30 9 50 236 6 687 9 34 Global Manufacturing & Services 48 27 94 27 51 830 23 1676 23 50 Health and Education 6 3 11 3 55 57 2 77 1 74 Infrastructure 18 10 35 10 51 409 11 1021 14 40 Oil, Gas, Mining, and Chemicals 9 5 18 5 50 194 5 429 6 45 Private Equity and Investment Funds 11 6 20 6 55 290 8 554 8 52 178 100 349 100 51 3679 100 7340 100 50 (Table continues next page) 71 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table A.1. Representativeness of the 2006­08 XPSR Sample (continued) (compared with 2001­03 net approvals population) Number of Investments Value of Investments ($ million) CY2006­08 XPSRs CY2001­03 NAP (c) = CY2006­08 XPSRs CY2001­03 NAP (c) = (a) (b) (a)/(b) (a) (b) (a)/(b) No. % No. % % Amt. % Amt. % % Regions: Africa 15 8 32 9 47 106 3 533 7 20 Asia 46 26 91 26 51 861 23 1895 26 45 Europe & Central Asia 54 30 99 28 55 1102 30 1764 24 62 Latin America and the Caribbean 48 27 97 28 49 1450 39 2753 38 53 Middle East and North Africa 12 7 23 7 52 126 3 275 4 46 World 3 2 7 2 43 35 1 121 178 100 349 100 51 3679 100 7340 100 50 Active/closed Active 104 58 199 57 52 2502 68 4400 60 57 Closed 74 42 150 43 49 1177 32 2940 40 40 178 100 349 100 51 3679 100 7340 100 50 Indicative performance: (as of 06/30/2008) (i) All investments: a With loss reserves 3 2 6 2 50 19 1 33 0 58 Without loss reserves 175 98 343 98 51 3660 99 7307 100 50 178 100 349 100 51 3679 100 7340 100 50 (ii) Equity only: a With loss reserves 0 0 0 0 ­ 0 0 0 0 ­ Without loss reserves 42 100 83 100 51 568 100 982 100 58 42 100 83 100 51 568 100 982 100 58 Countries (excluding regional): 58 74 Source: IEG. Note: XPSR = Expanded Project Supervision Report, CY = calendar year, NAP = net approvals population. Columns showing percentages may not add up to 100 due to rounding. a. Amounts with loss reserves are the IFC approved investments that are affected by loss reserves (and not the actual amount reserved). 72 APPENDIX B: PROJECT EVALUATION METHODOLOGY-- INVESTMENT OPERATIONS IEG's project evaluation ratings are based on the included in the financial market operations' Expanded Project Supervision Report (XPSR) sys- economic sustainability analysis. tem. Introduced in IFC in 1996, the XPSR process · Environmental and social effects: IFC's first involves a self-evaluation of the project by Policy and Performance Standards on Social & an IFC investment department, using corporate Environmental Sustainability (2006) consider guidelines. The ratings assigned by investment de- social and environmental sustainability to be partments are then independently verified (or an important component of development out- re-rated) by IEG in terms of bottom-line outcome come quality in the IFC-financed projects. The ratings and their respective subcomponents. XPSR's assessment of environmental and so- cial effects should cover: (i) the project's en- The development outcome rating is a bottom- vironmental performance in meeting IFC's line assessment, not an arithmetic average, of requirements; and (ii) the project's actual en- the project's results across four development di- vironmental impacts (through sub-projects in mensions, relative to what would have occurred the case of financial market operations), in- without the project. It measures a project's bus- cluding pollution loads, conservation of biodi- iness success, economic sustainability, environ- versity, and natural resources. More broadly, it ment and social effects, and private sector devel- should also cover social, cultural, and commu- opment impacts. nity health aspects, as well as labor and work- ing conditions and workers' health and safety. · Project business success: In financial mar- · Private sector development impacts (be- ket operations, project business performance yond the project): IFC's Purpose, specified in measures the project's long-term impact on Article I, is "encouraging the growth of pro- the financial intermediary's profitability and ductive private enterprises," and to that end, viability, using such indicators as capital ade- IFC shall "seek to stimulate, and to help create quacy, asset quality, management quality, earn- conditions conducive to the flow of private ings performance, and liquidity structure and capital, domestic and foreign, into productive balance sheet. In nonfinancial market oper- investment." This indicator addresses to what ations, project business performance measures extent the company has developed into a cor- the project's actual and projected financial porate role model--positive or negative--and impact on the company's financiers, that is, whether the project has contributed to IFC's lenders and equity investors. The principal in- purpose by spreading the growth benefits of dicator of a project's business performance is productive private enterprise beyond the proj- its real, after-tax, financial rate of return. ect company. · Economic sustainability: In nonfinancial market operations, this indicator evaluates the IFC's investment outcome rating is an assess- project's effects on the local economy, and on ment of the gross profit contribution quality of an the associated benefits and costs that are mea- IFC loan and/or equity investment, that is, with- sured by economic rates of return. In addition out taking into account transaction costs or the to the project's effects, subprojects' effects are cost of IFC equity capital. 73 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 · Gross contribution-loan: The primary in- of IFC's funding. This helps measure to what dicator for this rating is whether the borrower extent IFC has professionally executed its is current on its payments to IFC (interest, supervision. IFC's Operational Procedures pro- fees, etc.). It is also important to assess the vide guidance on what IFC considers an ap- likely future debt-servicing capacity of the client. propriate professional standard. · Gross contribution-equity: The nominal, · IFC's role & contribution: This is measured equity, internal rate of return (also called return by how well IFC fulfilled its role in terms of three on equity or ROE). The rating criteria for eq- basic operating principles: (i) Additionality/ uity investments are based on a comparison of Special Contribution Principle--"IFC should the nominal, equity, internal rate of return with participate in an investment only when it can the actual (or notional) fixed-rate, loan inter- make a special contribution not offered or est rate (FR) that was (or would have been) ap- brought to the deal by other investors"; (ii) Busi- proved for the same. ness Principle--"IFC will function like a business in partnership with the private sector and take The assessment of IFC work quality involves a the same commercial risks"; and (iii) Catalytic judgment about the overall quality of IFC's due Principle--"IFC will seek above all to be a cata- diligence and value added at each stage of the lyst in facilitating private investors and markets operation. It measures the IFC's performance in in making good investments." screening, appraisal, structuring, supervision and administration, as well as its role and contribution. For each of the above principles, a four-point rat- ing scale is used (excellent, satisfactory, partly un- · Screening, appraisal, and structuring: satisfactory, and unsatisfactory), except for the This measures the extent to which IFC profes- synthesis development outcome rating, which in- sionally executed its front-end work toward a sus- volves a six-point scale (highly successful, suc- tainable corporate performance standard. IFC's cessful, mostly successful, mostly unsuccessful, operating policies and procedures, as well as its unsuccessful, and highly unsuccessful). In IEG's bi- credit notes provide guidance on what IFC con- nary analysis, "high" refers to satisfactory or bet- siders an appropriate professional standard. ter on the four-point scale, and mostly successful · Supervision and administration: Super- or better on the six-point scale. Specific rating cri- vision, for this purpose, starts after commitment teria for each indicator are set out in table B.1. 74 Table B.1. Project Performance Indicators and Rating Criteria for IFC Investment Operations Rating Excellent Satisfactory Partly satisfactory Unsatisfactory PROJECT DEVELOPMENT OUTCOME: Project business Real sector: FRR > = WACC + 2.5% Real sector: FRR > = WACC Real sector: FRR > = WACC ­ 2% Real sector: FRR < WACC ­ 2% success Financial market: Project substan- Financial market: Project had a neutral Financial market: Project returns were Financial market: Project returns were A P P E N D I X B : P R O J E C T E V A L U AT I O N M E T H O D O L O G Y -- I N V E S T M E N T O P E R AT I O N S tially raised the FI's profitability and to positive effect on profitability and sufficient to cover cost of associated insufficient to cover cost of associated substantially improved its viability improved viability (targeted funding); debt, but did not provide adequate re- debt or harmed viability (targeted (targeted funding); High overall prof- adequate overall profitability expected turns to equity holders or detracted funding); Expected long-run returns to itability of the FI expected in the (general funding) from viability (targeted funding); Ex- equity holders less than cost of debt case of newly established FIs (general pected long-run returns to equity hold- financing (general funding) funding) ers do not provide a risk premium over the cost of debt financing (general funding) Economic Real sector: ERR > = 20% Real sector: ERR > = 10% Real sector: ERR > = 5% Real sector: ERR < 5% sustainability Financial market: Project substantially Financial market: Project positively in- Financial market: Project made no pos- Financial market: Project negatively increased the efficiency of financial fluenced the efficiency of financial mar- itive contribution to the efficiency of affected living standards or the effi- markets and/or the vast majority of kets and/or most of the subprojects are financial markets and/or a large por- ciency of financial markets and/or subprojects are economically viable economically viable as defined by: tion of the subprojects is not economi- the majority of subprojects are not and the project has made a substan- (a) subborrower portfolio quality is cally viable as defined by: (a) sub- economically viable as defined by: tial and widespread contribution to better than, or equal to, the higher of borrower portfolio quality is worse (a) subborrower portfolio quality is improving living standards. the rest of the FI's loan portfolio or than the higher of the rest of the FI's worse than both the rest of the FI's the market average; (b) the aggregate loan portfolio or the market average; loan portfolio and the market average; equity fund portfolio return before or (b) more than half of equity fund in- or (b) the aggregate equity fund portfo- management fees is satisfactory; or vestees have zero or negative equity lio return before management fees is (c) more than half of equity fund returns, while aggregate portfolio re- negative. investees have positive equity returns turn before management fees is less while aggregate portfolio return be- than satisfactory but no less than zero. fore management fees is less than satisfactory but no less than zero. Environmental Real sector: The project either: Real sector: The project is--and was Real sector: The project is not in Real sector: The project is not in and social (i) maintained the company's excellent over its lifetime--in material compli- material compliance with either IFC's material compliance with either IFC's sustainability environmental management or materi- ance with either IFC's current or at- current or at-approval requirements, current or at-approval requirements, ally improved the company's overall approval requirements, including World but deficiencies are being addressed and mitigation prospects are uncertain environmental performance (e.g., Bank Group environmental, health and through ongoing and/or planned or unlikely; or earlier noncompliance through training and addressing envi- safety policies and guidelines. actions; or earlier noncompliance (meanwhile corrected) resulted in sub- ronmental, social, cultural, and com- Financial market: The project meets (meanwhile corrected) resulted in stantial and permanent environmental munity aspects, as well as labor and either IFC's at-approval requirements or environmental damage. damage. 75 (Table continues next page) 76 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table B.1. Project Performance Indicators and Rating Criteria for IFC Investment Operations (continued) Rating Excellent Satisfactory Partly satisfactory Unsatisfactory PROJECT DEVELOPMENT OUTCOME: (continued) Environmental working conditions, or introducing an IFC's current requirements and its en- Financial market: The project does not Financial market: The project does not and social Environmental Management System vironmental effects are deemed ac- meet IFC's requirements, but the meet IFC's requirements and substan- sustainability (EMS) or corporate program for envi- ceptable overall. For all FI project shortfalls are either being corrected or tial negative effects are known or (continued) ronmental and social responsibility types, trained staff implement an ap- negative impacts are moderate. For likely, (e.g., the FI's EMS is completely broader than IFC's requirements; or propriate EMS that has been function- example: the FI's EMS is adequate, inadequate and nothing is known (ii) raised the environmental perfor- ing over the project life (as reflected but some subprojects have resulted about subproject performance); the mance of local companies (e.g., by also in acceptable environmental stan- in environmental damage; or the EMS has material shortcomings and raising industry standards and serving dards being applied to projects fi- subprojects visited have acceptable some subprojects have negative envi- as a good practice example for regula- nanced by the FI). The subprojects are environmental standards, but the ronmental effects; while the EMS ap- tors). In addition, the project consis- and have been in substantial material EMS is materially inadequate; or an pears adequate, a significant portion tently met IFC's at-approval require- compliance with IFC's requirements for FI (type 1) initially had no EMS, but of subprojects have negative environ- ments and environmental effects are the life of the project. has recently introduced a functioning mental effects; some subprojects have deemed acceptable in view of IFC's EMS. resulted in substantial and irreversible current requirements. IFC should be environmental damage. able to use projects rated excellent as role models for positive environmental effects. Financial market: The project main- tained the FI's excellent Environmental Management System (EMS) or materi- ally improved the efficacy of the FI's overall environmental risk manage- ment (e.g., through training and intro- duction of a well-functioning EMS) and the environmental performance of portfolio companies. In addition, the FI has provided transparent and detailed reports on time, verifying that the proj- ect (and subprojects, as applicable) has consistently met IFC's require- ments at approval and its environmen- tal effects are deemed acceptable in view of IFC's current requirements. IFC should be able to use projects rated excellent as role models for positive environmental effects. Table B.1. Project Performance Indicators and Rating Criteria for IFC Investment Operations (continued) Rating Excellent Satisfactory Partly satisfactory Unsatisfactory PROJECT DEVELOPMENT OUTCOME: (continued) Private sector All sectors: Considering its size, the All sectors: The project had some, but All sectors: The project had mostly All sectors: Substantial negative development project improved the enabling environ- no major positive impacts. negative impacts, which, however, are impacts of broad applicability and/or A P P E N D I X B : P R O J E C T E V A L U AT I O N M E T H O D O L O G Y -- I N V E S T M E N T O P E R AT I O N S impacts ment or otherwise made a substantial not expected to be of long duration or expected to be of long duration. (beyond project) contribution to the growth of private broad applicability (e.g., a failed proj- enterprises or efficient financial ect without substantial negative markets. demonstration effects). IFC INVESTMENT OUTCOME: Loan Fully performing and, through a sweet- (i) loan expected to be paid as sched- Loan has been rescheduled, or guaran- (i) loan is in nonaccrual status; or ener (e.g., income participation), it is uled; or (ii) loan is prepaid in full; or tee is called and in either case IFC (ii) IFC has established specific loss expected to earn significantly more (iii) loan has been rescheduled and is expects to receive sufficient interest reserves; or (iii) loan has been re- than a loan priced "without sweet- expected to be paid as rescheduled income to recover all of its funding scheduled but IFC does not expect to ener" would have earned if paid as with no loss of originally expected in- cost but less than the full dollar mar- recover at least 100% of its loan fund- scheduled. There is no indication that come. In the case of an IFC guarantee, gin originally expected. If all payments ing cost; or (iv) loan has been or is debt service payments will not remain all fees are expected to be received, to IFC are current, but there is doubt expected to be wholly or partially con- current in future. and guarantee not called (or called but whether payments can remain current verted to equity in restructuring of a expected to be fully repaid in accor- in future, then a partly unsatisfactory "problem" project; or (v) IFC experi- dance with the terms of the guarantee rating may be preferable. For example, ences a loss on its guarantee or risk- agreement). In the case of an IFC swap IFC may establish "flag" loss reserves management facility. or other risk-management facility, IFC of modest size (no more than 10%) for has not suffered any loss and expects reasons such as country conditions, no loss due to nonperformance of the which are not related specifically to swap counterparty. There is no indica- IFC's project. In these cases, a partly tion that debt service payments to IFC unsatisfactory rating may be used will not remain current in future. rather than unsatisfactory. Equity Benchmarks vary, depending on the nature of the equity investment. (Table continues next page) 77 78 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table B.1. Project Performance Indicators and Rating Criteria for IFC Investment Operations (continued) Rating Excellent Satisfactory Partly satisfactory Unsatisfactory IFC WORK QUALITY: Structuring, IFC's front-end work could serve as a Materially met IFC's good practice Material shortfall in at least one Material shortfalls in several areas appraisal, and best-practice example. standards. important area. or a glaring mistake or omission screening bordering on negligence in at least one important area. Supervision and IFC has always kept itself promptly IFC has kept itself sufficiently IFC's supervision was insufficient to IFC missed material developments, administration and fully informed about the project's informed to react in a timely manner monitor the project's and FI's perfor- and/or did not use information to and FI's performance in all material to any material change in the project's mance and/or IFC did not take timely intervene in a timely and appropriate areas and used this knowledge proac- and FI's performance and took timely and appropriate action. manner. tively to improve the project's develop- action where needed. ment outcome and/or IFC's investment outcome. Role and IFC's role was essential for the project IFC's role and contribution were in IFC's role or contribution fell short in a IFC's role was not plausibly additional contribution to go ahead and IFC made a major line with its operating principles (of material area. and IFC did not deliver its expected contribution to make it a success. being catalytic and making a special contribution. contribution). Source: IEG. Note: ERR = economic rate or return, FRR = financial rate of return, WACC = weighted average cost of capital. APPENDIX C: FURTHER DETAILS ON RESULTS CHARACTERISTICS-- INVESTMENT OPERATIONS Table C.1. Characteristics of Project Ratings, by Subindicators, 2006­08 LOW OUTCOMES HIGH OUTCOMES Unsuccessful unsuccessful unsuccessful Successful successful successful Mostly Highly Highly Mostly Project development DEVELOPMENT OUTCOME 6% 12% 10% 22% 38% 12% outcome ratings, 2006­08 28% 72% 2% 5% 9% 26% 43% 16% (by commitment volume) 15% 85% Partly Unsatisfactory Satisfactory Excellent unsatisfactory Project business success 20% 13% 33% 35% 33% 67% Economic sustainability 13% 9% 43% 34% 23% 77% Environmental effects 10% 27% 56% 8% 37% 63% Private sector development 5% 16% 51% 28% 21% 79% Partly Unsatisfactory Satisfactory Excellent unsatisfactory IFC investment return IFC INVESTMENT OUTCOME 16% 6% 57% 21% ratings, 2006­08 22% 78% (by commitment volume) 6% 4% 59% 31% 10% 90% Loan 4% 4% 83% 9% 8% 92% Equity 35% 7% 17% 41% 42% 58% (Table continues next page) 79 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table C.1. Characteristics of Project Ratings, by Subindicators, 2006­08 (continued) Partly Unsatisfactory Satisfactory Excellent unsatisfactory IFC work quality ratings, IFC'S OVERALL WORK QUALITY 2% 19% 65% 15% 2006­08 21% 79% (by commitment volume) 2% 14% 66% 18% 15% 85% Screening, appraisal, structuring 4% 22% 61% 13% 26% 74% Supervision and administration 1% 13% 70% 16% 14% 86% Role and contribution 2% 15% 53% 30% 17% 83% Source: IEG. Notes: IEG uses a binary interpretation of these evaluation results, which describes operations' ratings as either "high" or "low." By volume, figures are the percentages of the total committed IFC investment amounts in each outcome-rating group. The rates above indicate the percentages of all assigned ratings. Figure C.1. Combined Project Development Outcome and IFC Investment Return Characteristics, 2006­08 Number of operations: 9 Number of operations: 117 Commitments: $110 m (4%) Commitments: $2,384 m (81%) Project business success: 67% Project business success: 91% ESHS effects success rate: 75% ESHS effects success rate: 67% High-risk sponsor: 75% High-risk sponsor: 27% Instrument: ­Loan 0% Instrument: ­Loan 66% ­Equity 100% ­Equity 21% Equity success rate (4 invs.): 0% ­Loan and Equity 7% Equity aggregate real IRR: 7.2% ­Others 4% Work quality: ­High 100% Equity success rate (38 invs.): 97% ­Low 0% Equity aggregate real IRR: 61.6% Country risk: ­Improved 17% 1 Work quality: ­High 93% ­Unchanged 83% 2 ­Low 7% 66% HIGH ­Deteriorated % in strategic sectors (by #): 0% 44% 5% High development Country risk: ­Improved ­Unchanged 67% 33% High development Development outcome outcome ­Deteriorated 0% outcome High IFC return Low IFC return % in strategic sectors (by #): 59% 4 3 Number of operations: Commitments: 30 $186 m (6%) 17% 11% Number of operations: Commitments: 20 $432 m (8%) Project business success: 0% Low development Low development Project business success: 15% ESHS effects success rate: 46% outcome outcome LOW ESHS effects success rate: 47% High-risk sponsor: 67% Low IFC return High IFC return High-risk sponsor: 40% Instrument: ­Loan 30% Instrument: ­Loan 80% ­Equity 57% LOW HIGH ­Equity 0% ­Loan and Equity 7% Investment outcome ­Loan and Equity 5% ­Others 7% ­Guarantee 5% Equity success rate (19 invs.): 0% Equity success rate (3 invs.): 100% Equity aggregate real IRR: ­8.7% Equity aggregate real IRR: 27.6% Work quality: ­High 36% ­Low 64% Work quality: ­High 42% Country risk: ­Improved 48% ­Low 58% ­Unchanged 52% Country risk: ­Improved 50% ­Deteriorated 0% ­Unchanged 50% % in strategic sectors (by #): 43% ­Deteriorated 0% % in strategic sectors (by #): 50% Source: IEG. 80 APPENDIX D: RISK-ADJUSTED EXPECTED DEVELOPMENT OUTCOME REGRESSION: MODEL SPECIFICATION, ANALYSIS, AND PRELIMINARY RESULTS Years of evaluation and econometric testing show try Credit Risk score between approval and that project development results hinge signifi- evaluation. A higher value indicates a larger cantly on two types of factors: those external to improvement in the business environment. IFC--notably, country risk, sponsor risk, and An improving business environment creates product market risk; and those internal to IFC-- more and distributes better investment and the quality of IFC's work in project appraisal and growth opportunities, rewards entrepre- structuring, project supervision, and additional- neurial efforts, facilitates business growth, ity. It is important to note, however, that the so- and therefore is expected to translate into called external factors also come within IFC's more jobs, higher community impacts, and decision-making purview and that there can be in- greater tax revenues. Trends in the business teractions between external and internal factors. environment appear to be more important Distinguishing between the two and, in general than starting levels. assessing the sensitivity of development outcomes (ii) Sponsor/partner quality--the variable cap- to various factors, nevertheless can potentially tures the sponsor's experience, financial help in measuring, understanding, and rewarding capacity, commitment to the project, and performance. In general, risks can be offset by governance/business reputation. If the spon- strong work quality, although project develop- sor is rated low in these dimensions, spon- ment outcomes still tend to be lower when the sor quality is deemed to be low. This factor risk is higher. is rated on a binary scale, with 1 as high risk/low quality and 0 as low risk/high qual- With this understanding, IEG developed an initial ity, based largely on assessment of project model to provide views on project performance documentation and, where available, public that better consider country, sector, and product information and field visits/interviews. IFC is risk context, and thereby enhance understanding delivering development impact through part- about the quality of IFC's efforts in meeting dif- ners, typically private enterprises, and there- ferent development challenges. The conceptual fore their capacity, integrity, and commitment framework views development outcome of a proj- are an important factor of development ect as a function of two sets of factors: external impact. However, IFC's additionality may and internal to IFC (again noting possible inter- be higher when sponsor's quality is not very actions among them). high, in which case IFC's additionality may mitigate the risks arising from low sponsor development outcomeI = quality. The variable is measured as of time f (external factorsI, IFC-controllable of approval. factorsI) + I (iii) Market risks--captures the project's under- lying competitiveness in the market in which The model includes the following external factors: it is operating, and any market distortions, such as high tariff protection, degree of pres- (i) Changes in country business climate-- ence of state-owned enterprises in the sec- changes in the Institutional Investor Coun- tor, artificial monopoly positions, and other 81 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 distortions that typically result in low com- that are in some way highly correlated with fac- petitiveness. Rated on a binary scale with 1 tors and are already included in the model. as high risk/low competitiveness and 0 as oth- erwise. Clearly demonstrated market com- The set of IFC-controllable factors considered in petitiveness improves a venture's ability to the model are as follows: meet business adversity and survive in its early years to reach its development poten- (v) Screening, appraisal, and structuring qual- tial. Economic Rates of Return and devel- ity--rated on a binary scale with 1 as sat- opment impact in general tend to be lower isfactory or better, and 0 as less than in distorted market environments. Distor- satisfactory. tions drive a wedge between market and (vi) Supervision and administration quality-- economic prices, and financial and economic rated on a binary scale with 1 as satisfactory returns of a project, resulting in a divergence or better, and 0 as less than satisfactory. between private and social returns. Distor- (vii) IFC additionality--proxied by IFC's role tions are normally unsustainable over the and contribution, rated on a binary scale long term, creating also financial risks if a par- with 1 as satisfactory and 0 as less than ticular enterprise benefits financially from satisfactory. market distortions. The variable is measured as of time of approval. Table D.1 presents summary statistics. (iv) Project type--Rated on a binary scale with 1 for a greenfield project and 0 otherwise. The external variables in the model are consistent Greenfield projects involve new plant con- with consideration of risk in both the financial and struction and new operations and thus pose development worlds. Financial theorists and prac- higher risks compared to expansions of ex- titioners distinguish between the following main isting plants and operations. They pose "the types of risks: i) Country risk: the risk of loss on greatest challenge to structuring and risk cross-border exposure due to government ac- sharing." 1 tions; ii) Credit risk: the risk of loss due to bor- rower's default; iii) Business risks: uncertainties The model excludes some possible factors, such in the revenues and expenses of a business as- as whether the client is a new client or a repeat sociated with general industry trends, techno- client, IFC sector experience, and project size, logical or regulatory changes; iv) Market risks: Table D.1. Summary Statistics for Key Variables: 2000­05 vs. 2006­08 Average for Average for Direction and 2000­2005 2006­2008 magnitude of change Development outcome success (%) 0.57 0.72 Significant improvement Changes in country business climate 3.13 13.60 Significant improvement Sponsor risk 0.40 0.37 No significant change Market competitiveness 0.68 0.60 Improvement Project type 0.41 0.42 No change Screening, appraisal, & structuring work quality 0.51 0.74 Significant improvement Supervision & administration work quality 0.69 0.86 Significant improvement IFC role and contribution 0.79 0.82 No significant change Number of observations 361 173 Source: IEG. 82 A P P E N D I X D : R I S K - A DJ U S T E D E X P E C T E D D E V E L O P M E N T O UT C O M E R E G R E S S I O N risk of possible losses arising from changes in Table D.2. Determinants of Development Outcome- the market due to fluctuating or changing in- Probit Regression Summary, 2000­08 terest rates, foreign exchange rates, share prices and prices in general. In the development field, df/dx p > |z| risks to development outcome are commonly Changes in country business climate 0.006** 0.011 considered in World Bank approval and evaluation Sponsor risk ­0.09* 0.10 documents. The risks most often identified in Bank project documents are similar to the risk fac- Market risks ­0.14** 0.012 tors included in the model: unfavorable changes Project type ­0.10* 0.07 in policies, or law and order situation; technical Screening, appraisal, & structuring work quality 0.38** 0.000 capacity and commitment of government partners Supervision & administration work quality 0.35** 0.000 and/or the implementing agency. IFC role and contribution 0.55** 0.000 Number of observations 517 Regression results are presented in table D.2. All Pseudo R2 0.444 the coefficients have the expected signs and are sig- Source: IEG. nificant at the 5 percent or 10 percent level. It is Note: Coefficients displayed represent marginal changes in probability of successful de- clear from the results that factors controllable by velopment outcome due to unit change in explanatory variable, which for a discrete change IFC tend to dominate the external factors both in of dummy variable is from 0 to 1; p-values are in the second column; * significant at 10%; ** significant at 5%. terms of statistical significance and statistical impact. We next use the results in table D.2 to estimate ferences between predicted and actual develop- the impacts of risk and IFC-controllable factors on ment outcome success rates are due to unex- development outcomes by regional and industry plained factors. departments. Our point of departure is the real- ization that in an ideal situation of no risks and The results are presented in table D.3 below. As high work quality, the expected development suc- we can see, risks factors had the largest impact on cess rate should be 100 percent.2 We then simu- performance in Sub-Saharan Africa and in Middle late the probability of success by regional and East and North Africa, 12 percent and 11 percent, industry departments with actual risk parame- respectively, almost twice as large an impact as in ters and perfect work quality. This estimate of other regions. The impact of risk factors is less vari- development outcome success rates we call "po- able by industries than by regions. Communica- tential development outcomes" because it in- tions & Information Technologies and Global dicates what could be achieved with high work Financial Markets tend to have higher risk profiles quality, given the actual risk profile of projects un- as reflected in slightly higher development loss dertaken by the respective departments, i.e., po- due to risks taken. tential development outcome = f (actual risks, perfect work quality). The difference between For all departments, except Private Equity & In- the risk-free 100 percent rating and the potential vestment Funds and Health & Education, IFC- development outcome can therefore be attributed controllable factors tend to dominate external to the effect of the degree of risks taken. risk factors in terms of impact on development outcomes. The impact is particularly pronounced From the basic regression in table D.2, we obtain in the case of East Asia and the Pacific and Com- predicted development outcome success rates munications & Information Technologies. It ap- by regional and industry departments, i.e., pre- pears that Health & Education and Private Equity dicted development outcome = f (actual risks, & Investment Funds have achieved high levels of actual work quality). The difference between work quality. It is evident, however, that there is potential development outcome and predicted de- room for improvement in all regions and sectors. velopment outcome would then be due to gaps In addition, in Africa and Middle East and North in work quality. Finally, the residuals, i.e., the dif- Africa, even if we account for risk, the potential 83 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table D.3. IFC's Project Development Outcomes and Factor Attribution Difference between actual and max = 100% Development outcome Due to Potentiala Predictedb Actual Due to risk work quality Unexplained IFC, 2006 92% 65% 66% ­8% ­27% 1% IFC, 2007 94% 73% 72% ­6% ­21% ­1% IFC, 2008 93% 68% 75% ­7% ­25% 7% IFC, 2006­08 93% 69% 72% ­7% ­24% 3% Sub-Saharan Africa (15) 88% 60% 47% ­12% ­28% ­13% East Asia and Pacific (29) 93% 57% 54% ­7% ­36% ­3% South Asia (16) 92% 77% 75% ­8% ­15% ­2% Central and Eastern Europe (24) 95% 82% 84% ­5% ­13% 3% Southern Europe and Central Asia (26) 94% 87% 83% ­6% ­7% ­4% Latin America and the Caribbean (48) 95% 82% 83% ­6% ­13% 1% Middle East and North Africa (12) 89% 68% 50% ­11% ­21% ­18% Oil, Gas, Mining, and Chemicals (9) 93% 69% 56% ­7% ­24% ­13% Communication and Information Technology (15) 92% 61% 47% ­8% ­31% ­14% Infrastructure (18) 96% 88% 94% ­5% ­8% 6% Global Manufacturing Services (47) 93% 75% 63% ­7% ­18% ­12% Global Financial Markets (57) 92% 71% 77% ­8% ­21% 6% Food and Agribusiness (10) 96% 79% 78% ­4% ­17% ­1% Private Equity and Investment Funds (11) 94% 91% 73% ­6% ­3% ­18% Health and Education (6) 93% 93% 100% ­7% 0% 7% Source: IEG. Note: Numbers in parentheses indicate number of projects. a. Potential is Risk-Adjusted Expected Development Outcome (RAEDO), assuming perfect work quality. b. Predicted is RAEDO with actual risk profile and actual work quality. for success is higher but the potential is not problem in the econometric analysis of investment achieved largely because of shortcomings in work decision-making, is also potentially an issue, as quality. mentioned earlier. Thus continuous refinement of the model is needed going forward. While the risk-adjusted results provide a different perspective on results, it is still a work in progress, It is worth noting that the current M&E system is and further data and model refinements will be designed to measure the level of effectiveness of required to test and improve its reliability. Also, the institution at the project and aggregate levels, as with all models, it has certain limitations. For but does not offer a single measure of the com- example, most of the variables, and the model's parative magnitude of development impacts structural parameters, reflect IFC's experience. across projects. Therefore, since the RAEDO ap- Therefore, comparisons of performance are valid proach is also based on projects' development suc- only within IFC, across regions and industries, cess rates, it still cannot capture the differences and across time for IFC as a whole. Small sample that may exist with respect to these magnitudes. sizes for some of the departments affect the reli- This is an interesting but complex area for future ability of the estimates. Endogeneity, a perennial work, which IEG intends to pursue. 84 APPENDIX E: CHRONOLOGY OF IFC ADVISORY SERVICES Table E.1. Chronology of IFC AS Year Event Facilities and Initiatives 1981 Business Advisory Service (BAS) for the Caribbean and Central America established (closed FY97). 1985 South Pacific Project Facility (SPPF) established to assist and accelerate the development of productive, self-sustaining SMEs in Pacific Island countries. Foreign Investment Advisory Services (FIAS) created. 1986 Africa Region-Africa Project Development Facility (APDF) established. 1988 Technical Assistance Trust Funds (TATF) program instituted--to develop technical assistance (TA) projects to help strengthen the business environment in all IFC client countries, focusing on TAs to promote private sector growth. 1989 Africa Management Services Company (AMSCO) established to assist those SMEs that have substantial African ownership to become more sustainable and competitive in national and international markets. 1990 Pacific Enterprise Development Facility (PEDF) established to assist in and accelerate the development of productive, self-sustaining SMEs in Pacific Island countries (renamed PEP-Pacific in FY07). 1991 Polish Business Advisory Service (PBAS) established (closed FY96). 1994 Enterprise Support Service for Africa (ESSA) established (closed 2002, made part of APDF). 1997 Mekong Project development Facility (MPDF) launched to foster growth in the number and size of domestic private firms in the Mekong Region. 2000 China Project Development Facility (CPDF) to support the development of private SME s in the interior of China, with an initial focus on Sichuan province (renamed PEP-China FY07). Private Enterprise Partnership for Eastern Europe and Central Asia (PEP-ECA) to provide focused TA, with the goal of helping build successful private businesses in the former Soviet Union region (operating since 1987). Southeast Europe Enterprise Development (SEED) launched. 2001 Developing Enterprises in South Asia (DESA) created. 2002 Corporate Citizenship Facility (CCF) to demonstrate the business case for progressive approaches to corporate citizenship and to leverage the potential of IFC investments to act as a catalyst for improved environmental and social performance. Environmental Opportunities Facility (EOF) to provide catalytic project development funding and flexible investment financing for innovative projects that primarily address local environmental issues. (Table continues next page) 85 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table E.1. Chronology of IFC AS (continued) Year Event 2002 South Asia Enterprise Development Facility (SEDF) established to increase the number and growth rates of SMEs in Bangladesh, Bhutan, Nepal, and northeast India. Capacity Building Facility (CBF) initiated to fund partnerships and programs that support the four core pillars of the Bank Group SME strategy. SME initiatives--To support various initiatives such as: (i) addressing broader SME development issues (access to financing, business enabling environment, local economic development, and capacity building); (ii) funding pilot and partnerships projects; and, (iii) building local capacity for SME lending in target markets. North Africa Enterprise Development Facility (NAED) established to support the development of markets and institutions that are key to SME growth in (initially) Algeria, Egypt, and Morocco. Sustainable Financial Markets Facility (SFMF) established to enhance the environmental and social impact of financial intermediaries (FIs) operating in developing countries, and to strengthen FIs' competitiveness by improving their capacity to manage environmental risk and the opportunities arising from increased sustainability; and to have a strategic impact on the sustainability agenda of the broader financial community. Indonesia Enterprise Development Facility (IEDF). 2003 Program for Eastern Indonesia SME Assistance (PENSA) initiated to support the increased flow of capital to SMEs by strengthening SME banks, creating new SME financial products, and identifying and preparing projects for follow-on IFC investment; to support linkage programs related to IFC investments and to work with IBRD on improvements in the business enabling environment. Latin America and Caribbean Small and Medium Enterprise Facility (LAC SME Facility) established to promote private sector development through SMEs in selected countries in Latin America (e.g., target countries of Bolivia, Honduras, Nicaragua, and Peru) with the aim of fostering job creation and reducing poverty in the host countries. Iraq Small Business Financing Facility (ISBFF) established. 2004 DevCo established to put in place sustainable contractual agreements in which infrastructure services are privately provided, with an emphasis on the provision of services to those that currently do not enjoy access. PEP-MENA assumed the activities of PEP-ME and NAED facilities and established to provide TA to support private sector development in all countries in the MENA region. 2005 BIDF established to assist the public sector in Southeast Europe to increase private participation and investment in infrastructures that contribute to economic development (renamed PEP-SEI in FY07). Formerly known as SGBI transformed into Grassroots Business initiative (GBI). Established to strengthen and expand support for Grassroots Business Organizations by the World Bank Group and others. SLDF established to expand SEDF's South Asia SME Development Program from Bangladesh to Sri Lanka and Maldives. Global Corporate Governance Forum (GCGF) established to promote global, regional, and local initiatives to improve the institutional framework and practices of corporate governance in developing countries. PEP-SE initiated to develop targeted and innovative projects to support private sector development. 2006 (a successor program to SEED). Mozambique SME initiative (MSI) established to finance SMEs on a commercial basis and provide TA to investee companies and outside service providers in Mozambique. PEP-Africa established to enhance support to SMEs, support IFC direct investment through project development, and engage in improving the investment climate (a successor program to APDF). 2006 PEP-Philippines established to improve the business environment for SMEs to contribute to a broader-based economic growth and to sustainable poverty reduction in the Philippines. 86 A P P E N D I X E : C H RO N O L OG Y O F I F C A DV I S O RY S E RV I C E S Table E.1. Chronology of IFC AS (continued) Year Event 2006 EICDF established to find technical assistance that will benefit local communities with the focus on Africa. PEP-ACEH established to provide technical assistance focused on private sector development in Aceh/Nias region in Indonesia. PEP-SADI established to assist activities in agribusiness supply chain linkages, rural financing, and infrastructure in Indonesia. 2007 BICF Bangladesh Investment Facility CES facility (SBI, SBAP, GEF, PPSPF CES GEF) Operational/System Changes 2003 IDA-IFC MSME pilot program launched in 2003. 2005 Development and implementation of a standardized Advisory Services (AS) approval process. Project Supervision Review (PSR) process introduced. 2006 AS principles developed. AS operations organized around five business lines and business line leaders appointed. 2007 Pricing policy introduced for AS products. Guidelines for Bank Group coordination created; joint Bank Group review of AS. First core product review. 2008 AS guidelines created. Second core product review and target of 80% "core" products (developed and in development). First donor survey. Regional AS portfolio review meetings introduced. AS legal agreements database launched. 2009 IFC conflict-of-interest guidelines introduced (previously COI was governed by overall Bank Group COI guidelines). Performance and M&E 2005 AS training programs launched. 2006 New project M&E system introduced. Results measurement group for AS formed. PCR system introduced. Smart Lessons introduced for sharing of experiences (database launched 2007). 2007 Standardized performance indicators introduced. Project Completion Report (PCR) incorporated into iDESK. First IFC AS client survey launched. (Table continues next page) 87 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table E.1. Chronology of IFC AS (continued) Year Event Organizational Changes 2000 The Global Financial Markets Group, which encompasses the financial markets activities of the seven regional departments and the Financial Markets Advisory Department, was created. 2003 The reporting lines for the various Project Development Facilities were changed, placing them under the responsibility of the IFC regional directors in order to strengthen the facilities' integration with regional strategy, products, and services. 2004 Establishment of a funding mechanism for advisory services to consolidate the different sources of funding alternatives available within IFC. 2005 AS Corporate Cadre created. Advisory Services Portfolio Management Unit established. 2006 Joint World Bank/IFC Financial and Private Sector Development Vice Presidential Unit. 2008 Vice Presidency for advisory services established. Advisory Services Corporate Cadre expanded. Access to Finance moves to Business Advisory Services Vice Presidency. Regional sector leaders/BLLs appointed. Source: IEG. 88 APPENDIX F: ADVISORY SERVICES FACILITIES, BY REGION Table F.1. FY08 Advisory Services Facilities, by Region Facilitiesa Approximate share of Delivery units Business lines Region (funding side) FY08 AS expenditures (most common) addressed AFRICA PEP Africa 19% PEP-Africa PEP Africa addresses all Mozambique SME Initiative, CAS business lines. However, SME Solution Center Regional: Global: CIC most of the ESS, Infra- $26.5m. $19.5m. CES structure and BEE initia- tives projects are addressed by headquarters. EAST ASIA PEP-China 15% CEA & PACIFIC MPDF CES PEP-Pacific Regional: Global: CAS PEP-Philippines $28.4m. $7.8m. CIC PENSA CENTRAL AND PEP 8% PEP Financial Markets, Corpo- EASTERN CES rate Governance, Linkages, EUROPE Regional: Global: CIC SME Policy, Agribusiness. $22m CGM Energy Efficiency at design stage. LATIN AMERICA LACPb 8% LACP Until 2007, LAC facility & CARIBBEAN CES mainly addressed three Regional: Global: CAS business lines: ESS, BEE, $14.1m. $5m. CIC and Corporate Advice. Re- CGB cently, Infrastructure and CGF A2F business lines are added to the coverage of the facility. MIDDLE EAST & PEP-MENA 9% PEP MENA PEP-MENA addresses BEE, NORTH AFRICA ISBFF (Iraq facility) CAS (Dubai + financial markets, SME de- Regional: Global: headquarters) velopment and infrastruc- $19.2m. $3.3m. ture pillar (Table continues next page) 89 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table F.1. FY08 Advisory Services Facilities, by Region (continued) Facilitiesa Approximate share of Delivery units Business lines Region (funding side) FY08 AS expenditures (most common) addressed SOUTH ASIA SEDF-South Asia SME Development 7% SEDF Program, SEDF-Sri Lanka and SLDF Maldives (SLDF); BICF: Bangladesh Regional: Global: BICF Investment Climate Facility $15.5m. $2.8m. CES SOUTHERN PEP-SE (consolidation of SEED, BIDF) 9% PEP-SE Financial Sector Devel- EUROPE AND PEP-SEI PEP-SEI opment and Access to CENTRAL ASIA Regional: Global: PEP Finance, BEE, linkages $10.7m. $10.5m. CIC and Infrastructure CES GLOBAL · FIAS 25% ($62.2m, this amount FIAS · DEVCO includes just the global CIA/CAS · SME Initiatives projects, the projects GBD · CES Facilities (SBI, SBAP, GEF, delivered to regions are SME unit PPSPF, CES GEF) already included above) CES · EIDF (Extractive Industry Dev. 100% ($244.7m) COC Facility) CAG · TATF · GBI/GBF · Global Corporate Governance Forum (GCGF) Source: IEG. a. For global business units, see "Global" region. b. Starting in 2005, this facility extended its coverage from SME to other business lines. 90 APPENDIX G: PROJECT SAMPLE REPRESENTATIVENESS--ADVISORY SERVICES Table G.1. Representativeness of IEG's 2006­08 PCR Reviews (compared with 2006­08 PCR population) IEG project reviews take place following a self-assessment by IFC (Project Completion Report). Accordingly, the representa- tiveness of IEG review coverage can be determined based on the population of PCRs for a particular period of time. This table compares the breakdown of the IEG review sample from inception of the PCR system in 2006 up to June 2008, with the pop- ulation of PCRs completed since inception in 2006 through June 2008. For reference, the table includes the breakdown of the active AS portfolio. Active PCR PCR portfolio # population # sample # (as of (as of (as of June 2008) % June 2008) % June 2008) % 707 100 458 65 REGION Central and Eastern Europe 44 5% 66 9% 32 7% East Asia and Pacific 131 16% 141 20% 88 19% Latin America & Caribbean 92 11% 84 12% 53 12% Middle East and North Africa 87 10% 98 14% 62 14% South Asia 79 9% 54 8% 38 8% Southern Europe and Central Asia 83 10% 70 10% 54 12% Sub-Saharan Africa 189 23% 138 20% 101 22% Global 134 16% 56 8% 30 7% Grand total 839 100% 707 100% 458 100% PRIMARY BUSINESS LINE Access To Finance 226 27% 134 19% 90 20% Business Enabling Environment 170 21% 225 32% 162 35% Environment and Social Sustainability 135 16% 109 15% 63 14% Infrastructure 98 12% 75 11% 45 10% Value Addition to Firms 193 23% 164 23% 98 21% Grand total 822 100% 707 100% 458 100% (Table continues next page) 91 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table G.1. Representativeness of IEG's 2006­08 PCR Reviews (continued) (compared with 2006­08 PCR population) Active PCR PCR portfolio # population # sample # (as of (as of (as of June 2008) % June 2008) % June 2008) % IDA STATUS FY08 IDA blend 123 15% 92 13% 59 13% IDA-only 260 31% 200 28% 137 30% non-IDA 245 29% 270 38% 170 37% (blank) 211 25% 145 21% 92 20% Grand total 839 100% 707 100% 458 100% SIZE (Total Funding) X = < 15,189 4 0.5% 13 2% 7 2% 15,189 < X = < 686,287 519 62% 614 87% 410 90% X > 686,287 315 38% 79 11% 40 9% (blank) 1 0% 1 0% 1 0% Grand total 839 100% 707 100% 458 100% SIZE (Total Funding, Quartile) X = < 84,250 77 9% 177 25% 117 26% 84,250 < X = < 312,080 254 30% 352 50% 240 52% X > 312,080 507 60% 177 25% 100 22% (blank) 1 0% 1 0% 1 0% Grand total 839 100% 707 100% 458 100% PRODUCT TYPE (at June 2008) Developed 257 31% 151 21% 95 21% In development 307 37% 264 37% 171 37% Legacy 105 13% 146 21% 100 22% Other 154 19% 146 21% 92 20% Grand total 823 100% 707 100% 458 100% START FY 1996­2003 96 15% 63 12% 2004 120 17% 90 20% 2005 197 28% 127 28% 2006 197 28% 121 26% 2007 91 13% 52 11% 2008 6 1% 5 1% Grand total 707 100% 458 100% 92 A P P E N D I X G : P R O J E C T S A M P L E R E P R E S E N TAT I V E N E S S -- A D V I S O R Y S E R V I C E S Table G.1. Representativeness of IEG's 2006­08 PCR Reviews (continued) (compared with 2006­08 PCR population) Active PCR PCR portfolio # population # sample # (as of (as of (as of June 2008) % June 2008) % June 2008) % END FY 1996­2003 6 1% 4 1% 2004 29 4% 28 6% 2005 103 15% 89 19% 2006 180 25% 153 33% 2007 236 33%a 132 29%a 2008 152 21%a 52 11%a (blank) 1 0% Grand total 707 100% 458 100% Source: IEG. a. The lower proportions of reviewed PCRs in 2007 and 2008 reflect the exclusion from the review of 47 PCRs for projects that ended in 2007 and 2008 for which development effectiveness ratings could not reasonably be expected to be achieved (e.g., one-off conferences, workshops, and feasibility studies). In such cases, independent review of development performance is not meaningful. 93 APPENDIX H: PROJECT EVALUATION METHODOLOGY--ADVISORY SERVICES The development performance indicators and rating criteria applied by IEG in reviewing Advisory Services Project Completion Reports (PCR) are set out in table H.1 below. 95 96 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table H.1. Project Performance Indicators and Rating Criteria for IFC Advisory Services Operations Rating Definition Excellent Satisfactory Partly satisfactory Unsatisfactory Strategic Importance to achieving country Major priority issues addressed; Major priority issues addressed to Some priority issues overlooked; Addressed low priority issues; relevance strategic objectives, appropriate- National/regional level impact a large extent; Assistance was appropriate at Not appropriate given the con- ness at initiation and comple- was achieved; Potential major impact on direct initiation, but conditions changed ditions at initiation; tion, including whether Advisory Highly appropriate for conditions recipients and/or local community; and assistance was not adopted No cost recovery, or contribution Services was the appropriate at initiation and completion; Appropriate for conditions at initi- accordingly; was not appropriate instrument Appropriate client contribution ation and completion; Less than appropriate cost was achieved Majority of appropriate cost recovery achieved recovery achieved Output Immediate project deliverables All major outputs achieved with All major outputs achieved with At least one major deliverable not Few or no major outputs achievement (products, capital goods, services, excellent quality; or satisfactory quality achieved; or achieved; or or advice) More than expected outputs At least one major output of less Several major outputs of less achieved with at least satis- than satisfactory quality than satisfactory quality factory quality Outcome Short- or medium-term (positive All or most major outcomes Most of the major outcomes Less than half of the major out- Few or none major outcomes achievement and negative, intended or unin- achieved; achieved; comes achieved; achieved; tended) behavioral changes re- Client attributed changes in be- Client indicated the advisory proj- Client attributes limited influence Client attributes limited influence sulting from the advisory project havior and performance to the ect contributed to major changes on behavior/performance changes. on behavior/performance changes advisory project. in behavior and performance. or they had perverse effects Impact Intended longer-term effects of Exceptional benefits beyond All intended impacts on direct re- Intended impacts on direct bene- Intended impacts not achieved; or achievement the advisory intervention direct recipients, at the national, cipients have been achieved with ficiaries mostly achieved, but Negative impacts occurred regional, global level; attribution to the project backed attribution to the project is weak; Impacts extended nationally by solid evidence; Intended impacts partly achieved; or internationally as best practice Most direct impacts have been Intended impacts mostly achieved, and recommended for replication; achieved and some impacts were but some negative impacts All major impacts achieved with achieved beyond direct recipients occurred strong attribution to the project with attribution to the project backed by evidence from a solid backed by solid evidence. methodology. Efficiency Ratio of costs to benefits; Highly positive cost-benefit ratio; Positive cost-benefit ratio; Negative cost-benefit ratio; Highly negative cost-benefit ratio; Economy in use of resources; Resources expended highly Resources expended economically; Resources could have been Resources could have generally Cost in relation to alternatives economically; Resources reasonable in relation at times expended more been expended more economically; Resources far less costly than to alternatives economically; Significantly more reasonable alternatives More reasonable alternatives alternatives available and could available and could have been have been used used Development Synthesis (not an average) of the Six-point scale, ranging from highly successful (overwhelmingly positive development results and virtually no flaws) to highly unsuccessful (nega- effectiveness above five dimensions tive developments and no positive aspects to compensate). Source: IEG. APPENDIX I: FINDINGS FROM IFC-COMMISSIONED REVIEWS OF IFC ADVISORY SERVICES This appendix summarizes some of the main findings · The facility works in close cooperation with from 51 IFC-commissioned reviews that were com- the Bank Group and other donors. pleted up to December 2008. They encompass: 7 · Generally, the facility has a positive rela- program reviews; 16 product reviews; and 28 proj- tionship with IBRD. Three of five IBRD coun- ect reviews (of which 7 were impact evaluations). try managers said the facility had met or exceeded expectations. PSD specialists say Program Reviews the relationship has been very strong in IFC-commissioned reviews have been completed some countries but more interaction re- for six regional facilities, and one business line. garding SMEs is desired. · Links with investment is unclear to staff re- Findings on alignment include: garding the extent to which they should/ ·Strategy: should not be linked. · The facility does not have an overall strategy, just an aggregation of five business line plans. Findings on delivery include: · The rush to ramp up has affected long-term · Funding/pricing: planning. Strategies have not addressed mar- · Limited assessment of client willingness to ket conditions, articulated goals, constraints, pay and existence of market failure/s. In two the facility role, etc. cases, the client was likely to hire a consult- · Sector-based approaches are generally ant, and pay, without IFC's help. In another sound, but more comprehensive strategies case, the facility underwrote the cost of key should be developed. information systems that the company was ·Organizational structure: already using. · Some regional programs and a department · Almost none of the investment clients re- did not have formal relationship to the re- ceiving IFC AS paid the full cost. gional facility. The business line is mainly ·Project design: managed by a department, rather than facility · Project timetables are often unrealistic. staff. On the other hand, in the investment · Potential problems and time horizons have climate area, the structure is different and been underestimated. strategically coherent. · Pressure to "ramp up," rather than a delib- · The business line is run largely independent erate process of resource allocation. of the facility, with staffing split between the ·Project implementation: two. · Management of the business line projects has ·Coordination with others: suffered from lack of staff continuity, sporadic · Organization by pillars, rather than by coun- coordination between the objectives of the try, seems to constrain identification of syn- business line projects and IFC investment ergies with the Bank. operations, and low allocation of resources · Some awareness of each others' programs and staff incentives for the task managers. A (with the Bank), but limited exchanges of general lack of management attention to views and common programs. technical assistance projects appears to be 97 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 the principal factor, since IFC performs these not always reflect the relative needs of coun- same functions well in investment opera- tries in the region. For example, roughly the tions, often with the same implementing same level of funding was budgeted for proj- partner. ects in some countries where there is a huge ·Staffing: disparity in income levels. By business line, · Ninety-five percent of staff are co-terminus expenditure levels also show no relation- with the facility, 5 percent are open-ended; ship with objective indicators of country 34 percent of staff work in admin/mgmt needs for services, such as assistance on im- roles, indicating a high level of bureaucracy. proving the business environment, expand- Recruitment takes a long time, whether for ing private credit, and developing private one year or an open-ended hire. It is also infrastructure. hard to recruit the best professionals with a · All pillars address important issues regard- short-term offer. ing business development in the region. · Staffing is linked to donor funding, and it is However, for some pillars, the linkages be- difficult to recruit the best people on short- tween business development and develop- term contracts. But the facility has still ment impact would have to be supported by ramped up relatively quickly. Supervision empirical studies. quality is an issue. · The facility's programs have been relevant to ·Performance M&E: the private sector development needs in the · IFC needs to establish metrics to measure the countries concerned. Allocation of resources performance of the implementing organi- across countries seems reasonable, with no zations according to commercial industry major duplication of efforts among donors standards. and no major holes in coverage. · Confusion exists over logic models and in- · Results: dicators among staff. Too many and inap- · Of the nine programs reviewed in depth, all propriate indicators are chosen. Supervision were in accordance with the facility's strat- reports are not submitted in a timely fashion. egy, and addressed identified needs. Six of · The facility has been struggling to find the them were considered to have achieved sat- right process controls and tracking systems. isfactory or better effectiveness. However, in · Financial controls are in place, but budget two cases, each rated "good" for effective- and cost accounting could be strengthened. ness, linked investments were not yet oper- · The validity and reliability of results data are ational, and, accordingly, effectiveness was questionable. The focus should be on data based on the likelihood of good outcomes collection and simple reporting, rather than rather than realized outcomes. critical analysis. Baseline data is lacking. · The facility appears to have been effective · Very strong in this facility, contributing to in achieving certain objectives: i) helping a better definition of the facility's program to increase credit available to underserved (e.g., through lessons and results dissemina- markets; ii) improving the skills of business tion), and acknowledgement in headquarters. managers, reducing regulatory burdens on · There is room for improvement in financial businesses; and iii) securing private partici- management (budgeting, approval, and ex- pation in public infrastructure projects. At the pense tracking). same time, a number of projects did not ap- · There are a number of problems with the va- pear to have generated the intended out- lidity and reliability of performance data. comes. Generalizations from these findings are, however, limited, since the review looked at Findings on effectiveness include: only four projects in depth (projects that had · Relevance: been selected by the facility as "best examples" · The facility has carried out relevant activities, of the ones that generated significant devel- although the pattern of expenditures did opment impacts, thus not representative of the 98 A P P E N D I X I : F I N D I N G S F RO M I F C - C O M M I S S I O N E D R E V I E W S O F I F C A DV I S O RY S E RV I C E S population of facility's projects) and relied ·IFC Role, strengths, and weaknesses: heavily on interviews with staff and secondary · The main strength of the projects is derived data provided by the facility. from IFC's strategy and multifaceted ca- · Many activities have proved effective, such pacity to support commercially viable service as contributions to business legislation. How- providers (licensed and regulated financial ever, "the case that these programs have institutions). The unique value proposition been effective in achieving stated objectives of the IFC vis-à-vis other development agen- is harder to make." The review points to cies, donors, and investors is generated by some successful and some less successful the IFC's ability to employ unique combi- programs. nations of funding, technical support, and · Many changes within organizations sup- credibility to financial institutions and mar- ported by the facility are likely to persist, kets (it does not have to channel funding such as skills and process improvements. through public sector, like other donor · Some initiatives "have met with consider- organizations). The IFC's ability to engage able success," while others "had limited policy makers on financial sector matters reach and limited impact." enables the Corporation to support market · Three main success factors: i) sound plan- development. Few development agencies ning (e.g., consistent with the long-term have this capacity or mandate. plans of the client) and execution; ii) effec- tive follow-through; and iii) integration of Product Reviews projects within a particular sector. IFC-commissioned reviews have been completed · Better performance where IFC has significant for 16 products, 10 of which were in the CA busi- experience; steep learning curve for business ness line. lines, where IFC is working with new types of stakeholders and beneficiaries. Findings on delivery include: · Viability and scalability is most robust ·Project design: when the initiative is developed within a com- · A standard supply- and product-driven ap- mercially structured institution. In this busi- proach maintains consistency across proj- ness line, AS projects perform at their best when they enhance a direct IFC investment ects, but needs tailoring to meet specific or a related advisory program (synergies in market needs. program design and incentives to achieve suc- · The interventions were of mixed relevance cess are greater when linked). More like a tra- to the company's strategic goals because ditional donor grant program if not linked. of inconsistent implementation of needs as- ·Efficiency: sessments and lack of alignment around · The transaction costs associated with AS project goals and objectives. and investment operations with individual · Formal overarching plan or strategy for the FIs are high relative to the scale of these development of the toolkit is absent. operations. ·Project implementation: · Higher for developed pillars: Accumulated · Use of associations as an exit strategy has had experience allows IFC to replicate and scale mixed results, due largely to the availability up at minimum cost, although in other pil- of motivated local partners and historical lars requiring more innovative approaches, context. Alternative options may need to be efficiency remains to be demonstrated. considered. · The facility is capturing certain economies · The management structure for AS projects of scope and scale by replicating programs is evolving, with standards and procedures across countries. However, some projects for program design, oversight, and M&E being are unlikely to generate benefits commen- developed, but these are inconsistently ap- surate with the level of investment. plied across projects. 99 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 ·Staffing: port. Rationale for IFC intervention, however, · Good local staff for start-up, strong project is not always clear. management capabilities, but need for in- · Attempting to bring about the "change in the ternational commercial experts. organization's DNA" from that of a socially ·Coordination with others: oriented organization to a commercially vi- · Links with IFC investments have taken place able enterprise is an ambitious endeavor. on an ad-hoc basis, based largely on good · IFC strengths: i) Strong local staff backed personal relationships. by worldwide experience; ii) High level ·Performance M&E: support and credibility with governments; · A system for reporting on deliverables is in iii) Ability to leverage TA funds and man- operation, but it needs to be standardized. age partnership with key donors; iv) Global · M&E framework systematically tracks outputs reach and continuing presence in markets; and some outcomes, but limited focus on im- v) The ability to invest and provide liquidity pacts. Need for qualitative case studies to to the market; vi) Strong project manage- complement data. ment capabilities. · Lack of baseline data prior to the interven- · IFC weaknesses: i) A supply-driven approach tion and the absence of standard reporting to project design, based on a standard prod- metrics in the social investment space. uct (developed for the Russian market) · Baseline data and program monitoring is rather than on needs identified in a given mar- weak. ket through an ex-ante needs-assessment; · Measurable objectives and associated metrics ii) A shortage of specialists with commer- have not been developed. cial operating experience both to act as short-term resources for projects, but also Findings on effectiveness include: to advocate for leasing in headquarters; ·IFC Role, strengths, and weaknesses: iii) An evolving management structure for · Advocacy in operations has been a key AS projects, with standards and procedures strength, given the strong working rela- for program design, oversight, and M&E tionships that IFC enjoys at very high levels being developed, but still inconsistently ap- of government. This was critical to the adop- plied in the field; and iv) A lack of institu- tion of legislative agenda in all three of the tional mechanisms linking AS activities with mature projects. There may be need for investments. more attention to developing a mechanism to continue the advocacy role after com- Project Reviews pletion of the IFC project. Project reviews were completed for 28 projects, · IFC is in a very strong position to be the of which 4 were impact evaluations involving con- market leader with the product. trol group designs. · The team initiated the supply of training of this type, the market-making task is done, and Findings on results include: the work is no longer unique in most markets · Enrollment increased; revenue went up, but and cannot be justified in the country. Donors costs also increased. would be willing to subsidize, in poorer mar- · Higher quality treatment and outcomes with pri- kets, but more willing to do so if the business vate provision, compared with public provision. line is separated from IFC. Best to hand over · Better business behavior of trainees, but no sig- to local training companies, and have a foun- nificant difference in business results between dation manage the brand globally. those trained and not trained. · 79 percent of SMEs attributed success in se- · No significant difference in practices between curing contracts with the client to IFC sup- participants and nonparticipants. 100 APPENDIX J: HIGH-LEVEL COMPARISON OF IFC ADVISORY SERVICES WITH OTHER MULTILATERAL DEVELOPMENT BANKS Introduction Strategies and Objectives The purpose of this exercise is to provide per- spective on IFC's Advisory Services (AS) for pri- PSD and AS strategies vate sector development (PSD) by comparing During the late 1990s, most donor strategies for them with PSD-related advisory services provided PSD were based on the OECD Development As- by other donors. Across the major bilateral and sistance Committee (DAC) guidance for donor ac- multilateral donors, the appendix compares PSD tions to support private sector development. The strategies, volumes, and types of AS, delivery "DAC Orientations" addressed the fundamentals mechanisms, funding and pricing, monitoring of privatization, financial sector reform, and en- and evaluation systems, and results and lessons terprise development. Most donors tended to re- learned. produce the DAC framework without indicating areas of priority for their own interventions. None Information was gathered from websites and attempted to develop the analytic linkage be- telephone interviews with each of the major tween PSD and poverty reduction, nor were na- multilateral donors: the European Bank for Re- tional PSD assessments prepared that could be the construction and Development (EBRD), the Asian basis for tailored interventions. Development Bank (ADB), the Inter-American Development Bank (IDB), the African Develop- More recent PSD strategies have made progress ment Bank (AfDB), the European Commission on both of these issues. Most--including the 2002 (EC), and the European Investment Bank (EIB)-- PSD Strategy of the World Bank Group (WBG)-- and two bilateral donors: the U.K. Department now attempt to draw the analytical link between for International Development (DFID) and the PSD and poverty reduction by tracing the logical Danish International Development Assistance framework from improved competitiveness and (DANIDA). Interviews were conducted with staff and managers in sectoral and regional depart- productivity at the enterprise level to increased ments, as well as independent evaluation de- growth at the sector and economy levels, and partments. Documents reviewed included PSD calling upon a growing body of research to es- and regional strategies, descriptions of TA proj- tablish the link between economic growth and ects and programs, and independent and self- poverty reduction. Some PSD strategies, again evaluations of PSD TA projects and programs. including that of the Bank Group, call for assess- ments of the investment climate and institutional Quantitative benchmarking was limited by the capacity to direct project design (fewer carry lack of detailed data on PSD AS from many of the them out in practice--see below). donors, largely because of the result of inade- quate monitoring and evaluation systems. Thus, Most donor strategies for PSD aim to: the appendix presents comparisons only for do- · Improve the market conditions within which pri- nors and dimensions where the data seem to vate firms operate (improving the business measure the same concepts. Most of the quanti- environment, reforming the legal and regula- tative information was taken from independent tory framework, developing markets for finan- evaluations, or from annual reports. cial and nonfinancial services, strengthening 101 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 public and private sector institutions relevant for largely marginal in this field, has become less ap- PSD, improving governance). parent. The majority of donor strategies now claim · Make individual firms more competitive (fa- to assist at the macro and meso levels (figure J.1). cilitating privatization, helping firms adopt better technologies, building labor and man- Despite the agreement in principle to move away agement skills). Among the types of firms, from micro-level interventions, the practice of small- and medium-enterprises (SMEs) are the donors, both multilateral and bilateral, only weakly typical target group. reflects this consensus. While the leading multi- laterals do focus on the macro level, and some Table J.1 shows, for selected donors, the objectives bilaterals provide funds to multidonor business of PSD assistance, and of PSD TA activities more environment programs, there are few signs of specifically. Scanning the table, what stands out programs involving direct support to enterprises is the similarity of PSD strategies across donors-- being cut back. Canada and Sweden, the bilater- probably the result of efforts to harmonize donor als whose policy statements best reflect the emerg- practices via the OECD DAC guidelines, as well as ing consensus, still retain programs entailing the efforts of other coordinating groups (e.g., direct support to enterprises, including--in Can- the Committee of Donor Agencies for Enterprise ada's case--a large enterprise-to-enterprise match- Development). making program. The EC PSD Strategy says that particular attention should be given to macro- Levels of intervention level interventions, but also leaves room for micro- Most PSD AS strategies distinguish between three level programs that can crowd out private initiative "levels" of intervention for PSD: i) macro level (pol- and introduce market distortions. The recent icies), ii) meso level (institutions), and iii) micro evaluation of EC PSD activities concludes: level (firms) (table J.2). It is useful to make two more divisions. At the macro level, interventions can "Most meso- and micro-level programs are focus on classic macroeconomic policy (monetary, focused on provision of services, directly or fiscal, trade, and exchange rate), and the legal and through intermediate organizations: provi- regulatory framework. The meso (institutional) sion of a credit line, provision of BDS, organ- level includes both public and private sector insti- ization of business trips, and so on. These tutions. The private sector, of course, extends services are always provided at subsidized across the meso level (private sector institutions) rates and in the great majority of cases do not and micro level (nonfinancial enterprises). Some- tackle the causes of the malfunctioning of the times, donor PSD strategies include physical in- market. In other words, the program substitutes frastructure (telecom, ports, transport) under the the private sector instead of trying to reinforce macro level, but table J.2 excludes these physical the market. In that sense, it is possible to say that investments since they are not TA activities. there is a gap between the strategy proposed by HQ and its implementation in the field." In general, donor PSD strategies have begun to em- phasize interventions at the macro and meso lev- EBRD operates mainly at the micro level, both els, de-emphasizing micro-level interventions through the Turnaround Management and Busi- unless they have demonstrable impacts beyond the ness Advisory Services (TAM/BAS) programs, as beneficiary firm. This strategic shift away from di- well as other AS that are tied to the preparation rect, firm-level PSD support was the result of ac- and implementation of EBRD investments. EIB in- cumulated experience with projects. Because of tervenes exclusively at the micro level, as virtually their low outreach, micro-level interventions usu- all of its assistance focuses on preparing and im- ally failed to have much impact beyond the ben- plementing EIB investments. On the other ex- eficiary firms. The previous "division of labor," in treme, the Integrated Trade-Related Technical which multilaterals provided the greater part of Assistance program funded by DFID (and other enabling environment support, and bilaterals were donors) focuses almost entirely on the macro 102 A P P E N D I X J : H I G H - L E V E L C O M PA R I S O N O F I F C A DV I S O RY S E R V I C E S W I T H OT H E R M D B s Table J.1. Strategic Objectives Organization PSD objectives PSD TA objectives IFC PSD strategic directions: No overarching strategy for AS. Direction is provided by IFC PSD is a way of doing things, not a sector. Corporate Strategies and Road Maps. PSD is about a good balance between the complementary "IFC Advisory Services are an important and growing part functions of the state and the private sector. of IFC's business. They contribute significantly to IFC's additionality by improving the business enabling environ- Public policy for the private sector and direct support to the ment for the private sector, as well as the capabilities of private sector need to form part of a comprehensive approach companies." to development and reflect country and sector conditions. "A number of programs are being developed to promote Specific PSD objectives: Extending the reach of markets, combined investment and advisory services to increase improving access to basic services. IFC's value-added to projects." World Bank (2002). (IFC 2008) IFC priorities: Strengthening the focus on frontier markets, including SMEs and agribusiness Building long-term partnerships with emerging players in developing countries Addressing climate change, and environment and social sustainability activities Addressing constraints to private sector growth in infrastructure, health, and education Developing local financial markets through institution build- ing, the use of innovative financial products and mobilization (IFC 2008) ADB For public sector operations: (i) to support developing For public sector operations, TA seeks to help formulate the member country governments in creating enabling regulatory and institutional frameworks needed to make conditions for business, and (ii) to generate business markets work better and to build the capacity of market opportunities in ADB-financed public sector projects. For regulatory authorities. Specific areas of intervention for TA private sector operations, to catalyze private investments are policy reform, institutional development, privatization, through direct financing, credit enhancements, and risk corporate governance, financial sector, and SMEs. mitigation instruments. For both public and private sector (ADB 2000, 2007) operations, there are four areas of focus: (i) governance in the public and private sectors, (ii) financial intermedia- tion, (iii) public-private partnerships, and (iv) regional and subregional cooperation. ADB (2000) EBRD EBRD's PSD strategies are part of: (i) country strategies for For the TAM/BAS program, "to promote the economic each country, which include a private sector section, and transition through advice and mentoring at the enterprise (ii) sector-specific strategies (agribusiness, energy, natural level and the development of a sustainable infrastructure of resources, property, shipping, and transport), which cover business advisory services, and to contribute to improving the public and private sectors, depending on the subject. the policy and regulatory environment for business." (EBRD website) (EBRD 2007b) MSME strategy: "to provide support for MSMEs across For investment-related TA, to promote institutional reform all of the Bank's countries of operations, strengthen the and improved corporate governance. (EBRD website) financial sector infrastructure dedicated to financing growth of MSMEs of all sizes, improve the business environment for MSMEs, and develop the skill sets of entrepreneurs." (EBRD 2006) (Table continues next page) 103 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table J.1. Strategic Objectives (continued) Organization PSD objectives PSD TA objectives IDB Four "strategic directions": (i) development of an enabling The IDB's technical assistance supports private sector environment for business, (ii) financial support for specific development by restructuring and modernizing the public private sector projects, (iii) leveraging developmental impact sector, supporting investment sector reform, promoting in underserved markets, and (iv) engaging the private sector regional trade and integration, and supporting micro and in dialogue and action. (IDB, 2004) small businesses. (IDB 2004). AfDB The AfDB aims at inducing private sector growth in regional Technical Assistance Facility: member countries (RMC) by: (i) supporting reforms of the Policy advice and technical assistance to governments in policy/regulatory enabling environment for private sector in order to facilitate the creation of an enabling environment, RMCs through country dialogue and policy-based operations; promote privatization schemes, revise and rationalize (ii) improving the physical and financial infrastructure in investment codes and fiscal regimes, promote foreign direct RMC to enhance private enterprises productivity and com- investments, develop the financial sector and capital petitiveness; (iii) supporting the strengthening of human markets, etc. capital, in terms of expanded technical assistance, transfer Financial advisory services to governments for privatization of skills, know-how and technology; and (iv) catalyzing in- projects. flow of financial resources to RMCs through direct invest- ment and diversification of financial services. (AfDB 2004) Advisory services to private operators on the formation of new projects or the restructuring of existing ventures. Technical assistance to private sector clients in order to overcome important constraints or capacity deficiencies. Technical assistance to other economic agents, which play a role in promoting private sector development, such as business associations, etc. (AfDB 2004) EIB PSD strategies are prepared at the country level. To support the preparation and implementation of EIB (EIB website) investments. (EIB website) For Mediterranean countries, "to support activities upstream of projects such as policy, legal, regulatory, and institutional reform, sector development strategies, capacity-building and training." (EIB 2006) Sources: Shown in italics. level, funding country-level diagnostics of exter- instruments and extending access to smaller firms. nal and internal constraints to global trade. The Business Enabling Environment (BEE) and En- vironmental and Social Sustainability (ESS) busi- The EC, IBRD/IDA, IDB, and the other multi- ness lines work at both the policy and institutional lateral development banks intervene at all three levels. The Foreign Investment Advisory Service levels. The same is true for IFC, but macro-level (FIAS), Multilateral Investment Guarantee Agency Advisory Services are limited to legal and regula- (MIGA) TA to investment promotion agencies, tory frameworks. The Corporate Advice (CA) busi- and the Doing Business and Getting Finance as- ness line focuses on individual firms, supporting sessments are part of the BEE business line. Finally, privatization transactions, as well as providing as- the Infrastructure (INF) business line involves sistance to SMEs. Access to Finance (A2F) oper- both the micro level (public-private partnerships) ates mainly at the institutional level, assisting and the macro and meso levels (regulation of financial intermediaries in developing financial natural monopolies and related institutions), 104 A P P E N D I X J : H I G H - L E V E L C O M PA R I S O N O F I F C A DV I S O RY S E R V I C E S W I T H OT H E R M D B s Table J.2. Levels of Intervention for PSD AS Macro level Meso level Micro level Macroeconomic Legal and regulatory Public sector Private sector policy framework institutions institutions Individual firms Trade and exchange rate Regulation of natural Competition authorities Chambers of commerce Management skills and policies monopolies Banking regulators Employers organizations entrepreneurship Monetary policy and Competition policy Revenue and customs Labor unions Manpower and labor inflation control Bankruptcy law authorities skills Financial intermediaries Tax policy and fiscal Legal system Courts Technology, expertise, Trading exchanges expenditure quality management Anticorruption and R&D institutions BDS providers Labor market policy, transparency Access to finance Training institutions Quality, testing, and observance of labor Access to information Property rights Investment promotion certification centers standards agencies Financial sector regulation and supervision Privatization policy Source: IEG Figure J.1. Focus of AS Interventions in PSD Strategies Macro level Meso level Micro level Macroeconomic Legal and regulatory Public sector Private sector Individual policy framework institutions institutions firms IFC Corporate Advice IFC Access to Finance IFC Business Enabling Environment IFC Environ. & Social Sustainability IFC Infrastructure EC TA to Third Countries EC Instrument for Pre-Accession Assistance DFID Trade-Related TA EBRD TAM/BAS IDB PSD TA IDB MIF ADB PSD TA IBRD/IDA PSD TA EIB Source: IEG 105 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Table J.3. Integration of AS with Core Activities Private sector lending No equity or and investments Public sector lending lending operations EIB: Virtually all PSD TA is tied to IBRD/IDA: Of ESW delivered during fiscal DFID, DANIDA, and other bilateral existing or potential investments 2002­06, 41% were aimed at informing donors: PSD TA is independent (donor (donor interviews) Bank lending. interviews) EBRD: 88% of TA supports EBRD About two-thirds of a selected sample of investment projects (EBRD website) 119 loans was preceded by ESW, including nearly all development policy IFC: 20­30% of AS supports IFC loans equity and lending operations ESW (and sometimes TA) was generally used to inform country strategies. (all from World Bank 2008) ADB: 25% of TA approvals (2000­06) were for project preparation (ADB 2007) Sources: Shown in italics. including the assistance provided by the Public- Donors that lend directly to governments also Private Infrastructure Advisory Facility (PPIAF). integrate their PSD AS with lending operations, but the interventions are mostly at the meso and Integration with core activities macro levels, supporting policy and regulatory re- For donors that invest and lend directly to private forms and institutional development. IBRD/IDA firms, AS at the micro level tends to be closely and the other MDBs fall into this group. Most of linked with these core activities (table J.3). In these donors have called for closer links with these cases, trust-funded AS can be seen as an al- lending activities at the country level in order to ternative to project preparation funding from the improve the strategic focus of PSD TA. donor's administrative budget. This is the case with EIB, for example: virtually all of the PSD TA Finally, the bilateral donors, whose core activity provided by EIB is intended to assist in the prepa- is not investment and lending operations, offer "in- ration and implementation of potential invest- dependent" PSD TA. ment operations. Direct interventions versus market development The integration of IFC's AS with lending and in- Experience has shown that direct, subsidized vestments is less than EIB's, but is still impor- provision of both credit and Business Develop- tant. Advisory Services under the Infrastructure ment Services (BDS) tend to distort markets and business line in particular tend to be linked with have low sustainability. Most donors seem to potential IFC investments. Overall, the share of IFC have learned these lessons in their financial ser- AS that is tied to existing or potential investments vices interventions, but fewer have adopted the (measured by percentage of new project ap- "market development approach" to BDS. For ex- provals) has been between 20 percent and 30 ample, although the EC's PSD strategy explicitly percent. For the future, IFC's strategy is to increase adopts donor guidelines on BDS market devel- these linkages. opment, in practice the EC maintains programs 106 A P P E N D I X J : H I G H - L E V E L C O M PA R I S O N O F I F C A DV I S O RY S E R V I C E S W I T H OT H E R M D B s Table J.4. Volume of PSD AS Organization Number of Expenditure and program Coverage and time period activities (US$m) IFC Advisory All five business lines, total project-related and non-project- 450 $190.5m Services related expenditure, FY07 ADB PSD AS 2006 approvals 260 $241m 2004­06 avg./yr. 294 $213m EBRD PSD AS TA financed by Technical Cooperation Trust Fund (TCTF), 2007 -- $137.5m Of which: TAM/BAS -- (EUR 98.2m) $16.5m (EUR 11.8m) IDB MIF Nonreimbursable technical cooperation grants, 2007 116 $100m IBRD/IDA PSD ESW Economic and sector work in PSD sector, FY06 75 $10.261m and AS Nonlending TA in PSD Sector, FY06 27 $4.316m Sources: IFC (2007), ADB (2007), EBRD website, IDB MIF website, World Bank (2008). that provide subsidized BDS directly to SMEs. In Recent PSD and PSD AS strategies call for diag- a similar way, IFC maintains demand-side subsi- nostics of local conditions--in the investment dies for BDS to SMEs in its PSD projects with IDA climate, institutional capacity, or market develop- in the Africa Region. In countries where markets ment--to be carried out before an AS program is for advisory services are well-developed, IFC runs designed. In practice, these are seldom done, al- the risk of crowding out the private sector when though the Bank Group's Investment Climate As- it directly provides competing services. sessments, FIAS diagnostics, and Doing Business/ Getting Finance assessments are exceptions to Alignment between strategy and operations the rule. A frequent complaint found in evaluations of donor PSD assistance is that the actual activities Volume and Types of Advisory Services implemented are not well aligned with the stated Table J.4 presents information on the volume of PSD PSD or PSD AS strategy. Instead of following a AS activities of some of the major donors. In FY07, strategic, top-down approach, actual practice is IFC spent about $191 million on AS. This figure in- more consistent with an opportunistic, bottom- cludes project expenditures for all five business up approach. For donors, such as EIB that are pro- lines, as well as project-related expenditures (pro- viding PSD AS primarily as a complement to gram management and support, new business de- private sector investments, it makes sense that TA velopment, and monitoring and evaluation) and projects are made opportunistically. For other non-project-related expenditures. By comparison, donors, the lack of alignment between strategy ADB spent about $241 million on PSD AS in 2006, and project selection reflects unresolved issues of an increase from an annual average of $213 million staff incentives, the desire to disburse, competi- over 2004­06 (amounting to about 3 percent of tion with other donors, or a lack of relevance of ADB operations). EBRD provided about $138 mil- the strategy with country conditions. lion from its Technical Cooperation Trust Fund in 107 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 2007, up from an average of $112 million annually PSD AS outputs are rarely standardized, either in in recent years. IDB's Multilateral Investment Fund terms of format or approach. An evaluation of the (MIF), the primary source of PSD AS not funded Asian Development Bank's PSD AS concluded by IDB loans, spent about $100 million during the that there is scope for greater standardization in same year. IBRD/IDA spent about $15 million in many products (e.g., training seminars). The Bank FY06 on economic and sector work (ESW) and AS Group's core diagnostic reports on the investment in the PSD sector, about the same amount that was climate (Investment Climate Assessments, Doing spent per year during the previous five years. Business and Getting Finance indicators) stand out as highly uniform products. By area of activity, IFC has focused particularly on A2F and CA. By comparison, EBRD had also fo- Project Selection, Management, cused on the financial sector--39 percent to bank- and Delivery ing, the Direct Investment Facility (DIF), and the Direct Lending Facility (DLF); and 26 percent to Selection process infrastructure. EC support has been directed to "in- Several independent evaluations of donor-funded stitutional and structural reforms" (42 percent of PSD AS programs indicate that the process of PSD AS) and "enhancing human resources and ca- identifying and selecting projects is more ad hoc pacities" (24 percent), although this data is quite than the donors' PSD strategies would suggest, old (1994­2003). The general conclusion is that, and that quality-at-entry (QAE) processes are consistent with their PSD strategies, other donors weak. cover a wide range of "sectors" or "business lines" in their PSD AS. IFC's range of AS does not stand · An evaluation of the European Commission's out in this regard. However, IFC's expenditure PSD AS found that project decisions were made on ESS does not appear to be matched by the other because the choice seemed "evident" or was donors. an extension of past community support. The importance of sound diagnostic work be- There is very little comparable data on the mix of fore deciding where and how to intervene was PSD AS "outputs" provided by donors. The range underestimated. of outputs includes: · A 2007 evaluation of ADB's AS in all sectors found that AS formulation processes were in- adequate: there was no formal guidance on the · reports (sector and thematic studies, policy preparation of TA proposals; guidelines pro- notes, diagnostics, advisory reports) duced in 2003 were never finalized or adopted, · surveys, data collection, and data analysis and there were weaknesses in quality-at-entry · policy advice processes. The role of the Staff Review Com- · drafting of legislation, client document review mittee has diminished over time, and such · technology adoption advice meetings usually are waived. · capacity-building and change management in institutions Headquarters vs. field management The trend among most donors seems to be to · twinning arrangements with private firms initiate and manage PSD AS--particularly stand- · knowledge-sharing forums: conferences, sem- alone AS that is not integrated with the organi- inars, workshops, and training courses zation's core activity--from country or regional · preinvestment and preprivatization due diligence offices (table J.5). This is consistent with the trend · institutional development plans in IFC. Independent evaluations of several donor- · "how-to" guidance (technical notes, imple- funded programs have found similar advantages mentation plans, "best-practice" manuals, pro- decentralizing AS management to the field, includ- cedural guidelines) ing better identification of needs and tailoring of 108 A P P E N D I X J : H I G H - L E V E L C O M PA R I S O N O F I F C A DV I S O RY S E R V I C E S W I T H OT H E R M D B s Table J.5. AS Management and Personnel Location of project management Nature of TA personnel IFC Compared to IFC investment personnel, a higher percentage of Compared to IFC investment personnel, a higher percentage of AS projects are managed from field offices (e.g., Facilities) AS personnel are short-term consultants. EIB EIB headquarters in Brussels Mainly international consultants; some in-house sector econo- mists and engineers to assess and advise on individual projects EC PSD AS is increasingly delivered on a decentralized basis. EBRD TAM: team is led by a Senior Industrial Advisor TAM: Experienced directors and senior managers from devel- BAS: overall management and support from headquarters in oped countries, contracted on the basis of individual projects London; country operations managed from field offices BAS: local consultants who have undergone an accreditation process work directly with SMEs ADB Despite increased delegation to resident missions, AS projects With a relatively small core of professional staff, most of ADB's remain predominantly delivered from ADB's headquarters in AS delivery is outsourced to consultants. Manila. IDB PSD AS is increasingly outsourced to consultants. Source: Interviews; EBRD (2004, 2007a). projects to local conditions; quicker decision- into "project-based" packages of financing to making; opportunities for more intensive local ca- be carried out by consultants. AS was thus some- pacity building; and personnel costs. On the other thing IDB funded but no longer "did." This left hand, headquarters management has some ad- recipient countries to deal with the problems vantages over field management: AS projects are of managing consultants, and they were often more likely to be aligned with the organization's overwhelmed. AS strategy, in part because field staff are over- loaded with operational tasks; and headquarters Funding and Pricing Policy management has the advantage of better trans- fer of lessons learned across projects. Funding Donors use several sources of funding for PSD AS: In-house vs. outsourced personnel i) multi-donor trust funds, such as PPIAF and Several donors rely heavily on consultants--larger DevCo; ii) single-donor trust funds, such as the consulting firms as well as individual consult- Japan Special Fund (JSF), an untied grant program ants--to deliver AS (table J.5). Among those us- of the Government of Japan; and, iii) internal ing this model are EIB, ADB, IDB, and the EBRD resources contributed by the donor organiza- Turnaround Management program. The draw- tion. The trend is in the direction of greater use backs of outsourcing have been recognized in of multi-donor trust funds to finance PSD AS. several evaluations of AS programs. This, for example, is the case for both IDB and ADB. For IDB, the composition of financing for A working group of the IDB found that AS had non-reimbursable technical cooperation changed gradually changed from being a source of advice significantly during 1990­2001. From 1990 to and assistance provided mainly by the IDB's staff 1994, the Fund for Special Operations (FSO) was 109 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 the principal source of funding (54 percent of basis. This is the case with EIB, for example. In ad- total nonreimbursable TC), followed by donor dition, bilateral donors, such as DANIDA, seldom trust funds (34 percent) and the Multilateral In- require cost recovery from the client. vestment Fund (12 percent). From 1995 to 2001, the FSO represented 32 percent, donor trust There are signs of some movement toward cost- funds 19 percent, and the MIF 49 percent. For sharing with the client, motivated both by the ADB, trust funds are now a major source of AS desire to increase client ownership, and by shrink- funding, amounting to 38 percent of AS funding ing donor budgets. For the EBRD TAM/BAS pro- in 2006. In contrast, most of IBRD/IDA's ESW and gram, the typical subsidy is 50 percent of the AS is funded by internal resources (85 percent in consultant cost, but some local BAS offices apply FY06), with only a small share (15 percent) fi- a different contribution ratio. For example, a lower nanced by trust funds. client contribution may be applied in order to in- crease the incentive for SMEs to use consultancy Most donors find that pooled financing improves services. For larger firms, the required contribu- coordination with client countries' national devel- tion might be greater than 50 percent. opment strategies, institutions, and procedures. One of the drivers for pooling is the 2005 Paris ADB addressed the issue of cost-sharing for AS op- Declaration, which sets a target for 50 percent of erations in 2005 under its "innovation and effi- AS flows to be coordinated behind national de- ciency initiative," which stated that the share of velopment strategies by 2010. AS operations in a country's overall portfolio to be financed by ADB would be agreed upon dur- Recently, some donors have contributed internal ing the preparation of the Country Partnership resources to supplement external sources of Strategy. Thereafter, the funding proposed for funds. For example, in 2007, for the first time, each AS project could vary, reflecting the sector EBRD provided e4.7m of the total of e15m mo- and objectives of the AS, provided the aggregate bilized for the TAM/BAS program. Since 2004, IFC portfolio ceiling is respected. Since then, ceilings has contributed $840 million from IFC-retained have been established for 13 countries, ranging earnings to the Funding Mechanism for Techni- from 80 percent to 99 percent. cal Assistance and Advisory Services (FMTAAS). IFC's pricing policy for AS has evolved toward Most donors experience similar trade-offs and requiring greater contributions from the client. tensions with respect to funding sources. For The current policy (as of January 2007) estab- single-donor trust funds, there may be tensions lishes the objectives and principles behind the re- between the funding organization and the recip- quirement of client contributions: building client ient organization in terms of sector or country pri- commitment, minimizing market distortions by orities. Planning distortions may result from funds avoiding crowding out private sector provision of being accessed for areas of activity outside the pri- services, and targeting subsidies to public goods. orities identified in country strategies, and mul- In practice, the policy has yet to motivate a sig- tiple administrative procedures from different nificant increase in client contributions. funding sources can add to the administrative costs of providing AS. Monitoring and Evaluation Systems Until recently, most donors did not subject their Pricing policy AS activities to rigorous M&E requirements. Few Provision of AS has often come as a "free good" donors required project completion reports or provided to the recipient. In particular, PSD AS that ex-post project evaluations, either from the is linked with the donor's core activity (e.g., prepa- managing unit or from the agency's evaluation de- ration for investments) is usually offered on a partment. Even the monitoring of AS for man- completely nonreimbursable (i.e., subsidized) agement purposes was made difficult by the fact 110 A P P E N D I X J : H I G H - L E V E L C O M PA R I S O N O F I F C A DV I S O RY S E R V I C E S W I T H OT H E R M D B s Table J.6. Monitoring and Evaluation Systems for PSD AS Organization Self-evaluation Independent evaluation IFC Recent introduction of Project Completion Note for AS IEG validation of PCRs Recent introduction of activity-based costing External evaluations of Facilities are conducted at the request of donors. Some external evaluations of AS activities have been conducted, mainly in the A2F and CA business lines. AfDB Technical Assistance Performance Reports (monitoring Operations Evaluation Dept. evaluations of selected TA during implementation) projects Technical Assistance Completion Reports (evaluation Occasional Special Evaluation Studies (latest SES on TA in 6 months­1 year after project completion) 2007). EBRD BAS project evaluations (output indicators) External evaluations commissioned by donors TAM project evaluations (including ratings) EBRD Evaluation Department evaluation of BAS in 2007 EBRD Evaluation Department evaluation of TAM in 2004 EC Ex-post evaluations are prepared on an annual basis at the Occasional thematic evaluations. Most recent PSD sector level (but PSD is not defined as a "sector"). evaluation prepared in 2005. IDB Nonreimbursable Technical Cooperation (TC) projects are not Occasional PSD thematic evaluations and MIF evaluations. currently included in IDB's Project Performance Monitoring Report System (PPMR) and Project Completion Report (PCR) system. For the Multilateral Investment Fund (MIF), the PPMR system, an annual report on project execution, was introduced in 2000. In general, however, the projects do not have impact evaluations or ex-post evaluations. Source: Donor interviews and websites. that AS costs were bundled with other activities, tors were mainly output-oriented from the design so it was not possible to report on them separately. stage, and usually baseline data is not collected. For donors like EIB who provide AS exclusively for the purpose of preparing and implementing in- Most evaluation departments prepare indepen- vestments, it may not be cost effective to require dent evaluations of PSD activities or AS activities separate AS evaluations. on an occasional basis. Among the better recent reports are the EBRD's evaluations of the TAM Partly because of the efforts of the MDB Evaluation and BAS programs, the IDB's MIF evaluations, Coordination Group, monitoring and evaluation and DANIDA's recent meta-evaluation of private systems for PSD AS have begun to improve and be- and business sector development interventions. come more consistent across donors. Most of the multilateral donors, including the IFC, now re- Results and Lessons Learned quire a project completion report for TA (table J.6). The main issue with these evaluations is their Findings from recent evaluations lack of focus on outcomes. This is due mainly to Independent evaluations of PSD AS activities the fact that the AS project's performance indica- were conducted recently for EBRD, ADB, EC, and 111 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 IBRD/IDA. A brief summary of the findings of all the firms assisted agreed that they were mate- these evaluations follows. rially closer to being profitable, stand-alone private companies than they would have been otherwise. EBRD Business Advisory Services. A 2007 eval- An issue of concern is that TAM is totally depen- uation of EBRD's Business Advisory Services (BAS) dent on donor funding, and the unreliability of this concluded that BAS projects were successful overall, funding threatens the program's sustainability, and were consistent with EBRD's transition im- constrains its ability to meet the demand for its ser- pact objectives. BAS consultants also have benefited vices, and reduces its efficiency. from involvement with BAS, not just financially but also in terms of capacity-building. However, the ADB Technical Assistance. A 2007 evaluation evaluation found that BAS impacts largely stop of ADB's technical assistance in all sectors found at enterprise level, and the population of BAS that nearly three-quarters of sampled TA proj- enterprises is small in the context of national ects, in five case-study countries, achieved or ex- economies. Benefits that accrue to consultants are ceeded their intended outputs. Executing agencies a by-product of the BAS process and are one-off reported that training had resulted in some im- rather than a targeted exercise in capacity-building. provement in staff performance and that recom- mendations had been partly acted on. The evaluation also found that overall true mar- ket development activities for the program were The evaluation also found that: scant. Establishing the link between number of · More needs to be done to improve coherence projects and market development is hampered by between lending and nonlending activities. loose program design and lack of verifiable indi- · Serious efforts need to be made to increase cators at the outset. When market development country ownership and, in appropriate cases, did take place, it was not part of a strategic ap- to delegate more authority and accountability proach to addressing the barriers to consultancy to EAs. market development. · More needs to be done to recognize in TA op- erations that there is a wide range of institu- In terms of demonstration effects, the evaluation tional capacity in Asia-Pacific countries and found that few BAS projects prove the case for across sectors within countries. ADB's current new, innovative or "atypical" types of consulting. one-size-fits-all approach to TAs needs to be In-depth interviews suggested that 87 percent reconsidered. of projects could be thought of as "standard," so · Isolated short-term inputs are not appropriate they would be unlikely to demonstrate the ben- in such areas as policy reform, change man- efits of new types of services. In addition, most BAS agement, and capacity-building. These require country programs make little attempt to dissem- longer-term interventions, assistance, or en- inate their results. gagement by ADB. · To improve process efficiency, AS approval and EBRD Turnaround Management. The 2004 administration procedures could be simplified. Turnaround Management evaluation reported that · While there was some evidence of coordination about 1,500 TAM projects were carried out be- with other funding agencies, in some cases tween 1998 and 2002 in all of the EBRD's countries there was also evidence of competition for of operations, except Turkmenistan, involving over specific types of AS projects, particularly be- million in donor funding. The evaluation 96 tween ADB and the World Bank. found that TAM has been highly successful. The majority of the companies visited acted on TAM's EC PSD activities in third countries. A 2005 advice and made significant changes to their evaluation of the EC's support for private sector businesses. The vast majority of companies visited development in third countries was quite critical reported higher capacity utilization, labor pro- of EC's PSD interventions. It found that: i) pro- ductivity, sales, market share, and profits. Nearly gram objectives were not systematically geared 112 A P P E N D I X J : H I G H - L E V E L C O M PA R I S O N O F I F C A DV I S O RY S E R V I C E S W I T H OT H E R M D B s toward achieving the objectives stipulated in the · Broader and more sustainable results are ob- EC PSD strategy; ii) key constraints bearing on suc- tained from interventions at the macro and cess were not sufficiently addressed; iii) most meso level rather than the micro level. Firm- meso- and micro-activities lacked sufficient out- level support is low in outreach, which makes reach and were not targeted on the most adequate it difficult to achieve broader PSD impacts be- beneficiaries, and, iv) lessons from the past were yond the beneficiary firms. inadequately taken into account. · Interventions at all levels should be targeted more at local market deficiencies identified by AS has generally weak performance on efficiency. an assessment of the actual conditions. This ap- There is a lack of transparency regarding how plies to the policy and regulatory framework, much AS programs cost, whether the benefits public and private institutions, and markets. justify the expenditure, and whether donors are Some progress has been made by developing getting value for money. tools for assessing the business environment, but more needs to be done to develop method- IBRD/IDA TA and ESW. The recent IEG-Bank ologies for assessing the quality of institutions evaluation of the World Bank's TA and economic and the functioning of markets. and sector work in all sectors concluded that · Interventions to improve the business envi- most ESW and TA met their stated objectives to ronment should be encouraged, as long as at least an average extent, although their effec- there is sufficient government commitment. tiveness was greater in shaping Bank lending and Support to intermediary organizations can be strategy than in providing support directly to a way of influencing public policy for the pri- client countries. The indirect effects of ESW and vate sector. TA on client countries--through Bank lending-- · Long- or short-term support within broader were greater than the direct effects. Between 65 programs, leads to better and more sustainable percent and 80 percent of users of Bank ESW outcomes. and TA in client countries gave ratings of average · Despite the fact that there is no one-size-fits- and above on the extent to which ESW and TA met all approach to PSD interventions, it is impor- their stated objectives; between 74 percent and tant to adopt a methodical procedure for 87 percent of such users in the Bank (task team selecting areas of intervention in a country, leaders for loans and strategies) gave such ratings. which should at least include the following steps: a critical assessment of the priority areas In the PSD sector, Investment Climate Assess- of interventions, selecting an area in which the ments (ICAs) were most often named by survey donor has a comparative advantage, and an respondents as having informed policies. In assessment of whether the preconditions for Malaysia, changes in the labor law and in the reg- intervening in a given area have been met. istration of property were attributed to the ICA. · Assumption of ownership, involvement of local It has also led the government to establish a com- actors, and building of institutions in recipient mittee to ensure that deregulation and improve- countries on the basis of the transfer of regu- ments in public service delivery were carried out latory, facilitation, and intermediation compe- smoothly. In Serbia, the ICA was credited with the tencies is a necessary condition for sustainability. country's regaining momentum in the privatiza- tion process and in attracting foreign investment, Conclusions: IFC's Relative Strengths among other changes. In Guyana, the ICA was and Comparative Advantages cited as having informed the country's National Compared with other donors that provide PSD AS, Competitiveness Strategy. IFC appears to have the following strengths: · Well-designed diagnostics. The IFC, along Lessons learned with other units in the Bank Group, has been Some common lessons have emerged from in- a leader in developing quantitative indicators dependent evaluations of PSD AS: of the quality of the investment climate, the ease 113 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 of doing business, and the ability of firms to ac- EIB, EBRD, the IDB's IIC, and IFC; those that lend cess finance. These efforts provide the means to governments, like IBRD/IDA and the regional of assessing initial conditions in client countries development banks; and bilateral donors that do to guide Advisory Services design, as well as not lend or invest directly--it is possible to pro- allow for evaluation of results. They have been pose areas in which IFC may have advantages rel- appreciated by client countries and are used by ative to other donors in the delivery of AS: other donors as well. · A strong "matrix" of headquarters and field of- · Global knowledge: The ability to mobilize the fices that allows for synergies between staff best global expertise in specialized areas, along with specialized expertise and those with local with knowledge of international best practice knowledge. can be persuasive with clients. · Strong analytical capacity within the World · Pricing policy: Although some donors (no- Bank Group, giving IFC a potential compara- tably EBRD and ADB) have made progress in tive advantage in Advisory Services strategy defining cost-recovery policies, the IFC is rel- and project design. atively advanced in its thinking in this area. · Investment and lending operations that can · Monitoring and evaluation: Most donors be linked with Advisory Services, helping to im- do a poor job of separating AS from other ac- prove the performance of both types of activ- tivities for purposes of monitoring, defining ities (although this is an advantage shared with performance indicators for AS, and conducting EBRD, EIB, and the IDB's IIC). ex-post evaluations. Although some donors · Ability to take a leadership role in coordinating have begun to adopt better M&E systems for PSD AS among donors, in part because of its AS (again, EBRD and ADB), IFC is probably global presence and also because it receives ahead in implementing the system. funding from many of the same donors. Like most donors, IFC's weaknesses mostly relate The other side of the coin is that IFC does not have to the divergence between strategy and practice. a comparative advantage, relative to other donors, AS are often selected on an ad hoc basis rather than in some areas: being closely aligned with country and sector · Macroeconomic policy, in which IBRD/IDA, the strategies. At the same time, synergies across the IMF, and some of the regional development World Bank Group and with other development banks have greater analytical capacity and more partners are not fully exploited. The recently- appropriate instruments. adopted pricing policy has not resulted in a sig- · Some meso-level interventions, in particular in- nificant increase in client contributions--the share stitutional development, for which the regional of projects with a client contribution has increased development banks tend to have a greater un- only slightly since the policy was adopted. And derstanding of country context and better part- although the M&E system for Advisory Services nerships with clients. establishes monitoring, self-evaluation, and in- · Longer-term capacity-building, which many bi- dependent evaluation processes, the usefulness lateral donors are better able to provide. of the system is limited by the quality of perfor- · The direct provision of advisory services in mance indicators being used. As it stands, most in- countries where markets for these services are dicators measure outputs (at best), not outcomes, relatively well-developed. With the exception and baseline data is rarely collected. of low-income and post-conflict countries, direct support may not add value, can crowd Looking across all types of donors--those that out private providers, and can give beneficiaries lend and invest directly to the private sector, like an unfair advantage over their competitors. 114 ENDNOTES Executive Summary 3. En general, al vencimiento operativo anticipado 1. IFC Articles of Agreement: Article 1--Purpose. las operaciones han registrado no menos de 18 meses 2. IEG 2008c. de ingresos operativos, lo que habitualmente ocurre 3. At early operating maturity, operations have gen- cinco años después de la aprobación. erally recorded at least 18 months of operating revenue, 4. Estas modalidades de resultados entre distintas which is typically five years after approval. regiones y sectores son en general congruentes con las 4. These result patterns across regions and sectors autoevaluaciones de la IFC, aunque con cierto sesgo are broadly consistent with IFC's own self-assessments, optimista en las calificaciones autoadjudicadas, que en although with some optimism bias in self-ratings, which promedio fueron 5% más altas que las asignadas por were, on average, 5 percent higher than those assigned el IEG. by IEG. 5. En muchas oficinas en los países, fuera de los cen- 5. In many country offices, outside regional hubs, tros regionales, los funcionarios de servicios de asesoría IFC Advisory Services staff significantly outnumber de la IFC, cuyo número es considerablemente mayor investment officers and are the face of IFC in the que el de los oficiales de inversiones, son la cara visi- country. ble de la IFC en el país. Résumé Analytique 1. Statuts de l'IFC, Article I. Arabic Executive Summary 2. IEG 2008c. . .i 3. Lorsqu'elles atteignent leur régime de croisière, .2008c ii les opérations ont généralement eu au moins 18 mois .iii de recettes d'exploitation, habituellement cinq ans 18 . après l'approbation de l'intervention. .iv 4. Ce schéma de résultats dans les régions et les sec- teurs correspondent en aux évaluations internes de l'IFC, encore qu'on constate un certain biais optimiste 5 dans les auto-appréciations, de 5 % plus élevées en . .v moyenne que les appréciations de l'IEG. 5. Dans nombre de bureaux extérieurs, centres ré- gionaux exceptés, le personnel des services-conseil de . l'IFC est en nombre nettement plus élevé que les spé- cialistes de l'investissement, et c'est l'IFC qui est perçu comme représentant la Société dans le pays. Advisory Panel Statement 1. This apparently was the case for 89 out of 289 Resumen Ejecutivo operations (excluding BEE projects) between Janu- 1. IFC, Articulo 1 del Convenio Constitutivo. ary 2007 and January 2008--or roughly one-third of the 2. IEG 2008c. projects. 115 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 Chapter 1 Chapter 2 1. For further discussion of the empirical connec- 1. To some extent this reflects the replacement of tions between knowledge and development, see World existing clients with new clients (rather than necessarily Bank Institute 2008. a preference for existing clients). 2. World Bank database. 2. Objectives with trackable data included: whether 3. World Bank database. the project was in an IDA country; in a strategic sector 4. World Bank database. (infrastructure, financial markets, health and educa- tion, or agribusiness); or was south-south in nature. 5. Again, this figure increases a little if advice in the 3. The pattern is similar by volume of activities. Financial and Private Sector Development area is 4. In line with the MDB good practice standards of included. the evaluation of private sector investment operations, 6. From the 2007 annual reports of the respective this review concentrates on the results of projects that development banks. were evaluated in the last three years. AFDB: Private sector approvals in 2007 rose to UA 5. Self-ratings by investment officers were, on [unit of account] 1 billion (1.67 total approval volume) average, 5 percent higher overall than those assigned so the share of private sector investments reached to by IEG. 60 percent, compared with UA 278.5 million in 2006 6. See Independent Evaluation Group 2007b for (1.05 total volume), or 47 percent of total lending. further details on performance of IFC-supported proj- ects in 2005. ADB: In 2006 ADB adopted a new medium strategy, 7. Project evaluations in late 2008 have been able to which places catalyzing private sector investment as its incorporate the possible effects of the crisis in their pro- highest priority. In 2007, the private sector operations jections going forward, since these projects were sub- totaled $1.7 billion (out of $10.1 billion), or 17 percent, stantially implemented at the time of the crisis (they significantly above recent levels. were approved in 2003), thus the crisis effect is less EBRD: The private sector share of annual business marked than for projects approved more recently. volume increased to 86 percent in 2007, from 80 per- 8. See, for example, Asian Development Bank 2007a. cent in 2006. 9. The evaluated sample was small (six projects), but the ratings were generally consistent with those de- IDB: During 2007, the IDB approved 17 non-sovereign- termined in a recent health sector study carried out by guaranteed transactions, consisting of 13 loans and IEG, which also found an improving trend in sector 4 guarantees, totaling $2.1 billion (out of $8.97 billion) performance. 23 percent of total lending. During 2006, the Bank ap- 10. Independent Evaluation Group 2009. proved 20 private sector transactions totaling $920 mil- 11. See Sharpe, Alexander, and Bailey 1995. lion for projects, or 14 percent of total lending ($6.4 12. IEG evaluates IFC's E&S work quality in a proj- billion). In December 2006, the Board of Executive ect (appraisal, supervision, and role & contribution) sep- Directors approved changes to the Bank's basic arately from IFC's overall work quality. organization directed at improving the Bank's opera- 13. As opposed to an explicit trade-off between tional efficiency and capacity to fulfill its fundamental profitability and project development impact. purpose. The changes include the creation of a new 14. The Asian crisis, for example, can be isolated as Vice President for Private Sector and Non-Sovereign- a primary reason for the significant deterioration of de- velopment, business, and investment outcomes for Guaranteed Operations. projects approved in the mid-1990s. 7. World Bank estimates suggest that between 130 15. Of 37 projects approved in the three years and 155 million people fell into extreme poverty as a following a crisis in major MICs (Brazil, Indonesia, result of higher food prices. Korea, Mexico, the Philippines, the Russian Federa- 8. See World Bank 2008c. tion, and Turkey), 67 percent achieved high develop- 9. See World Bank 2008c and 2008d. ment results (compared with 61 percent otherwise). 10. See, for example, Reinhart and Rogoff 2008. Projects in Brazil, Korea, and Russia were particularly 116 E N D N OT E S successful. In contrast, the performance of the 96 eval- peers in a number of familiar business indicators, such uated projects that were already under way when a as shareholder return (by an approximate ratio of 2:1). crisis hit (in Argentina, Brazil, Indonesia, Mexico, the 11. Prior to this time, organization of AS was some- Philippines, Russia, Thailand, Turkey, and Uruguay) what ad-hoc, and dependent on how each facility was were much weaker. Of these projects, 54 percent set up. In 1997, for example, IFC's AS work was de- achieved high development outcome ratings, com- scribed as being: feasibility and prefeasibility studies; pared with 64 percent for noncrisis exposed projects. project identification studies; strengthening the en- 16. This is the share of active investment approvals abling environment for private sector development; or between 2005 and 2007 (704 projects with $20 billion capacity building for private businesses and government net commitment), relative to the active projects in officials. portfolio in June 2008 (1,716 projects with $33 billion 12. See Independent Evaluation Group 2007b. net commitment). 13. Regional facilities and global business units are also referred to as donor-funded operations. Chapter 3 14. Additionally, some business lines have stronger 1. For further discussion of the empirical connec- links with Washington. Staff working on Infrastructure tions between knowledge and development, see World Advisory Mandates, for example, tend to have closer ties Bank Institute 2008. See also Dosi, Teece, and Chytry to Washington and to the investment stream than to other advisory business lines--and projects can some- 1998. times proceed without much engagement with the 2. See, for example, Lewis 2004. See also Stewart main regional facility. 2002. 15. Entry--new products/approaches being 3. For a fuller discussion of respective roles of gov- introduced/ tested in single clients/single markets with ernment and the private sector in knowledge genera- no or limited results measurement to date. IFC also may tion and exploitation, see World Bank 1999. have limited internal expertise in this product area 4. See Dahlman and Westphal 1981 for more on at entry but must have a senior IFC staff person iden- how markets are imperfect institutional devices for fa- tified as the leader of this work. Products should not re- cilitating trading in many kinds of technological and main in the Entry category for more than 24 months or managerial know-how. for two subsequent Product Reviews. Products may 5. See Contractor and Nejad 1981, Arrow 1971, and move from Entry to In Development, Entry to Other, McCulloch 1981. or Entry to Exit. Products that are currently in the Other 6. Independent Evaluation Group 2008c. category may move to Entry if there is broad imple- 7. It should be noted that in FY07, IFC's invest- mentation or plans to replicate across multiple regions. ment commitments made up about a half of MDB fi- In-Development--products that have growing de- nancing for private sector operations in developing mand, high potential for scaling up and replication countries. across markets, and have some results that provide 8. See International Finance Corporation 2008a and evidence to continue IFC's investment in and delivery IFC Strategic Directions, FY08­10 and FY07­09. of such products. IFC should have some in-house ex- 9. In FY08, IS employed 1,538 staff and 706 con- pertise in this area. Products should not remain in the sultants. IS consultants also tend to be paid consider- In-Development category for more than 36 months or ably more than those used for AS, implying that they for three subsequent Product Reviews. Products may are brought in to carry out tasks that require greater move from In Development to Developed, In Devel- skill and experience. opment to Other, In Development to Exit. Some prod- 10. At a more general level, the Global Most Admired ucts currently in this category may require longer to Knowledge Enterprises (MAKE) rankings consider and mature. In such cases the products should be moved rank organizations according to factors that include into the Entry category, which did not exist in the orig- organizational learning, innovation, and creation of a inal product review. This would give the product up to corporate knowledge-driven culture. In 2008, McKin- 5 years to reach Development. sey, Google, and Royal Dutch Shell were the strongest Developed--products that have been scaled up performers. MAKE winners typically outperform their and replicated across at least three regions and have un- 117 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 dergone some form of rigorous results measurement 19. It should be noted that no new approvals are per- activity, such as experimental or quasi-experimental mitted for products placed in the Exit category. design conducted by an external party. The results sup- 20. See OECD website (www.oecd.org) for full dec- port continued work in these areas and new projects laration. In September 2008, a High-Level Forum of should reflect lessons learned in the design. These Ministers from over 100 countries, heads of bilateral and products should be appropriate for implementation multilateral agencies, donors and many international sol- in frontier markets and IFC should have highly experi- idarity organizations was convened in Africa to follow enced, senior staff leading product development in up on the Paris Declaration. Among other things, the these areas. Products may remain in this category for forum concluded that aid fragmentation remained a an indefinite period of time. Products may move from major challenge, and that aid partnerships should be en- Developed to Exit. couraged, in line with the Paris Declaration principles. Exit--Products will be moved to the exit category 21. It should be noted that business line leaders are for a variety of reasons. Some may exit as demand and not invited to take part in developing CASs. donor/partner interest declines signaling that the key 22. Based on a review of publicly available strategies, work has been completed, or priorities are shifting evaluation reports, and interviews with representa- to other areas. Similarly, IFC may exit a product when tives of the European Bank for Reconstruction and other parties become available to provide the same Development (EBRD), the Asian Development Bank product as well or better than IFC, or when IFC no (ADB), the Inter-American Development Bank (IDB), longer has sufficient competence in the area (e.g., loss the African Development Bank (AfDB), the European of product leaders/specialists). Other products may be Commission (EC), and the European Investment Bank exited based on our inability to achieve desired re- (EIB)--and two bilateral donors: the U.K. Department sults, cost recovery and/or scale/efficiency. for International Development (DFID) and the Danish Other--This category is for idiosyncratic products International Development Assistance (DANIDA). that are appropriate to a particular country/market at 23. IFC 2008b. a given point in time but are not expected to reach scale 24. Based on expenditures in June 2008, overall or be replicated broadly. Products may remain in the leverage of IFC: donor funds was approximately 1:1.5. "Other" category for an indefinite period of time as long 25. See Independent Evaluation Group 2007a, for as desired results and cost recovery are achieved and a more detailed discussion of the pros and cons of the product is NOT implemented in more than two re- this funding approach. gions. Applying this definition to current products will 26. An example of procedural constraint is the result in movement of several products from this cat- need for a government to tender competitively for a egory to Entry or Exit. fee-based service, no matter how small in value. 16. The 1997 IFC Annual Report, somewhat less 27. Eleven percent in the former, as opposed to 2 specifically, defined IFC's AS products as either: feasi- percent in the latter. bility and prefeasibility studies, project identification 28. See International Monetary Fund 2008. studies, strengthening the enabling environment for pri- 29. See, for example, IEG 2007a. vate sector development, and capacity building for pri- 30. It should be noted that the Latin America and vate businesses and government officials. the Caribbean region has introduced project approval 17. Two notable exceptions were IEG reviews of decision meetings, similar to that used for BEE, in- four SME facilities in 2005, and the IEG review of the volving Bank staff and peer reviewers. This approach Private Enterprise Partnership in 2007. The review of has been applied in recent months to the CA and ESS the Africa Project Development Facility contributed to business lines. the understanding that working directly with a small 31. IEG 2007a. number of SMEs was relatively costly and that it would 32. It should be noted that the Infrastructure busi- generally be more efficient to work with a larger num- ness leader appears to the exception in this sense. ber of SMEs on more of a wholesale basis. 33. The impact evaluations that have been carried 18. Prior to 2006, different facilities and business out, or have recently been commissioned, typically units had their own, separate M&E approaches and had one of two aims: (1) to evaluate pilot projects systems. prior to roll-out and replication, and (2) to evaluate proj- 118 E N D N OT E S ects that require testing several approaches to identify 37. It is ultimately a decision for IFC management which is most effective. on how to allocate its resources for impact evalua- 34. The latter has been included as a quality di- tions, but care needs to be taken not to overexamine mension since 2008. some topics and leave others underresearched. In an 35. IFC's Results Measurement Network's own qual- ideal setting, as IEG's Annual Review of Development ity review in early 2008, of supervision and completion Effectiveness 2008: Shared Global Challenges pointed documents for projects approved between December out, the decision to fund impact evaluations in a given 2005 and December 2007, had similar findings: barely area would take into account the following five crite- half of supervision documents evidenced clear under- ria: i) the value of answering the question in terms of standing of outcomes and impacts, with persistent benefits and costs of a specific project, ii) the value of problems being a lack of baseline data reporting, lim- answering the question for other current or future ited data tracking, and low use of standardized indica- projects, iii) the cost of the evaluation, iv) the innova- tors; and weak data/evidence to support completion tive nature of the project, and v) the likely feasibility of report ratings, frequent use of "too early to tell" when designing a convincing impact evaluation. outcomes could have been observed, and overly opti- 38. See IEG 2008a for a more detailed discussion of mistic development effectiveness ratings. the quality and coverage of M&E systems in IFC, in- 36. Building on this definition and drawing on the cluding at the programmatic level. good practice standards of official audit and evaluation 39. For the full IFC Corporate Scorecard, see IFC agencies, four dimensions of evaluation independence 2008a. have been recognized by the MDB Evaluation Coop- 40. In the latter case, the development interests of eration Group: donors (external or internal) and IFC IS are typically well (i) Organizational independence--It ensures that aligned, although as an investor IFC will also need to the evaluation unit and its staff are not under the consider balance sheet impact, which poses a conflict- control or influence of decisionmakers who have re- of-interest risk if such interests supersede develop- sponsibility for the activities being evaluated and that ment goals. they have full access to the information they need 41. As of January 1, 2009, IFC has its own indepen- to fulfill their mandate. dent Conflicts Office, which has issued IFC-specific directives and guidelines to address AS/IS business (ii) Behavioral independence--It measures the conflicts. extent to which the evaluation unit is able and will- 42. Internal IEG document. ing to produce high quality and uncompromising 43. The PEP-ECA study (IEC 2007a) found a similar reports and to disclose its findings to the Board pattern. without Management-imposed restrictions. 44. Internal IEG document. (iii) Protection from outside influence--This refers 45. See, for example, IEG 2007b. to the evaluation unit's ability to decide on the de- 46. The scope of the review did not extend to pri- sign and conduct of evaluations without interfer- vate consultancy firms involved in the delivery of knowl- ence; its control over staff hiring, promotion, and edge services in developing countries, such as PwC firing within a merit system; and its access to ade- and DAI. quate resources to carry out the mandated re- 47. These include a one-off look at the AS market sponsibilities effectively. in 2007; and benchmarking of IFC linkages operations. (iv) Avoidance of conflicts of interests--It guaran- 48. For more detail on patterns in official aid flows, tees that current, immediate future, or prior see World Bank 2008a. professional or personal relationships and consid- 49. In 21 of 64 cases. erations are not allowed to influence the evaluators' 50. The share of projects without development ef- judgments or create the appearance of a lack of fectiveness and impact ratings is fairly consistent across objectivity. Specific criteria were developed by the business lines. Evaluation Cooperation Group to measure the de- 51. A large proportion of ratings of cannot tell re- gree of independence along these four dimensions. flects frequent changes of indicators during project 119 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 implementation in most business lines as a result of 57. Using t-tests of statistical difference, at a 95 per- M&E staff efforts to standardize indicators. It has been cent level of confidence. observed that task leaders abandoned the initial set of 58. It should be noted that size was found to be an indicators, often set intuitively to reflect the project goals important explanatory variable in the case of PEP-ECA. and objectives and adopted newly established stan- 59. For further elaboration, see IEG 2007a. dard indicators, which either could not be measured, 60. Other multilateral development banks involved given that the change occurred during implementation, are ADB, AfDB, EBRD, IDB, and the Islamic Develop- did not have baseline data, or did not appropriately re- ment Bank. flect the goals of the specific project. Although stan- dardization of indicators is desirable, in some cases it Chapter 4 led to confused reporting of project results. 1. These result patterns across regions and sectors 52. This issue was first raised in IEG's FY02 annual are broadly consistent with IFC's own self-assessments, review, which was completed and submitted to CODE although with some optimism bias in self-ratings, which in early 2003, and has been a recurring theme in IEG were, on average, 5 percent higher than those assigned annual reviews since then. See, for example, IEG 2007b by IEG. and 2008b. 53. The majority of ESS operations have been man- Appendix D aged from headquarters. 1. IFC 1999, p. 29. 54. Seventy-nine percent of BEE operations in high- 2. The historical likelihood of default as ranked risk IDA countries, which made up nearly a half of re- by Moody's for example shows that over a normal five- viewed operations in these countries, were rated high year period only 0.1 percent of AAA US corporate bonds on development effectiveness. default (see Credit and Default Risks, available at: 55. Based on IEG and Development Outcome Track- http:// personal.fidelity.com/products/fixedincome/risks. ing System data. shtml. 56. In these cases, IFC role and contribution was rated high 95 percent of the time. 120 BIBLIOGRAPHY African Development Bank. 1997. African De- and Corporate Change. Oxford: Oxford Uni- velopment Report: Fostering Private Sector De- versity Press. velopment in Africa. Abidjan: African Douglas, D., and James Walsh. 2004. Study of the Development Bank. Efficiency and Effectiveness of the African ------. 2004. Private Sector Development Strat- Development Bank Technical Assistance Fund: egy. Tunis: African Development Bank. Final Report. Abidjan, Côte d'Ivoire: African Arrow, K. 1971. Essays in the Theory of Risk Bear- Development Bank. ing. Chicago, IL: Markham. European Bank for Reconstruction and Devel- Asian Development Bank. 2000. Private Sector De- opment. 2004. Turn Around Management velopment Strategy. Manila: Asian Develop- Programme. Special Study. London: EBRD. ment Bank. ------. 2006. Micro, Small, and Medium Enter- ------. 2007a. Annual Report, Portfolio Per- prises Strategy. London: EBRD. formance. Operations Evaluation Department. ------. 2007a. Business Advisory Services Pro- Manila: Asian Development Bank. gramme. Special Study. London: EBRD. ------. 2007b. Special Evaluation Study on the ------. 2007b. Turn Around Management (TAM) Performance of Technical Assistance. Opera- and Business Advisory Services (BAS) Strate- tions Evaluation Department. Manila: Asian gic Plan, 2008­10. London: EBRD. Development Bank. ------. 2008. Building Stronger Businesses: The ------. 2008. Increasing the Impact of the ADB's Turn Around Management and Business Ad- Technical Assistance Program. Manila: Asian visory Services Programme. London: EBRD. Development Bank. European Centre for Development Policy Man- Contractor and Nejad. 1981. "International Tech- agement. 2007. Study on the Provision of Tech- nology Transfer: Major Issues and Policy Re- nical Assistance Personnel: What Can We Learn sponses." Journal of International Business from Promising Experiences? Brussels: ECDPM. Studies 12 (2):113­35. European Commission. 1998. A European Com- Dahlman, C., and Larry Westphal. 1981. "The munity Strategy for Private Sector Develop- Meaning of Technological Mastery in Relation ment in African, Caribbean, and Pacific to Transfer of Technology." Annals of the Acad- Countries. Brussels: European Commission. emy of Political and Social Science No. 458, ------. 2003a. European Community Co- (Nov). operation with Third Countries: The Com- Denmark, Ministry of Foreign Affairs. 2004. mission's Approach to Future Support for the Meta-Evaluation: Private and Business Sec- Development of the Business Sector. Brussels: tor Development Interventions. Copenhagen: European Commission. DANIDA. ------. 2003b. Guidelines for European ------ 2007. Synthesis of Evaluations on Tech- Commission Support to Private Sector De- nical Assistance. Copenhagen: DANIDA. velopment, Version 1. Brussels: European Dosi, Giovanni, David J. Teece, and Joseph Chytry Commission. (eds.). 1998. Technology, Organization, and ------. 2005. Evaluation of European Com- Competitiveness: Perspectives on Industrial munity Support to Private Sector Development 121 I N D E P E N D E N T E V A L U AT I O N O F I F C ' S D E V E L O P M E N T R E S U LT S 2 0 0 9 in Third Countries. Brussels: European Com- International Monetary Fund. 2008. Enhancing mission. the Impact of Fund Technical Assistance. Of- European Investment Bank. 2006. Facility for fice of Technical Assistance Management. Wash- Euro-Mediterranean Investment and Part- ington, DC: IMF. nership (FEMIP) Trust Fund: Evaluation of Ac- Lewis, W.W. 2004. The Power of Productivity: tivities at 30.09.2006. Brussels: EIB. Wealth, Poverty and the Threat to Global Sta- Inter-American Development Bank. 2003a. A Re- bility. Chicago, IL: University of Chicago Press. view of Selected Non-Reimbursable Techni- McCulloch, Rachel, 1981, "Technology Transfer cal Cooperation: Guyana, Nicaragua and to Developing Countries: Implications of In- Bolivia. Washington, DC: IDB. ternational Regulation," Annals of the Amer- ------. 2003b. Synthesis of OVE Evaluations of ican Academy of Political and Social Science, Bank Action for Private Sector Development. 458 (Nov):110­22. Washington, DC: IDB. Organization for Economic Cooperation and De- ------. 2004. Private Sector Development Strat- velopment. 1994. Development Assistance egy. Washington, DC: IDB. Committee Orientations for Development Independent Evaluation Group (IEG). 2007a. IFC Cooperation in Support of Private Sector De- Advisory Services in Eastern and Central Asia: velopment. Paris: OECD. An Independent Evaluation of the Private ------. 1995. Support of Private Sector Develop- Enterprise Partnership Program. Washington, ment. Paris: OECD. DC: World Bank. Reinhart, Carmen M., and Kenneth S. Rogoff. ------. 2007b. Independent Evaluation of IFC's 2008. "Is the 2007 U.S. Subprime Crisis So Development Results 2007. Washington, DC: Different? An International Historical Com- World Bank. parison." American Economic Review 98 (2): ------. 2008a. IEG's Biennial Report on Opera- 339­44. tions Evaluations. Washington, DC: World . Schulpen, L., and P Gibbo. 2002. "Private Sector Bank. Development: Policies, Practices and Prob- ------. 2008b. Independent Evaluation of IFC's lems." World Development 30 (1):1­15. Development Results 2008. Washington, DC: Sharpe, William., Gordon Alexander, and Jeffrey World Bank. Bailey. 1995. Investments. New Jersey: Pren- ------. 2008c. Using Knowledge to Improve De- tice Hall, chapters 8, 11, and 25. velopment Effectiveness: An Evaluation of World Bank Economic and Sector Work and Stewart, Thomas A. 2002. The Wealth of Knowl- Technical Assistance, 2000­06. Washington, edge: Intellectual Capital and the Twenty- DC: World Bank. First Century Organization. London: Nicholas ------. 2009. Improving Effectiveness and Out- Brealy Publishing. comes for the Poor in Health, Nutrition, and United Kingdom, Department for International Population: An Evaluation of World Bank Development. 2006a. Developing Capacity? Group Support since 1997. Washington, DC: An Evaluation of DFID-Funded Technical World Bank. Co-operation for Economic Management in International Finance Corporation (IFC). 1999. Sub-Saharan Africa, Synthesis Report Vol. 1. Project Finance in Developing Countries I. London: DFID. Washington, DC: World Bank. ------. 2006b. How to Provide Technical Coop- ------. 2008a. IFC Roadmap 2009­11. Wash- eration Personnel. London: DFID. ington, DC: IFC. World Bank. 1999. World Development Report ------. 2008b. "Advisory Services," Presentation 1998/99: Knowledge for Development. New to Executive Directors, December. http:// York: Oxford University Press. advisoryservices.ifc.org/uploads/documents. ------. 2002. Private Sector Development Strat- ------. IFC Strategic Directions, 2008­10 and egy: Directions for the World Bank Group. 2007­09. Washington, DC: World Bank. 122 BIBLIOGRAPHY ------. 2008a. Aid Architecture: An Overview of ------. 2008c. Global Economic Prospects 2009. the Main Trends in Official Development As- Washington, DC: World Bank. sistance Flows. Washington, DC: World Bank. ------. 2008d. Outlook for Remittances Flows ------. 2008b. Using Knowledge to Improve De- 2008­10. Washington, DC: World Bank. velopment Effectiveness: An Evaluation of World World Bank Institute. 2008. Measuring Knowledge Bank Economic and Sector Work and Techni- in the World's Economies. Washington, DC: cal Assistance, 2000­06. Independent Evalua- World Bank. tion Group. Washington, DC: World Bank. 123 IEG Independent Evaluation Group Independent Evaluation Group-IFC E-mail: AskIEG@ifc.org Telephone: 202-458-2299 Facsimile: 202-522-0931 ISBN 978-0-8213-7986-8 SKU 17986