THE WORLD BANK W O R L D B A N K O P E R A T I O N S E V A L U A T I O N D E P A R T M E N T Sharing Knowledge Innovations and Remaining Challenges An OED Evaluation OPERATIONS EVALUATION DEPARTMENT S H A R I N G K N O W L E D G E INNOVATIONS AND REMAINING CHALLENGES A N O E D E VA LUAT I O N CATHERINE GWIN 2003 THE WORLD BANK http://www.worldbank.org/oed WASHINGTON, D.C. © 2003 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone 202-473-1000 Internet www.worldbank.org E-mail feedback@worldbank.org All rights reserved. 1 2 3 4 05 04 03 The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Cover photos: World Bank Photo Library. ISBN 0-8213-5712-3 eISBN 0-8213-5713-1 Library of Congress Cataloging-in-Publication Data has been applied for. CONTENTS Acknowledgments vii Foreword ix Abbreviations and Acronyms xi Executive Summary xiii 1. Introduction 1 Knowledge Initiative Launch 1 Evaluation Design 2 2. A Highly Relevant Strategy 5 Discerning the Bank's Knowledge Strategy 5 The Strategy's Strengths 6 The Strategy's Design Shortcomings 7 3. Internal Knowledge Sharing to Improve Bank Operations 9 Expanding Network and Regional Knowledge Sharing among Staff 9 Thematic Groups 10 Advisory Services 14 Expanded Use of Bank Internal and External Web Sites 16 Linking to Learning and Research 17 Remaining Challenges 17 4. Sharing Knowledge with Clients and Partners 19 Regional Approaches to Leveraging Knowledge for Development 19 Increasing the Knowledge Dimensions of Country Programs and Projects 20 Effectiveness of Bank Knowledge Sharing with Clients 22 Development and Integration of Global Knowledge Initiatives 23 Integrating Knowledge Sharing 32 5. The Supporting Institutional Infrastructure 33 Resources and Governance 33 Technology 36 People 37 Summing up 37 iii iv | S H A R I N G K N O W L E D G E 6. Conclusions and Recommendations 39 Endnotes 95 Bibliography 101 Annexes Annex A: Knowledge-Sharing Chronology (1996-02) 41 Annex B: Inventory of Bank Knowledge-Sharing Activities 43 Annex C: Lessons from the Literature on Knowledge Management 45 Annex D: Evaluation Framework for Knowledge-Sharing Programs and Activities: Criteria, Inidicators and Evidence 49 Annex E: Methodology Note 51 Annex F: Examples of Comparator Knowledge Initiatives 55 Annex G: Client Survey Report 59 Annex H: Management Response to the OED Review Sharing Knowledge: Innovations and Remaining Challenges 81 Annex I: Chairman's Summary: Committee on Development Effectiveness 91 Boxes Box 3.1: A Consultative Process for Validating Content 11 Box 3.2: Task Teams Welcome Direct Support of Project Implementation by Thematic Groups 12 Box 3.3: There Is No Shared Practice for Budgeting, Programming, and Reporting of Thematic Group Actions 13 Box 3.4: Free and Fee-Based Access to Knowledge Through Rapid Response 14 Box 3.5: Targeting Web Content to Intended Audiences Poses Challenges 17 Box 4.1: A Continuous Learning Framework Provides for Exchange among Project Teams 20 Box 4.2: Shifting the Emphasis from Lending to Knowledge 21 Box 4.3: GDLN Programming Links City Officials 27 Box 4.4: GDN Major Activities since 1999 31 Box 5.1: Conditions for the Effective Use of Shared Knowledge 37 Tables Table 2.1: Stylized Results Chain for Knowledge Sharing 8 Table 4.1: Location of Global Knowledge Initiatives 25 Table 4.2: World Bank Lending for GDLN Distance Learning Centers 27 Table 4.3: Mixed GDLN Trends, FY01­mid FY03 28 Figures Figure 2.1: Increasingly Favorable Staff Views on Access to Knowledge 6 Figure 2.2: Clients and Experts Give Bank Knowledge High Ratings on Quality and Relevance 7 Figure 3.1: Thematic Group Activities Provide More Indirect than Direct Support to Operations 11 C O N T E N T S | v Figure 3.2: External Requests Dominate Advisory Services 15 Figure 3.3: Most Requests Are for Publications and Data 15 Figure 4.1: Shared Knowledge Is Well Focused, but Not Well Adapted or Disseminated 22 Figure 4.2: Where Clients Would Like the Bank to Do Better 24 Figure 4.3: GDN Expenses by Core Activities (FY02) 30 Figure 5.1: Bankwide Expenditures on Knowledge Management Rose through the Strategic Compact Period 34 Figure 5.2: Network Expenditures on Knowledge Management Exceed those of the Regions 34 Figure 5.3: The Majority of FY02 Expenditures Were for General Knowledge Management Activities 34 ACKNOWLEDGMENTS T his report was prepared by a team led by OED is grateful to the Norwegian Ministry Catherine Gwin. Other team members were of Foreign Affairs for financial assistance to Deepa Chakrapani, Diana Corbin, Judith support a number of activities, including the Hahn, and Zamir Islamshah, with additional conduct of the country surveys, specialist input provided by Bozena Krupa and Georgia reviews of Bank knowledge products, and a Wallen. The team received support from client workshop. OED also thanks the Swiss Parveen Moses and Annisa Cline-Thomas. Agency for Development Cooperation for its Background papers were provided by David support of client and expert participation in Bevan, Jeffrey Cummings, David Goldwyn, the evaluation workshop. Finally, the evalua- Kenneth King, and Simon McGrath, Elizabeth tion team is grateful to the participants at Kleemeier and Keith Stallard, Alex Romis- its knowledge evaluation workshop, held in zowski, Louise Walker and colleagues, and Washington, D.C., in November 2002, for Maureen Woodall. David Shirley provided their time and input. expert advice on the design of the client survey and prepared the survey report, based on inter- The review was completed under the guidance views conducted by Katarzyna Furman, Azizul of Victoria Elliott, manager, OEDCM. Islam, Tausi Kida, Peter Knight, and B. B. Niang. The study was published in OED's Partner- ships and Knowledge Group, under the The evaluation has benefited from discussions, direction of Osvaldo Feinstein, by the comments, and assistance from numerous peo- Communications and Publications staff of ple. Special thanks are due to the many gov- the Knowledge Management Unit, including ernment officials and civil society representa- Patrick Grasso, lead knowledge management tives interviewed in Bangladesh, Brazil, officer; Vivian Jackson, publications officer; Poland, Senegal, and Tanzania. The report Caroline McEuen, writer/editor; and Juicy also benefited from extensive consultations Qureishi-Huq, publications assistant. with Bank staff, notably Kevin Cleaver, Basil Kavalsky, Bruno Laporte, and Klaus Tilmes, as well as others too numerous to acknowledge Director-General, Operations Evaluation: Gregory K. Ingram by name. In addition, comments and guidance Director, Operations Evaluation Department (Acting): Nils Fostvedt from external advisers are gratefully acknowl- edged, namely, Joseph Abbey, Keith Bezanson, Manager, Corporate Evaluation and Methods: Victoria Elliott Dorothy Leonard, and Zainal A. M. Yusof. Task Manager: Catherine Gwin vii FOREWORD T he World Bank's commitment in 1996 to ier access to Bank and other development become a global knowledge bank proposed knowledge. broad-ranging internal and external changes Improved access does not, however, guaran- aimed at expanding the sharing of knowledge tee that the shared knowledge will be adopted, among staff, clients, and partners. While the adapted, and applied. For that to happen, transfer of knowledge and information had knowledge sharing has to be embedded in always been a dimension of the Bank's role, work processes. And, so far, the Bank's new the knowledge initiative sought to broaden the activities have not been well integrated into scope and raise the profile of this function. core lending and nonlending processes. The purpose was to improve the quality of Two management shortfalls account in Bank operations and enhance the capacity of large part for this evaluation finding of weak client countries to achieve development goals. integration of knowledge sharing with other Over the period FY97-02, the Bank has spent core business processes. First, although high- some $220 million on corporate, network, and level leadership has stimulated innovations in Regional knowledge sharing activities, and Bank knowledge services, management has over $60 million on its three main global not adequately defined the roles and respon- knowledge initiatives. sibilities of corporate, network, and Regional units for making knowledge sharing a way of doing business. Nor, according to staff, has it Key Findings established adequate incentives for incorpo- Overall, this review finds that the Bank has rating knowledge sharing into operational made good progress in establishing the tools processes. Second, in contrast to good prac- and activities to support its initiative, but it tice in other organizations and other process- has not established adequate business process- es of the Bank, there is no systematic moni- es and management responsibilities for achiev- toring and evaluation of knowledge sharing ing the strategic intent of making knowledge programs and activities. sharing a way of doing business and empow- ering clients. In the six years of the initiative, Recommendations the Bank has substantially upgraded its infor- mation management system, initiated a wide To more fully realize the knowledge initiative's variety of new activities for the aggregation potential to contribute to the scaling up of and sharing of knowledge, and fostered a effective Bank interventions and enhance the more open, knowledge-sharing culture within capacity of clients to achieve development its staff. These steps, taken together, have pro- goals, this review recommends that the Bank vided staff, clients, and partners faster and eas- take three sets of actions: ix x | S H A R I N G K N O W L E D G E 1. Management should exercise more strategic example, networks should set clear objec- direction and oversight over the Bank's tives for anchor, thematic group, and advi- knowledge processes. For this, management sory services direct support of operational should: define clear responsibilities and teams; and Regional and country units accountabilities of corporate, network, and should make explicit the knowledge objec- Regional units for integrating knowledge tives and strategies of Country Assistance sharing into the Bank's core business Strategies and projects. processes; ensure that incentives are aligned 3. Vice-presidential units should set moni- with responsibilities, especially at the task torable outcome objectives and supporting manager level; and establish a strategic performance indicators for their respective approach to the Bank's role in existing and knowledge-sharing programs and activities, any new global knowledge initiatives. and they should agree, Bankwide, on pro- 2. Network and Regional units should tightly cedures to be established for monitoring link their knowledge-sharing activities to and evaluating Bank knowledge-sharing lending and nonlending processes. For programs and activities. Gregory K. Ingram Director-General, Operations Evaluation ABBREVIATIONS AND ACRONYMS AAA Analytical and advisory services AFR Africa Region APQC American Productivity and Quality Center AROE Annual Report of Operations Evaluation AUDI Arab Urban Development Institute CAE Country Assistance Evaluation CAS Country Assistance Strategy CDF Comprehensive Development Framework DEC Development Economics DLC Distance Learning Center EAP East Asia & Pacific Region ECA Europe & Central Asia Region ENV Environment Department ESSD Environmentally & Socially Sustainable Development Network ESW Economic and sector work FKIS Financial Sector Knowledge Information Service FPSI Financial and Private Sector Investment Network GDLN Global Development Learning Network GDN Global Development Network IBRD International Bank for Reconstruction and Development (World Bank) ICT Information and communication technology IDA International Development Association ISG Information Solutions Group IT Information technology LCR Latin America & the Caribbean Region LICUS Low-Income Countries Under Stress LIL Learning and Innovation Loan MAC Marco de Aprendizaje Continuo MNA Middle East & North Africa Regional Office NGO Nongovernmental organization OCS Operational Core Services OECD Organization for Economic Cooperation and Development OED Operations Evaluation Department xi xii | S H A R I N G K N O W L E D G E OPE Overall Performance Evaluation PAD Project Appraisal Document PREM Poverty Reduction & Economic Management Network PRSP Poverty Reduction Strategy Paper QAE Quality at entry QAG Quality Assurance Group RRU Rapid Response Unit SAR South Asia Region TRS Time Recording System WBI World Bank Institute EXECUTIVE SUMMARY I n 1996, the World Bank made a commitment external changes. This evaluation examines to become a global knowledge bank. The the relevance of that strategy and the institu- Bank's stated intention was to develop a tional infrastructure put in place to implement world-class knowledge management system it. It also reviews the effectiveness of the strat- and to improve and expand the sharing of egy's three main areas of innovation: development knowledge with clients and part- · Network and Regional internal knowledge- ners. The objectives of this commitment were sharing activities among Bank staff to improve the quality of Bank operations and · Regional and country external knowledge enhance the capacity of clients to achieve sharing with clients development results. · The three Bank-supported global knowl- Since FY97, the Bank has spent some edge initiatives that have the broadest US$220 million for corporate, network, and knowledge-sharing scope are: The Develop- Regional knowledge-sharing activities and ment Gateway, The Global Development more than US$60 million for its three main Learning Network, and the Global Devel- global knowledge initiatives, the Development opment Network. Gateway, the Global Development Learning Network (GDLN), and the Global Develop- ment Network (GDN). These programs and Summary Findings activities have sparked a lot of innovation, with potentially significant benefits for the The Bank's commitment to a comprehensive scaling up of effective Bank interventions and knowledge initiative was timely and for empowering clients to improve develop- appropriate ment outcomes. But new knowledge-sharing The transfer of knowledge and information programs and activities need to be much bet- has always been a dimension of the Bank's ter integrated into the Bank's core business role. And clients, partners, and the interna- processes to achieve the initiative's broad tional community at large have long seen the objectives. For this, senior management must Bank as a main source of high-quality devel- provide greater strategic direction and over- opment analysis and expertise. The 1996 sight to corporate, network, and Regional knowledge initiative raises the profile of this units. aspect of the Bank's role, in order to foster the changes inside and outside, the Bank needed to leverage knowledge for development more Evaluation Scope effectively. The knowledge initiative aims to expand The initiative, which was responsive in knowledge sharing as a way of doing busi- 1996 to rapid changes in information technol- ness--not a separate line of business. It also ogy, has become increasingly relevant in the proposes, appropriately, a comprehensive intervening years to changes in international strategy to bring about both internal and development practices and the Bank's own xiii xiv | S H A R I N G K N O W L E D G E agenda. Both those changes have put increased not well integrated with operational activities. emphasis on ownership, partnership, and · The 80 thematic groups are useful for net- results--all processes heavily dependent on working, learning about cutting-edge stud- sharing knowledge. ies and experiences, and finding experts on Bank knowledge has become faster and particular operational matters, but they do easier to access not provide enough direct support to task teams. The Bank has made good progress since 1996 · The Bank's 24 advisory services aim to in providing staff, clients, and partners with enhance Bank operations, but more than faster access to Bank knowledge and expertise. half of the requests serviced come from out- This improvement has been the result of side the Bank, and more than half seek help actions in five areas: (1) substantial upgrades in finding publications or statistics readily in the Bank's information management system available elsewhere. Far less of the advice and global connectivity; (2) more systematic provided focuses on supporting operational collection of Bank information and lessons of work than was originally envisioned. experience, and their active dissemination to · While the Bank's Web sites have become a staff, clients, and partners; (3) greater interac- much-used resource, current usage suggests tion among staff across the institution and that the Bank needs to do more to tailor its with clients, around shared work areas; (4) Web-based content to its multiple audi- innovations in collaborative analytical work ences. Internally, staff find that the Intranet and peer-to-peer exchanges across client coun- has not provided enough detailed informa- tries; and (5) leveraging of technology for tion relevant to core work processes and global knowledge sharing. Both staff and practices, and internationally, people in clients report that they value the improved client countries still account for only some accessibility and timeliness of Bank knowledge 10-20 percent of the total use of the public and information. This view emerges clearly site. from a succession of Bank staff surveys under- taken since 1997 and a five-country client sur- There are strong examples of external knowl- vey conducted for this review. edge-sharing innovations in Bank lending and nonlending activities. But, on the whole, But weak links to operations limit the knowledge sharing is not being well integrated effectiveness of the new knowledge-sharing into country programs and projects. Few activities country programs contain explicit knowledge So far, the Bank's new activities consist prima- objectives and strategies as summarized in rily of knowledge aggregation and sharing-- recent country assistance strategies (CASs), processes that by themselves do not guarantee and most of those programs are in countries that the shared knowledge will be adopted, where Bank lending is limited by country per- adapted, and applied. For that to happen, formance or need. For most countries where knowledge sharing has to be embedded in lending dominates the Bank's program, work processes. But, for the most part, the knowledge sharing is not yet being treated as a new activities have not been tightly linked to strategic activity. Moreover, only one-third of the Bank's core lending and nonlending tasks. operational staff interviewed for this evalua- As a result, staff and clients do not view the tion think that the initiative has changed the new knowledge-sharing programs and activi- way projects are designed and supervised; ties as sufficiently relevant to their operational slightly less than one-third think that the ini- work. tiative has increased support for knowledge The main internal innovations for improv- capacity building in operations. ed knowledge sharing--thematic groups, advi- Progress in launching the three global sory services, and expanded Web site use--are knowledge initiatives reviewed--the Develop- E X E C U T I V E S U M M A R Y | xv ment Gateway, GDLN, and GDN--has been agement has not adequately defined the roles rapid, and the Bank's leadership has mobilized and responsibilities of corporate, network, and the participation of partners. It is too soon to Regional units for making knowledge sharing a see evidence of the contributions of these pro- way of doing business and embedding it in core grams to actual development results. But so lending and nonlending processes. Second, in far the programs have not met four major sharp contrast to practices of industry leaders, challenges that must be surmounted to ensure knowledge sharing is largely unmonitored their success: (1) ensuring their continuing within the Bank, and self- and independent utility in a rapidly changing technological and evaluation of knowledge-sharing activities is development environment; (2) achieving much less systematic than for other Bank financial sustainability; (3) consolidating gov- processes. Third, despite the introduction of ernance and oversight arrangements; and (4) knowledge sharing in the Bank's mission state- defining and managing the Bank's evolving ment and in staff performance evaluations, role as each program matures. Also, the three staff continue to feel that they lack the incen- programs are only beginning to be integrated tives needed to make knowledge sharing a rou- into Bank country programs and projects as tine part of their work. ways of leveraging Bank and other develop- ment knowledge in support of development Conclusions objectives. As a result of these weaknesses in reach, Overall, the Bank has made more progress in content management, and operational linkage, establishing the architecture to support its the new knowledge-sharing activities and pro- knowledge initiative than in creating the gov- grams have had limited impact on Bank client ernance arrangements and work processes for countries. Improvements in client access to carrying it out. As a result, the strategic intent Bank knowledge can be ascribed primarily to of making knowledge sharing a way of doing increased use of the Internet and e-mail. business has been only partly realized--a Clients also view staff as less arrogant and process that in other leading knowledge man- more open to collaboration and knowledge agement organizations has tended to take from sharing. Still, dissemination of the Bank's three to five years. The Bank, now entering the knowledge remains inadequate at the country seventh year of the knowledge initiative, needs level, beyond central government personnel to move deliberately to embed knowledge and a narrow circle of other individuals. More sharing in its core operational processes by use of local expertise is needed to align the providing more direct support to task teams Bank's knowledge with country conditions. and more knowledge capacity enhancement Knowledge capacity building and capacity uti- for clients, and it needs to manage its knowl- lization are, therefore, key to the successful edge services for results. leveraging of knowledge for development. But clients find the Bank more effective in sup- Recommendations porting individual training than in helping to To realize more fully the knowledge initiative's build sustainable institutional capacity for potential to enhance Bank operations and acquiring and using knowledge. empower clients to meet their development Inadequate oversight and incentives are goals, three sets of actions are needed: major impediments 1. Management should exercise more strategic High-level leadership in the Bank has stimulat- direction and oversight over the Bank's ed implementation of the knowledge initiative. knowledge processes. To accomplish this, But there have been three main shortcomings management should: define clear responsi- in the institutional infrastructure put in place bilities and accountabilities of corporate, to support programs and activities. First, man- network, and Regional units for integrating xvi | S H A R I N G K N O W L E D G E knowledge sharing into the Bank's core busi- teams; and Regional and country units ness processes; ensure that incentives are should make explicit the knowledge objec- aligned with responsibilities, especially at the tives and strategies of Country Assistance task-manager level; and establish a strategic Strategies (CASs) and projects. approach to the Bank's role in existing and 3. Vice-presidential units should set outcome any new global knowledge initiatives. objectives and supporting performance 2. Network and Regional units should tightly indicators for their respective knowledge- link their knowledge-sharing activities to sharing programs and activities, and they lending and nonlending processes. To should agree, Bankwide, on procedures to achieve this, networks should set clear be established for monitoring and evaluat- objectives for anchor, thematic group, and ing Bank knowledge-sharing programs and advisory service support of operational activities. 1. INTRODUCTION In 1996, the Bank made a commitment to expand the sharing of development knowledge among staff and with clients and other development partners. The purpose was twofold: to improve the quality of Bank operations and to enhance the capacity of developing countries to meet development goals. This review assesses the relevance of this knowledge initiative as well as the effectiveness of new knowledge-sharing activities and global knowledge programs undertaken to implement it. The review also examines the institutional infrastructure that has been built over the past six years to support the initiative. Knowledge Initiative Launch gral part of the newly created thematic Net- In his Annual Meetings address of October works, the Overall Performance Evaluation 1996, Bank President James Wolfensohn (OPE) for staff, and the Bank's first formal emphasized that development knowledge is a mission statement. By the beginning of 1999, global public good that belongs to everyone-- the Bank had also taken steps to upgrade its and from which everyone should therefore information management system, introduce a benefit. He also highlighted the potential of variety of new Network and Regional knowl- new information technology to dramatically edge-sharing tools and activities, and design increase the reach of development knowledge, several global knowledge partnerships. and he noted that realizing this potential As a result, the Bank is now recognized as a would require a "partnership for creating and leading knowledge management institution.2 sharing knowledge, and making it a major Its knowledge initiative also provides a model driver of development." Citing the World for expanded knowledge-sharing activities in Bank's unique breadth of development experi- other international development agencies ence, Mr. Wolfensohn proposed that the Bank (King and McGrath 2002). Monitoring and commit to investing in the necessary systems, evaluation processes have not kept pace with in Washington and worldwide, to enhance its leading industry practices. Increasingly, ability to gather development information and knowledge management leaders are instituting experience and share it with clients--"to metrics for measuring the value added of their become, in effect, the Knowledge Bank" knowledge-sharing programs. Despite calls for (Wolfensohn 1996). the Bank to do the same, it has not yet come The Bank moved quickly over the next sev- up with a relevant framework and measures eral years to implement this knowledge initia- for assessing performance and ensuring tive.1 The initiative was included as one of four accountability of knowledge-sharing programs pillars in the Bank's 1997 Strategic Compact and activities. for operational renewal and reform. Knowl- The purpose of this evaluation is to begin to edge management and sharing became an inte- address this deficit by reviewing the Bank's 1 2 | S H A R I N G K N O W L E D G E knowledge initiative and recommending ways grams initiated by the Bank with the broadest to strengthen its programs and assess their program scope--the Development Gateway, effectiveness on an ongoing basis.3 the Global Development Learning Network (GDLN), and the Global Development Net- work (GDN).5 Evaluation Design Scope. Initially, the objective of the Bank's Framework. In the absence of a framework and knowledge initiative was described as building metrics for assessing the Bank's knowledge- a world-class "knowledge management" sys- sharing programs and activities, OED proposed tem that would enable greater knowledge an evaluation framework and discussed its fea- sharing among staff and with clients and part- tures with management. To inform the design ners. Soon after its launch, the terminology of the framework, OED consulted a number of changed: "knowledge sharing" replaced experts in the field and commissioned a review "knowledge management" as the phrase used of the knowledge management literature.6 The in the Bank to refer to the initiative's strategy literature indicates that successful knowledge- and programs. Knowledge sharing is, there- sharing programs require: fore, the term used in this review, except when referring specifically to the enabling informa- · A well-articulated knowledge strategy that tion management systems and budget expendi- is tightly aligned with an organization's tures, which are coded as knowledge manage- business strategy and clients' needs. ment.4 Bank documents identify "knowledge · Programs and activities that are linked to sharing" as one element in a cycle of knowl- core business processes and practices and edge creation, sharing, and use. They also driven by expressed knowledge needs of describe the knowledge-sharing process as: staff and clients--whether for tacit knowl- edge that resides in the mind or for codified The systematic capture of knowledge knowledge documented on paper. Such pro- from research and experience; grams and activities must be recognized organization and storage of knowledge sources of accessible, cutting-edge, opera- and information for easy access; tionally relevant knowledge that reaches and transfer/dissemination of knowl- intended clients and audiences. edge, often as a two-way exchange. · Supporting institutional infrastructure Early Bank documents emphasized that the based on an integrated approach in three knowledge initiative was not a new line of essential areas: technology, people, and business for the institution, but instead an processes.7 improved way of doing business, designed to The evaluation framework created for this help countries and the wider international review uses these characteristics of strategic development community leverage knowledge alignment, quality of shared knowledge, acces- in support of development results. Implemen- sibility, and operational usefulness to measure tation has therefore involved all Bank units-- the Bank's knowledge initiative against the corporate, network, and Regional. This standard OED evaluation criteria of relevance, review, as the first evaluation of the imple- efficacy, and efficiency. In doing so, the evalu- mentation process, focuses on the programs ation framework poses three sets of questions: and activities introduced by the sector/themat- ic Networks (which have primary responsibil- · Relevance of the Bank's knowledge-sharing ity for development knowledge sharing among strategy: Is the knowledge-sharing strategy staff) and the Regions (which have primary aligned with client needs and the Bank's responsibility for the Bank's external knowl- poverty reduction mission? Does the Bank edge sharing with clients). The review also have a comparative advantage in advancing examines the three global knowledge pro- the knowledge-sharing objective? I N T R O D U C T I O N | 3 · Efficacy and efficiency of the Bank's pro- extent of knowledge capacity building as an grams and activities: Are activities designed explicit project objective to achieve the strategic objective? Do they · More than 30 additional interviews with have clearly defined outcome objectives Bank management and staff aligned with the Bank's core business priori- · Expert reviews of the degree of innovation, ties? Do they make quality knowledge more quality, and relevance of the Bank's knowl- accessible to intended clients in ways that are edge and knowledge-sharing efforts in four useful for improving development results? issue areas--education, power, water, and Do they, or are they likely to, achieve their public expenditure management intended outcomes? To what extent have · Reviews of the GDLN and the Develop- they, or are they likely to, capture and share ment Gateway knowledge in cost-effective ways? · Most important, a survey of the views of · Adequacy of the Bank's supporting institu- officials, academics, nongovernmental tional infrastructure: Has the Bank put in organizations (NGOs), journalists, and rep- place the governance processes, skills, and resentatives from the private sector in five incentives as well as the technological client countries on which Bank knowledge capacity needed to implement the knowl- services have proved most useful in their edge initiative successfully? And has it done development efforts.9 so using the least resources necessary?8 The relatively heavy reliance on surveys and interviews reflects a current weakness in all of Approaches. In making its assessment, the the Bank's knowledge-sharing programs and review uses several approaches: activities--the dearth of monitoring and · Review of the literature on knowledge man- reporting of inputs and outputs, and self- agement and transfer assessment of outcomes against intended · Desk reviews of relevant Bank policy and measurable objectives. Indeed, a main finding strategy documents and program reports of this report, as discussed in Chapter 5, is the · Surveys of 15 network advisory services low evaluability of the overall initiative and its and 28 thematic group leaders component programs. This necessarily limits · Structured interviews of 25 task team mem- the ability of this review to make judgments bers to obtain information on the actual use about the accomplishments of the knowledge of various knowledge-sharing activities in initiative. It also leads to the report's recom- improving the design and implementation mendation that strengthening monitoring and of Bank-supported programs, and the evaluation should be a priority. 2. A HIGHLY RELEVANT STRATEGY The Bank's knowledge strategy is innovative, broad ranging, and responsive to challenges fac- ing client countries, the wider development community, and the Bank itself. It builds on one of the Bank's widely regarded areas of comparative advantage--its global knowledge of devel- opment issues and experiences. And the strategy is much welcomed in client countries. But shortcomings in the strategy's design pose challenges to achieving the goals of improv- ing operational quality and empowering clients to acquire and use knowledge more effective- ly. In particular, the strategy does not adequately link knowledge-sharing programs and activ- ities to core lending and nonlending processes. Nor does it encourage the Bank to adapt its knowledge services continually as technology and the development environment change. Discerning the Bank's Knowledge Strategy · Change the culture of the institution to encourage a more open, team-based learn- Since the launch of the knowledge initiative ing environment. the Bank has produced no overarching docu- ment outlining how the initiative's objectives As outlined in the Strategic Compact, will be achieved. There has, however, been a implementation of the initiative was to be succession of strategic documents and phased, with a preliminary focus on improving progress reports. This review considers these the effectiveness of staff, and the eventual goal documents as providing an informal statement of meeting the needs of both internal and of the Bank's knowledge-sharing strategy.10 external users of Bank knowledge. The intend- The initial strategic statement, incorporated ed knowledge-sharing improvements were in the 1997 Strategic Compact, called for expected to produce four outcomes to support "retooling the Bank's knowledge base." The better-quality Bank operations and improve main elements of the plan were to: on-the-ground development results, including: · Develop an institutionwide "world-class · Faster access to relevant development knowledge management system" to collect, knowledge and information by staff, synthesize, and disseminate knowledge and clients, and partners make it more readily accessible to staff, · Greater diffusion of best practice from clients, and partners. Bank-supported and other operations · Establish thematic networks that would · Expanded reach of Bank knowledge servic- build help desks and knowledge databases. es to multiple stakeholders · Expand the client capacity-building work · Enhanced client capacity for knowledge use. of Economic Development Institute (now Subsequent strategic documents have grad- the World Bank Institute, WBI). ually shifted the focus from building the 5 6 | S H A R I N G K N O W L E D G E Bank's internal knowledge capture and dis- nology revolution and to use that revolution to semination capacities to expanding knowledge accelerate development outcomes. It supports sharing with clients and other development development changes that put increased partners. And, since 2000, the focus has emphasis on ownership, partnership, and included building client capacity to acquire results--all processes that place a high premi- and use knowledge effectively. In October um on the exchange of ideas and information. 2001, these three goals were pulled together in The strategy also reinforces the Bank's own a presentation that responded to a Board commitment to do a better job of learning request for the Bank to clarify the Knowledge from experience in order to improve opera- Bank strategy in an effort to serve client and tional quality. And it responds directly to staff operational needs better. comments that they could not readily access the knowledge required to do their work (See figure 2.1, which reports on views from a The Strategy's Strengths series of staff surveys that begins in 1997). The knowledge strategy responds to chal- The knowledge strategy also builds on the lenges facing developing countries, the inter- Bank's widely acknowledged comparative national development community, and the advantage as a source of development knowl- Bank.11 It aims to reduce developing countries' edge. A broad range of stakeholders in client risk of falling behind in the information tech- countries, both inside and outside government, holds this view; and they identify leveraging knowledge for development as an important FIGURE 2.1. dimension of Bank assistance. Development Increasingly Favorable Staff Views on Access to Knowledge experts who conducted reviews for this report share the view that the technical quality of Bank knowledge is high (figure 2.2). 100 But both clients and experts distinguish between the unquestioned technical soundness of the Bank's shared knowledge and its more 80 limited qualities of objectivity and applicabili- ty to country circumstances (See Chapter 4 for a fuller discussion of these issues). Overall, the Bank's knowledge strategy is an 60 able innovative and appropriately broad expansion orvaF of existing Bank knowledge services. Although the transfer of knowledge has always been a % 40 key component of the Bank's service to its clients, the strategy raises the knowledge-shar- ing profile of the Bank's overall role. It there- 20 fore envisions both internal changes--for col- lecting knowledge and connecting staff--and external changes--entailing innovations in the 0 knowledge dimensions of Bank operations and 1997 1999 2002 the initiation of global knowledge partnership programs. The strategy also highlights the need Access to adequate knowledge and information for change in the Bank's culture as well as its Encouragement to be innovative technology infrastructure. Since being introduced, the initiative has received continued high-level attention. Presi- Source: Bank staff surveys, 1997, 1999, 2002. dent Wolfensohn has repeatedly emphasized A H I G H LY R E L E V A N T S T R AT E G Y | 7 FIGURE 2.2. Clients and Experts Give Bank Knowledge High Ratings on Quality and Relevance Client View Expert View 6 5.25 5.04 5.10 5 4.80 4.81 4 3 2 1 0 Quality Relevance Best practice Quality Relevance knowledge Source: Thirty Bank Client-Feedback Surveys (FY99­02); OED Five-Country Client Survey; OED Expert Sector Knowledge Reviews. the importance of knowledge for development challenges to achieving its goals. First, the and encouraged rapid progress in the imple- strategy has not provided an adequate plan for mentation of new programs and activities. accomplishing the quick transition it envisions Knowledge sharing and capacity building have from building and championing new knowl- been addressed in each Strategic Forum since edge-sharing tools to mainstreaming knowl- 2000. And expenditures of Bank budget and edge sharing as a way of doing business. The trust fund resources for programs and activi- experience of other organizations suggests that ties--coded as "knowledge management"-- successful knowledge-sharing programs are have totaled US$220 million from FY97 built over three to five years, and proceed in through FY02.12 In addition, the Bank has stages, from initial advocacy to eventual insti- expended some US$63 million for the three tutionalization. A successful transformation global knowledge initiatives covered in this requires a plan for building new knowledge- review. Taken together, the three characteris- sharing tools and activities, creating awareness tics--responsiveness to development chal- of their value for key business objectives and lenges, grounding in Bank comparative advan- integrating the knowledge processes into core tage, and level of innovation and appropriate business practices. Although a 1999 progress scope--make the knowledge-sharing strategy report on the Bank's knowledge initiative rec- highly relevant. ommended that the Bank develop such a plan, the recommendation was not carried out (Prusak 1999). While the Bank has introduced The Strategy's Design Shortcomings new knowledge-sharing tools and activities, it Despite these positive features, the strategy has has not set out objectives for integrating two major design shortcomings that present knowledge sharing into core business process- 8 | S H A R I N G K N O W L E D G E es. Moreover, though the Bank began to give ment--and in an increasingly crowded field of increased attention to external knowledge- development knowledge.13 The underlying sharing tasks by 2000, no specific, time-bound assumption seems to be that once knowledge- objectives have been set for this second phase sharing tools and activities are established and of the initiative. Nor has the Bank spelled out staff become active users, knowledge sharing the external knowledge-sharing roles of the will be built into operations in more systemat- Regions, networks, and the WBI. ic ways that will enhance clients use of knowl- The second shortcoming is the lack of a edge in achieving development goals. This mechanism to benchmark effectiveness and assumption is built on an implicit chain of make adjustments. Specifically, the knowledge results (portrayed in table 2.1). But without strategy does not contain a way to check on measurable objectives associated with specific progress against explicit, monitorable objec- business processes, the contribution of individ- tives and to alter course as needed. Nor does it ual programs and activities to intended results contain a way to ensure continuing relevance cannot be made explicit, and the activities' in a rapidly changing technology environ- effectiveness assessed. TABLE 2.1. Stylized Results Chain for Knowledge Sharing Inputs Outputs Outcomes: Intermediate Outcomes: Longer term Impact Transformation of World Bank Capture, organization, and Improvements in quality Enhanced client capacity Improved into Knowledge Bank sharing of knowledge of Bank operations to acquire and use poverty results · People/culture change · Collection and synthesis · Efficiency through faster · Expanded access to · Technology · Storage and maintenance access global knowledge and · Knowledge processes · Dissemination and · Greater diffusion of best information · Leadership and support ready access practices · Cross-country connection · Relevant knowledge · Connectivity among peers · Expanded reach to to experts and peers multiple stakeholders 3. INTERNAL KNOWLEDGE SHARING TO IMPROVE BANK OPERATIONS Since the 1996 launch of the knowledge-sharing initiative, new or improved tools and activities have been created to capture and share development knowledge. Staff now have easier access to knowledge relevant to their work. But there are still three challenges to improving the oper- ational effectiveness of the Bank's internal knowledge-sharing activities: linking more directly to core operational work; capturing lessons learned and good practices more systematically; and strengthening content management to ensure the quality and operational relevance of distributed knowledge. Expanding Network and Regional content in a mostly supply-driven way. This Knowledge Sharing among Staff work generated sector and regional and coun- This chapter reviews the evolution of internal try repositories of existing, codified knowledge knowledge-sharing activities and assesses the and newly packaged knowledge (in formats contribution of three major new activities-- such as toolkits, briefs, and newsletters). It also thematic groups, advisory services, and led to a major increase in online distribution of expanded use of the Internet and Intranet--to development knowledge and information. operational effectiveness. In general, the It was quickly recognized, however, that the Bank's internal knowledge-sharing activities new knowledge repositories were not well aim to improve operational effectiveness by suited to the exchange of the tacit knowledge harnessing existing knowledge, disseminating about operational experiences that staff car- it broadly, and supporting staff in incorporat- ried in their heads. Additional processes were ing lessons and good practices into the design created--such as network-led training events, and supervision of operations. thematic groups, and advisory services--to Under the Bank's matrix management struc- promote the exchange of expertise. By the end ture, the task of improving the aggregation and of 1999, these activities and expanded use of sharing of development knowledge has fallen the Bank's Internet and Intranet were main- largely to the new network/sector boards, streamed across networks as key knowledge- which have a knowledge management mandate sharing vehicles. as one of their five core functions. Working on The Regions also developed internal knowl- the assumption that a great deal of knowledge edge-sharing tools and activities--although in existed, but in forms that made it difficult for the absence of a mandate defining the Regions' task teams to access, the networks' initial internal knowledge-sharing responsibilities, efforts focused on gathering the Bank's existing their scope has varied. And since 2000, the formal, codified knowledge and disseminating Regions have increasingly focused on extend- it as new knowledge products or as Web site ing their knowledge-sharing activities to the 9 10 | S H A R I N G K N O W L E D G E transfer and brokering of knowledge with childhood education, or urban poverty) and to clients (as discussed in the next chapter). gather and disseminate relevant knowledge.14 As a result of the new internal knowledge- Currently there are some 80 thematic groups, sharing efforts and the enabling information down from a peak of 125 in 1999. The peak technology, staff access to Bank information number came in response to the 1998 intro- has been considerably improved. This finding duction of designated budget resources for any is illustrated in the increasingly "favorable" group with a workable plan, and the decline responses in successive Bank staff surveys (See followed the scaling back of resources at the figure 2.1). end of the Strategic Compact.15 But there are two aspects of the network Groups carry out a common array of activ- and Regional efforts to improve internal ities, although the funding they receive and the knowledge sharing that limit their overall rele- emphasis they put on the diverse activities vance and effectiveness. First, while Bank oper- vary. Figure 3.1 lists main activities as identi- ations are increasingly multisectoral in fied by thematic group coordinators surveyed approach, the bulk of knowledge capture and by OED. Groups range in size from 25 to 200. sharing is organized by network and sector. In Membership is voluntary and open to all staff, interviews conducted for this review, staff and some groups have a small number of noted that the "silo" structure of the internal external members. Group leadership is also knowledge-sharing function does not meet the voluntary; in most cases, limited time is allo- needs of multisectoral operational work. Sec- cated for coordinating group activities.16 ond, there is inadequate coordination between In general, staff find thematic groups useful network knowledge-sharing activities and to their work. This finding comes from a vari- country and project teams. Few network inter- ety of thematic group surveys conducted since nal knowledge-sharing activities are embedded 1999.17 Staff identify brown bag lunches and directly in core work processes. More activities other face-to-face events as useful ways to net- are focused on providing access to knowledge work, learn about cutting-edge studies and and expertise than on ensuring that knowledge experiences, and find out who has expertise on is shared in ways that promote its adaptation particular operational matters. Thematic and use--for example, by enhancing active group Web sites are also fairly well regarded as team-based knowledge sharing. Also, activities sources of readily accessible sector or themat- have done more to push out knowledge than ic information, related Bank knowledge, and, pull it in--thus missing out on opportunities to in many cases, links to external knowledge refresh the Bank's knowledge through ongoing and expertise. But staff report that thematic field experience, reduce reinvention of the groups tend to be less effective in ensuring the wheel, and scale up successful programs. As a quality of distributed knowledge, engaging result, internal knowledge-sharing activities are nonheadquarters staff, and providing direct not sufficiently relevant to the day-to-day oper- support to task teams. ational work of frontline staff. This finding is Staff views on the value of thematic groups' discussed below in respect to three key knowl- published materials--papers, notes, and toolk- edge-sharing vehicles: thematic groups, adviso- its--are mixed. This appears to reflect, in part, ry services, and expanded use of Bank internal shortcomings in quality control that affect all and external Web sites. anchor knowledge products. For example, there are no routine practices for validating lessons learned and promising practices. Only Thematic Groups a third of 15 recently surveyed network and The Bank's thematic groups (known outside Regional units report having lessons learned the Bank as communities of practice) were or good practice validation processes in place. established to enhance the transfer of tacit And only one of these units reports having a knowledge among staff working on shared process for regularly eliminating outdated les- problems (such as decentralization, early sons or good practice cases; four units report I N T E R N A L K N O W L E D G E S H A R I N G T O I M P R O V E B A N K O P E R AT I O N S | 11 FIGURE 3.1. Thematic Group Activities Provide More Indirect than Direct Support to Operations Workshops Web sites Publications Training tivitiescA Quality activities Partnerships Support to task teams Other 0 5 10 15 20 25 Number of groups Source: OED thematic group survey. that their materials have no time limits.18 This undercuts the ability of Network knowledge activities, including those of thematic groups, BOX 3.1: to serve as reliable sources of community A Consultative Process for Validating Content knowledge. A notable example of good vali- The Administrative and Civil Service Reform Thematic dation practice is described in box 3.1. Group has developed a consultative method for assuring While the size of thematic group member- the quality of new materials for its Web site. Supported ship suggests substantial reach, the number of by a partnership among leading organizations in civil active participants--one of the key success fac- service strategy and management, the Web site aims to tors of communities of practice--tells a differ- provide users with access to key principles and current ent story. A little less than a third of thematic research issues. Quality control for the site is the respon- group members responding to a 1999 sibility of the World Bank, in collaboration with desig- Bankwide survey described their participation nated staff in partner organizations. When new material as active, and the percentage remained rough- is being considered for inclusion on the site, Bank staff ly the same (30-35 percent) in subsequent ask other partners as well as thematic group members to years. Of those who are active, a still smaller comment on its quality and relevance. Staff with relevant number actually contribute to sharing commu- expertise are asked to validate the material's quality. nity knowledge. Moreover, headquarters staff Strong objection from any partner prevents posting of the make up the bulk of active members.19 The- material. matic group coordinators consider this level of staff participation too low for the groups to Source: OED thematic group survey. serve as highly interactive communities that 12 | S H A R I N G K N O W L E D G E contribute significantly to solving work process problems or scaling up successes. As a BOX 3.2. result, the systematic capture and sharing of Task Teams Welcome Direct Support tacit knowledge remains the thematic groups' of Project Implementation by The- biggest challenge. As explained by one group matic Groups coordinator: The Community-Based Rural Devel- We would like to make a more lively and opment Thematic Group, which aims engaged community that will actively con- tribute knowledge. We are doing okay with to ensure that best practices inform explicit knowledge, but need to do more with ongoing operations, holds field-based tacit knowledge, especially capturing it from workshops for task teams and clients all Regions and drawing on the rich experi- working on community-based rural ence of the field. This is not being done sys- development projects. With input tematically right now. Either there is a very from strong practitioners (identified formal but isolated instance of a product, for by the Quality Assurance Group as example, a PREM Note, or there is a very having outstanding performance), the- informal exchange over e-mail that is not matic group members help workshop recorded. [OED thematic group survey.] participants address operational External knowledge management studies issues. Client feedback indicates that show that while tacit knowledge accounts for lessons learned in workshops help to about 80 percent of the knowledge sought by expedite ongoing project implementa- operational staff, it makes up only 10-20 per- tion. Task teams underwrite workshop cent of knowledge shared. For effective cap- costs out of their own project funds. ture and sharing of tacit knowledge, organiza- Source: OED interviews. tions need to rely more on peer-to-peer exchanges, iterative knowledge sharing, and team learning than on knowledge briefs, one- shot conferences or videoconferences, and Thus, the most frequent recommendation electronic discussion spaces. The challenge is made by staff is that thematic groups should to provide support for team-based, iterative, increase the operational relevance of their and face-to-face knowledge sharing. This sup- activities: port is needed on an ongoing basis in all stages · A 2000 thematic group self-assessment of an operation, including design and imple- highlighted that groups provided only lim- mentation, and in the capture as well as the ited operational support. transfer of operational experience (APQC · Staff feedback from a 2001 Human Devel- 2000b). opment Network survey emphasized that Even though staff value the networking and groups should strive to meet the real-time professional development opportunities pro- demands of task managers and make vided by membership in thematic groups, most stronger linkages between knowledge man- do not see the groups as sufficiently relevant to agement and lending operations. day-to-day operational work (OED survey of · More than half of the respondents to a thematic group coordinators, 2001). Typically, 2002 survey by the Poverty Reduction and groups provide more indirect than direct sup- Economic Management (PREM) Network port to task teams (as indicated in figure 3.1), indicated that thematic groups didn't meet, although when teams are directly supported-- or only partly met, staff operational needs. through such activities as on-demand peer reviews of projects, support for project work- Increased networking among staff facilitat- shops, or help in locating expertise--the prac- ed by thematic groups has been enabled by the tice receives strong staff support (see box 3.2). Bank's enhanced communications technolo- I N T E R N A L K N O W L E D G E S H A R I N G T O I M P R O V E B A N K O P E R AT I O N S | 13 gy--especially the Web-based and e-mail con- matic groups will contribute significantly to nections discussed below. But other institution- improving the quality and effectiveness of al supports needed for effective thematic group Bank operation. On the otherhand, they are activities are limited. In particular, the support, not necessarily held accountable for achieving management direction, and monitoring of specific business goals or provided with corre- many thematic groups fall short of leading sponding resources and incentives. practice. Over the last three to five years, lead- Budgeting, programming, and reporting ing industry practice has moved from simply vary considerably across groups, and there has enabling communities of practice to exchange been no attempt to agree on performance stan- ideas to more purposeful interactions that dards (see box 3.3). The public sector themat- include regular accounting of activities and ic groups are among the few that use funding outcomes. The Bank has not made this shift. guidelines and a transparent mechanism to After an initial phase in which the Bank "let establish strategic focus and accountability. a thousand flowers bloom," the end of the On the whole, thematic groups have not been Strategic Compact budget "hit knowledge expected to pursue explicit outcome objec- management in general and thematic groups in tives, monitor outputs against those objectives, particular like a ton of bricks" in the words of and assess and report on results. In addition, one interviewee. The resulting consolidation staff point to a lack of management support, in of thematic groups in 2002 and 2003 was terms of time and incentives, as a major con- probably inevitable and not in itself a prob- straint to thematic group participation.20 There lem. But the resulting configuration of groups is, in other words, a significant mismatch appears to reflect diverse enabling environ- between perceived roles of thematic groups ments, and not a deliberate matching of the- and actual management support and perform- matic groups to the Bank's evolving business ance requirements. A representative view of strategy. For example, each of the three thematic group coordinators is that: "Themat- departments in the Environmentally & Social- ly Sustainable Development (ESSD) Network has taken a different approach to the role of thematic groups. The Environment Depart- BOX 3.3. ment (ENV), an important area for the Bank, There Is No Shared Practice for Budgeting, Programming, has decided not to support thematic groups and Reporting of Thematic Group Actions and, instead, to carry out knowledge-sharing Among the 28 thematic groups surveyed by OED: activities through seven anchor thematic teams. The Agriculture and Rural Develop- · There is a clear association between a thematic group's sec- ment Department, after a significant drop in tor and whether it receives budget resources, but there is no 2001-02 in thematic group participation and clear link between the type of activities undertaken and the budget, has committed to a reinvigoration of provision of budget resources. groups in 2003; and the Social Development · While half of the thematic groups receive direct allocations Department has differentiated between four of budget resources and prepare work programs, only formal groups managed by anchor staff and three-quarters of those funded prepare formal work pro- five informal groups that receive less support. grams detailing deliverables and expenditures for which While both formal and informal communities they can be held accountable. of practice also exist within other organiza- tions--with clear differences in objectives, · For the most part, reporting is limited to inputs and out- modes of operation, and extent of manage- puts. Only 2 of the 28 groups reported specific indicators ment support--there is no comparable strate- of progress toward stated outcome objectives. gic clarity about the purpose, functions, and level of support of thematic groups across the Source: OED thematic group survey. Bank. On the one hand, it is assumed that the- 14 | S H A R I N G K N O W L E D G E ic groups either need to be drastically re- and services, such as weekly topical e-mail thought or re-sourced--Or else the Bank needs updates. Most try to respond to all queries to drop the pretense that they are doing more within 48 hours, an exception being the finan- than they are doing now"(OED thematic cial sector service (FKIS), which provides in- group survey). Inadequate management sup- depth responses only to selected queries. And port and oversight, a shortcoming that cuts at least one advisory service--Rapid across not just thematic groups but the entire Response--provides both free and fee-based knowledge initiative, is discussed in more services related to private sector investment to detail in Chapter 5. a targeted external audience (see box 3.4). All services track incoming requests and virtually all have experienced increases in the number Advisory Services of requests received for at least an initial two The Bank's advisory services (sometimes called to three years of operation. The increases sug- help desks) are low-cost mechanisms designed gest that the advisory services are helping to to serve as one-stop shops for access to Bank make Bank information more readily accessi- information and expertise. By the end of 2002, ble. there were 24 advisory services: 3 Regional The nature and origin of requests indicate, services and the rest at the corporate, network, however, that the advisory services play a dif- or sector level. ferent role from that intended--of more limit- All advisory services operate on a small ed contribution to Bank operational effective- scale. Out of 15 services surveyed for this ness. Staff report that they find advisory review, the 12 that reported the size of their services useful primarily in meeting requests staff had an average of 1.5 full-time-equivalent for information on document references, avail- staff (and some had additional short-term con- able experts, and specific data--as opposed to sultant or intern time).21 The nine services that requests for more synthesis or in-depth pres- provided annual budget figures for FY02 entation of substantive knowledge supportive reported budgets in a range of US$20,000 to of operational work.23 Furthermore, an exter- US$173,000, for an average of US$87,000 nal review of all advisory services requests and (taking out one outlier that operates on a dis- responses (29,000) in FY00-01 shows that tinctly different basis, as described in box 3.4).22 Most advisory services cite contribut- ing to internal quality and effi- BOX 3.4: ciency gains as their primary Free and Fee-Based Access to Knowledge Through objective and identify Bank net- Rapid Response work and Regional staff as The goal of the Rapid Response Unit (RRU) is to their primary clients. A majori- improve public policy design and implementation on ty of advisory services also privatization and investment climate issues by provid- identify designated external ing clients with access to the Bank's knowledge groups as either primary or sec- resources and expert advice. For the general public, ondary clients. free services include access to a database of papers, While their main activity is case studies, and Web sites; a forum for discussing pol- to respond to queries, many icy issues; and a monthly electronic newsletter. Fee- services also produce and based services, in the form of small advisory transac- maintain knowledge databases tions that require up to five days to complete, are and Web sites that provide fre- offered to Bank member governments, NGOs, and quently requested information. multilateral institutions. Some advisory services also produce proactive products I N T E R N A L K N O W L E D G E S H A R I N G T O I M P R O V E B A N K O P E R AT I O N S | 15 over half (58 percent) of the requests came tinctly higher rates of repeat users than oth- from outside the Bank (Fonseca 2003). Of ers). In addition, just over half the queries those external users, 43 percent were from six sought help in finding publications and statis- developed countries. And a large majority (83 tical data that are readily available elsewhere percent) of all those making requests were (see figures 3.2 and 3.3 for summary data on one-time users (though some services had dis- the origin and type of requests). FIGURE 3.2. External Requests Dominate Advisory Services Field staff use 2% General Bank users public 15% 42% Outside users 58% Headquarters Private staff use 40% sector 14% NGOs 1% Students 14% Other 2% Development organizations 12% Source: Fonseca 2003. FIGURE 3.3. Most Requests Are for Publications and Data Other 20% Bank work on country specific topic 3% Bank's publications Identification of Bank staff and documents by project, topic, country 4% 37% Bibliographic information 6% Bank policies, programs, and training 7% Statistical data Solution to a 15% specific problem 7% Source: Fonseca 2003. 16 | S H A R I N G K N O W L E D G E These numbers suggest that even though it does not reflect the degree of detail neces- the advisory services were created as a new sary for operational work. For example, in knowledge-sharing structure, the majority of privatization of railways, what kinds of queries might have been handled by previous- social mitigation measures worked? Who ly existing structures, such as Bank libraries, was affected? What happened to unemploy- ment? (OED task team survey and Project or by expanded use of Web sites. The advisory Appraisal Document [PAD] review.) services have provided only a limited amount of operationally focused, synthesized knowl- Staff also say that it takes too long to find edge. While they do not cost much, it should what they want. This is due, in large part, to be understood that they serve more as referral many units having established their own Web than as advisory services. In addition, they site structures instead of using more standard- serve a wider range of audiences and purposes ized, Bankwide frameworks for knowledge than is intended by their primary objective of aggregation and dissemination. enhancing operational effectiveness. The Bank's external Web sites now reach 700,000 users a month from a wide variety of audiences, including Bank staff, development Expanded Use of Bank Internal and specialists, and the general international devel- External Web Sites opment community.26 Bank staff are especially Since 1994, when the Bank launched its Web- heavy users of the external Web sites, account- based Intranet and Internet services, Bank ing for less than 2 percent of all users but 16.5 Web sites have become a much used resource, percent of the amount of material requested. both across the Bank and globally.24 In addi- Although developing country use has grown tion, feedback--from inside and outside the by as much as 300 percent over the 2000-02 Bank--indicates that some 90 percent of all period, residents from all developing countries users of the Bank's public site find it useful to combined account for some 10-20 percent of their work. These aggregate usage statistics total usage, while those from the United States confirm the value of the Bank Web sites as an account for about 25 percent. essential tool of the Bank's knowledge initia- It is hard to say how effective the external tive.25 Nevertheless, disaggregated patterns of Web sites are in serving priority audiences internal and external use of the Web sites because much of the Web-based material is reveal shortfalls in reach, content quality, and, not designed, segmented, or tracked for dis- operational usefulness. tinct user groups, though several particular Bank staff are regular and heavy users of client areas exist (for example, for journalists, the Intranet--more for information about bond investors, NGOs and civil society, and Bank transactional services (such as requests children). The current usage numbers are a for badges) than for development knowledge, reminder of the continuing digital divide as a which is mostly published on the external site, limitation on the Web as a means of reaching or work process information related to opera- developing country audiences. Current usage tional tasks. This is one explanation for field has also led external experts to recommend staff accounting for a smaller proportion of that the Bank "improve its audience focus and Intranet usage (10 percent) than their propor- understanding of audience needs and inter- tion of total staff (30 percent) (World Bank ests, with a particular emphasis on external 2002b, p. 38). According to staff, the Intranet users" (World Bank 2002b). This improve- does not provide enough detailed information ment will require the Bank to better tailor its relevant to core work processes and practices. Web-based content to its multiple audiences. A typical staff survey response is: Box 3.5 illustrates the dilemma of using the Bank's external Web sites for both the distri- The data on the Web sites are the kinds of bution of state-of-the art knowledge and information that are easily available to all-- client capacity building. I N T E R N A L K N O W L E D G E S H A R I N G T O I M P R O V E B A N K O P E R AT I O N S | 17 sized in the ENV Web assess- ment, if its Web strategy is to BOX 3.5: be effective it needs to be set Targeting Web Content to Intended Audiences within an overall strategy for Poses Challenges network knowledge sharing. World Bank staff and local researchers jointly pro- That overall strategy is, howev- duced several papers on power sector reform, with the er, still a work in progress in aim of building local research interest and capacity. ESSD; and, in other networks, This material was subsequently posted on the Bank's no such strategy building has external Web site and earned the praise of targeted been attempted. local audiences. But an expert review of the papers found shortcomings in their quality and broad appli- Linking to Learning and cability. Even though the papers were relevant for Research capacity-building purposes, posting them on the Bank Web site without a clear indication of their purpose On the whole, knowledge-shar- obliged users to determine their utility. The example ing activities are not closely illustrates the need for the Bank's Web sites to do more linked to the Bank's learning to guide users to materials relevant to their needs. programs. Although knowl- edge staff play a role in the Source: OED expert review and follow-up interviews with staff. design of learning events and courses, overall coordination in setting priorities and planning These findings suggest that the Bank is not programs is closely linked in only some net- adequately managing its Web content. The work and Regional sector departments. A Bank's Information Solutions Group (ISG) has recent effort under the Learning Framework recently assessed the external Web sites of all has piloted a team-based learning process that Bank departments and provided advice to each has generated considerable interest among department on how to improve the structure, operational staff. This pilot effort may offer labeling, and presentation of their Web-posted new ways of providing active knowledge shar- materials. The purpose of these content assess- ing that both links closely with staff learning ments is to increase both the quality of the processes and increases support of task teams. individual Web sites and consistency across But on a broader scale, there is a need to bet- sites. But left unaddressed are the broader ter integrate knowledge and learning processes issues of knowledge needs assessment, quality across the institution. control, and audience focus.27 The Environ- Nor is there sufficient linkage with the mental Department has done a more searching Bank's research program. A recent PREM- assessment (based on both an external review Development Economics (DEC)-WBI initiative and an electronic client feedback survey) that is a deliberate attempt to strengthen linkages examined these broader issues. The outcome across knowledge creation, sharing, and learn- was a plan to consolidate and redesign the ing, and to connect directly to the knowledge ENV Web sites and establish guidelines, stan- needs of country teams. But it is too early to dards, and responsibilities for maintaining assess its impact on either the quality of oper- individual Web sites. The plan also envisions a ations or the enhancement of knowledge shar- long-term Web strategy, defined in coordina- ing with clients. tion with the other departments in the net- work, which aims at distinct purposes: Remaining Challenges strengthening operational work, aligning activities to sector strategies, and serving a It is difficult to assess the costs and benefits of broad international audience. Yet, as empha- the Bank's efforts to improve knowledge shar- 18 | S H A R I N G K N O W L E D G E ing among staff because the inputs and out- of outputs, it is difficult to measure the extent puts are not adequately monitored and of the contribution or value for money. reported. Assessing impact on operational Improvements in operational quality can be effectiveness is even more difficult because achieved by upgrading skills of individual there are no well-defined, monitorable, and staff, improving how teams work together, time-bound intermediate outcome goals and building up the knowledge base of profession- accompanying performance indicators. For al groups, or making innovations in institu- example, internal knowledge-sharing activi- tional policies or practices. The objective of ties have not set distinct goals and related per- any knowledge-sharing activity needs to be formance indicators for such varied knowl- expressly linked to one of these four ways of edge-sharing objectives as individual skill improving quality. building, team-based support and problem Overall, the record of the Bank's efforts to solving, or sector or institution-wide knowl- improve knowledge sharing among staff sug- edge building. This shortcoming is discussed gests the need for three specific improvements. further in Chapter 5. First, knowledge-sharing activities need to do The quality of Bank lending and nonlend- more to make knowledge accessible to staff in ing operations has improved since the intro- ways directly supportive of core work process- duction of the knowledge-sharing initiative, es--for example, by increasing direct support with sustained improvement in most quality of task teams by network and thematic indicators for the project portfolio and eco- groups; rethinking the role and mode of oper- nomic and sector work (ESW) since 1997.28 ation of advisory services; and making Obviously many factors have fostered these Intranet information more readily useful to improvements, making it impossible to show operations. Second, network and Regional direct causality between knowledge sharing sector departments need to manage content and improvements in the quality of opera- more actively to ensure the quality, intended tions. It seems reasonable to conclude that the reach, and operational relevance of distributed heightened efforts to harness and share Bank knowledge. Third, networks and Regions need knowledge and experience have been a con- to work together to achieve more systematic tributing factor. But in the absence of clear capture, feedback, and sharing of lessons outcome objectives and systematic monitoring learned and good practice. 4. SHARING KNOWLEDGE WITH CLIENTS AND PARTNERS While the transfer of knowledge to clients and the wider international development communi- ty has always been part of the Bank's role, the knowledge initiative aims to expand this func- tion through innovation in the knowledge dimensions of country programs and projects and Bank support of global knowledge initiatives. These efforts, particularly over the last two years, are providing an increasing number of promising practices. Still, improvements in leveraging knowledge for development have not been mainstreamed across country programs. And operational staff are only gradually coming to see themselves as knowledge brokers and to view knowledge sharing as a core part of their work. The result is that while client access to Bank knowledge and information has improved, weaknesses remain in the Bank's in-country knowledge dissemination, adaptation of global knowledge to local circumstances, and support for strengthening of client institutional capac- ity to acquire and use knowledge. Regional Approaches to Leveraging and databases--to client audiences. In addi- Knowledge for Development tion, Regional units have designed new knowl- edge-sharing activities aimed at strengthening The aim of expanding knowledge sharing with the Bank's exchange of knowledge with clients clients and partners has involved primarily in the context of country programs and proj- two sets of activities: Bank activities at the ects. For example, the Africa Region (AFR) has Regional and country levels and the Bank's developed a debriefing program that aims to support for several global knowledge initia- capture and distribute combined tacit knowl- tives.29 This chapter reviews the extent to edge of Bank staff, clients, and partners. The which the Regional and country activities have Latin American & Caribbean (LCR) Region intensified the Bank's knowledge interface has introduced a highly interactive "continuous with its clients and explores the start-up of learning framework" as a way to leverage three global knowledge initiatives and their knowledge better through client and staff integration into Bank operations. learning (described in box 4.1). The Middle Since 2000, the Region's knowledge-sharing East & North Africa (MNA) Region is piloting activities have increasingly shifted their focus a new ESW programmatic instrument for from knowledge sharing among their staff to a countries that do not borrow from the Bank, more outward orientation. This has involved but seek advisory services, training, or diagnos- extending the reach of their internal knowl- tic work. And the Europe & Central Asia edge-sharing tools and activities--for example, (ECA) and East Asia & Pacific (EAP) Regions Regional and country Web sites, newsletters, have concentrated on regionwide knowledge- 19 20 | S H A R I N G K N O W L E D G E sharing programs focused on helping countries progress reports) presented to the Board in improve their competitiveness in the global FY02 and in the first three-quarters of FY03 knowledge economy. to determine the extent to which the CASs set While the reflection of different Regional out a strategic approach to knowledge. conditions in these initiatives gives the pro- Although all 60 CASs identify areas of knowl- grams broad strategic relevance, the Bank's edge generation and policy dialogue, only 20 knowledge initiative must have impact at the percent treat the transfer of knowledge as a country level if it is to contribute to greater strategic objective. Most of the CASs that do development effectiveness. For this, knowledge so, propose a significant shift in the focus of sharing as a way of doing business needs to be the Bank's program from lending to knowl- systematically embedded in Country Assistance edge services. This is true for middle-income Strategies (CASs) and projects, so that relevant countries such as Brazil, China (see box 4.2), knowledge gets to those who need it, at the and Thailand, as well as a number of Middle right time and in a useful form; and, where Eastern countries where International Bank needed, sustainable capacity is created for for Reconstruction & Development financing acquiring and using knowledge. Furthermore, is not in high demand. It is also true in Low- systematic feedback of operational experience Income Countries Under Stress (LICUS) is necessary to build country and Bank capaci- clients, such as Yemen, where financing is lim- ty to adjust, scale up, and innovate. ited by weak country performance. But for most countries where lending still dominates the Bank's program, knowledge is not yet Increasing the Knowledge Dimensions of being treated as a strategic tool. Country Programs and Projects A random sample of 21 (10 percent) of the To assess the efficacy and efficiency of knowl- projects approved in FY01 was also examined, edge initiative integration into CASs, this and 19 project task managers and additional review examined all 60 CASs (or CAS task team members were interviewed to deter- mine the impact of the Bank's knowledge ini- tiative on how knowledge was being built into operations.30 Virtually all projects included ele- ments (or, in investment projects, components) BOX 4.1: focused on building client knowledge capacity A Continuous Learning Framework Provides for (OED task team survey and PAD review). And Exchange among Project Teams most projects revealed the use of a wide vari- The Continuous Learning Framework, or Marco de ety of knowledge transfer vehicles to reach dif- Aprendizaje Continuo (MAC), is an LCR collabora- ferent client groups. In discussing these com- tive process designed to improve implementation of ponents, staff emphasized that knowledge ongoing projects by sharing expertise among practi- transfer had always been part of Bank opera- tioners. MACs bring together key practitioners to per- tions. But most staff did not describe the activ- form such tasks as producing a detailed assessment of ities as part of an overall project knowledge- project implementation, conducting a technical assess- sharing framework aimed at creating cond- ment mission, or presenting a technical workshop. A itions for generating, acquiring, and using MAC can take place during project preparation, knowledge (Fonseca 2003). Nor did most staff supervision, or implementation. Initiated at the report that they had changed the way they request of governments, MAC exchanges require only work as a result of the Bank's knowledge ini- limited funding from the Bank, mainly in the form of tiative. Rather, some two-thirds (64 percent) staff time to participate in MAC meetings. of the sample of task members interviewed reported that they do not think the Bank's Source: LCR documentation. knowledge initiative and related new tools and activities have changed the way projects are S H A R I N G K N O W L E D G E W I T H C L I E N T S A N D PA R T N E R S | 21 being designed and implemented. While they long-term partnerships with national or see efficiency gains from new knowledge-shar- local institutes. The MNA Region, for ing tools, they do not see a fundamental example, has established such a relationship change in the way staff are operating. Lack of with the Arab Urban Development Institute time and adequate incentives were the most (AUDI), a Regional organization that frequent reasons given for this response. brings together mayors and government Nonetheless, there are examples from all officials from more than 400 towns and Regions of specific efforts to build knowledge cities. MNA has worked with AUDI to into operations in new ways. develop the institute's help desk and Web site on urban development and to organize · Making ESW a process, not just a product. the Children and Cities Conference held in There are recent examples in Regions of December 2002. designing ESW to engage clients in more · Expanding peer-to-peer knowledge sharing. fully generating analytical reports, includ- Practitioners from Bank-supported projects ing the collection and analysis of data and in one country are increasingly being used in disseminating and debating the reports.31 for knowledge sharing with peers in other One example, the 2003 Guatemala Poverty countries. For example, the 2002 Thailand Assessment, has been designed as a multi- CAS proposes to help Thai experts dissemi- year program of analytical work and tech- nate lessons from Thailand's social invest- nical assistance and has involved the estab- ment fund and land titling programs to lishment of long-term working relations those working on similar projects in Cam- with in-country organizations. Regional bodia and the Lao People's Democratic staff note that this collaborative process has Republic. helped improve the realism of the study's · Brokering knowledge to support reforms. analysis and the relevance of its recommen- Some country teams have employed a bat- dations. tery of knowledge-sharing mechanisms, · Building structured institutional relations. including study tours, videoconferences, In several countries, Bank teams are and cross-country dialogue to build client attempting to provide sustainable knowl- ownership of sectorwide reforms. To help edge sharing and transfer by establishing build understanding and informed dialogue for a major reform in Eritrea's telecommunications sector, for example, the Eritrea Telecom- BOX 4.2: munications technical assis- Shifting the Emphasis from Lending to Knowledge tance project made it possible As stated in the China CAS 2002: "Operationally, the for Eritrean officials to observe program would be shaped by factors reflecting the information technology appli- changing nature of the relationship between China cations in the Andhra Pradesh and the World Bank Group. In particular (a) the grow- State in India and in Silicon ing importance of the Bank's advisory services relative Valley in the United States. The to lending in the coming years. . . and (b) the potential Bank also maintained intensive gains to both sides, with China positioned not only to dialogue with officials through receive the Bank Group's assistance, but also share les- the regular use of videoconfer- sons of its successful experiences more broadly and ences. Eritrean officials praised contribute to the thinking on global development the adaptation of knowledge to issues." their country's circumstances and the utility of the knowl- Source: World Bank 2002a, p 30. edge made available to them. 22 | S H A R I N G K N O W L E D G E The CAS and project findings indicate that importance of leveraging knowledge for devel- the Bank's knowledge-sharing initiative is not opment, and most commend the Bank's efforts yet well integrated into CAS objectives or to intensify the knowledge dimensions of operations, although the number of examples country programs. A majority of the respon- of promising practice is growing.32 Client dents to OED's survey (as well as participants views of current knowledge services confirm in the follow-up workshop) agree that the this finding. Bank is moving in the right direction.33 Specif- ically, they cite improvements in the accessibil- Effectiveness of Bank Knowledge Sharing ity and timeliness of Bank information and with Clients identify the Internet as the biggest contributor to this progress. In addition, they note that A key element in assessing the effectiveness of staff have become less arrogant and more open the Bank's external knowledge sharing is to collaboration and knowledge sharing. clients' perceptions of the quality, reach, and As reported in Chapter 1, clients rate high- usefulness of the Bank's knowledge initiative ly the technical soundness of the Bank's and knowledge services. For this evaluation, a knowledge initiative. But the views of survey five-country survey of some 120 influential respondents are less positive on reach and persons (from government, NGOs, the media, applicability in individual country circum- and the private sector) was conducted through stances. Moreover, while they value the Bank's face-to-face interviews. Although the survey's role in the transfer of knowledge, they would main findings, together with those from a fol- like the Bank to do more to help build knowl- low-up workshop, show strong support for edge capacity within their countries (see fig- the Bank's knowledge initiative, they also ures 4.1 and 4.2). identify persistent weaknesses in the Bank's delivery of its knowledge services, which a Inadequate dissemination of the Bank's knowl- majority of clients urge the Bank to improve. edge at the country level is the area respon- Overall, clients interviewed affirm the dents most commonly cite as needing improve- Figure 4.1: Shared Knowledge Is Well Focused, but Not Well Adapted or Disseminated Strongly 5 agree 4.44 4.24 4.01 4 3.96 3.89 3 2 1 Strongly 0 disagree Focusing Strengthening Providing Incorporating Disseminating information capacity links local knowledge information Source: OED client survey. S H A R I N G K N O W L E D G E W I T H C L I E N T S A N D PA R T N E R S | 23 ment. Beyond central government personnel accessibility and credibility of Bank informa- and a narrow circle of other well-informed per- tion through collaboration with national insti- sons, the reach of Bank knowledge is said to be tutes. One respondent's view is representative: limited. Two contributing factors that clients "We cannot place reliance on the Bank forever. cited frequently are that Bank information is The local institutions have to be assisted in easily accessible only by computer, and is not expanding their capacity. This will require a lot available in local languages.34 Clients indicate more from the Bank in the area of transferring that there is also a "government bias" in dis- skills and knowledge at the institutional seminating information.35 Stakeholders from level."37 But some respondents also stress the Poland and Senegal suggest, however, that gov- need for greater government ownership of this ernments have the primary responsibility for process, noting that their governments under- disseminating Bank information and making it emphasize project components related to accessible on a countrywide basis. These stake- capacity building. holders maintain that the absence of govern- In offering views on how to make improve- ment motivation to disseminate knowledge is a ments, clients interviewed urge the Bank to significant part of the distribution problem. accelerate changes in the knowledge dimen- sions of project design and implementation, Adaptation of global knowledge to country and to reduce the variation in the emphasis on contexts is a second area clients view as weak. knowledge work among Bank task managers. The majority of survey respondents say that To achieve these changes, clients suggest that a the Bank presents "ready-made" solutions strategy for knowledge dissemination and that are not adapted to individual country cir- capacity building be clearly defined in project cumstances. They argue that the Bank is reluc- plans. As figure 4.2 illustrates, clients give par- tant to consider alternatives to the models and ticular emphasis to enhanced efforts by the solutions that it outlines in policy advice and Bank to incorporate local knowledge and col- documentation. Most respondents suggest laborate with local experts, strengthen institu- that increased use of local expertise as well as tional capacity, and expand in-country knowl- more local involvement in generating knowl- edge dissemination. edge would improve the alignment of Bank knowledge with country circumstances.36 Sev- eral stakeholders also note that adapting glob- Development and Integration of al knowledge to country circumstances is easi- Global Knowledge Initiatives er when there is acknowledged indigenous In addition to its other knowledge-related capacity for doing so. Therefore, respondents efforts, the Bank has designed and launched from Brazil, for example, see the adaptation of several global knowledge initiatives. These global knowledge as a responsibility of their programs aim to expand knowledge sharing officials. and help client countries build their capacities Capacity building--individual and institution- for acquiring and applying knowledge relevant al--is viewed as essential. But clients perceive to their core development goals.38 A recent the Bank as more effective in supporting indi- management-commissioned report reviewed vidual training (through study tours, work- the Bank's overall handling of this "portfolio" shops, and other forms of project-related train- of knowledge initiatives. The report, which ing) than in helping to build sustainable examined the degree of innovation and rele- institutional capacity for acquiring and using vance of the initiatives to Knowledge Bank knowledge. They therefore emphasize that an objectives, made several recommendations for enhanced focus on institutional capacity build- improvement that are now under management ing would be catalytic--contributing to consideration.39 The review here of the global improved client-country ability to adapt and knowledge initiatives has a somewhat narrow- use Bank information, as well as to increase er focus: It examines the start-up and likely 24 | S H A R I N G K N O W L E D G E FIGURE 4.2: Where Clients Would Like the Bank to Do Better Focus on clients'knowledge needs eness High Provide knowledge in ways that strengthen individ- tiv ec ual capacity eff Bank Provide links to international expertise and Incorporate local knowledge & collaborate with of knowledge local expertise s w viet Provide knowledge in ways that strengthen Low institutional capacity Clien Disseminate knowledge to people who need it most in country Sufficient More Client demand for more Bank emphasis Source: OED client survey. long-term effectiveness of each of the three The Development Gateway global initiatives with the broadest knowl- The Development Gateway is an Internet por- edge-sharing scope. They are the Development tal on development issues, through which users Gateway, the GDLN, and GDN (see also OED can access studies and information, exchange 2002b, forthcoming). services, and collaborate on new strategies and While the initiatives each have unique pro- programs (Walker 2003). Its business plan gram features, they share three characteristics: identifies four categories of beneficiaries: gov- they are multicountry in scope; they aim to ernments, the private sector, civil society organ- expand knowledge sharing both within and izations, and donor organizations. Since its across countries and to build client capacity; inception in 1999, the Development Gateway and they rely on partnerships for program has launched four portal-based knowledge activities and funding. In 2001, responsibility services--Knowledge/Topics, DgMarketplace, for two of the initiatives--the Development AiDA, and Country Gateway. In addition, the Gateway and GDN--was transferred to inde- Development Gateway has supported the start- pendent governing bodies, although the Bank up of 44 country-based Gateways. It has also remains involved. The GDLN is still adminis- developed an open-source technology platform tered as a department of WBI and led by the and applications for the management and Bank (see table 4.1 for country locations of exchange of knowledge and information relat- each initiative). ed to its core services. All four of the Develop- The Bank has had two objectives in pro- ment Gateways' knowledge services have moting these initiatives: to expand the avail- become fully operational in a short period of ability of development knowledge as a global time and show evidence of reaching intended public good and to improve its own effective- beneficiaries. They vary, however, in the degree ness by incorporating these initiatives in of value they add. Brief reviews of each knowl- country operations. Progress in achieving the edge service follow. first objective is discussed in the following summaries of the initiatives. Progress on the Knowledge/Topics assembles and publishes second objective is considered in a single sec- Web content on some 30 development issues, tion, as an issue that cuts across all three ini- including e-government, HIV/AIDS, the tiatives. knowledge economy, and the link between S H A R I N G K N O W L E D G E W I T H C L I E N T S A N D PA R T N E R S | 25 TABLE 4.1: Location of Global Knowledge Initiatives (by number of country-level entities per Region as of June 30, 2003) Region GDLNa Development Gateway GDN Africa 9 6 1 East Asia and Pacific 11 4 1 Europe and Central Asia 11 15 2b Latin America and the Caribbean 14 13 1 Middle East and North Africa 3 2 1 South Asia 3 4 1 Europe and North America 10 1 3 Total 61 44 10 a. These numbers of GDLN-affiliated centers were provided by WBI's GDLN division, and differ from listings provided by several Regions. See footnote 48 for an explanation of the discrepancies. b. While in each Bank Region GDN has established a partnership with one institution that implements programs regionwide, Europe and Cen- tral Asia are treated as two Regional areas. trade and development. As of June 30, 2003, and to sellers by providing ready access to there were some 24,000 registered users, with information on business opportunities. While the site receiving about 93,000 visitors month- much of the information on DgMarketplace is ly (the Bank's Internet site, after nearly 10 years available elsewhere, it is unique in its aim to of existence, has more than 600,000 unique expand opportunities for nontraditional bid- visitors monthly). Content is contributed by ders, particularly small and medium-size enter- roughly 130 organizations, and content editors prises in developing countries. Having recently from inside and outside the Bank manage each begun to charge a fee for some of its services, topic area. Many topic areas also have adviso- the market will provide feedback on who is ry committees that bring diverse perspectives to actually served and the value of the services the aggregation of material.40 provided. This approach has delivered credible and AiDA, a unique service in the development high-quality content. Although some outside community, publishes a comprehensive direc- groups continue to object to what they view as tory, in searchable form, of data on World the role of the Bank in "filtering" global dis- Bank, other multilateral, and all Organization semination of development knowledge,41 the for Economic Cooperation and Development controversy that accompanied early plans for (OECD)­member bilateral projects. While the Development Gateway has declined as the there are other sources of project information, Gateway has widened the site's base of con- there is no comparator that aggregates infor- tributors, editors, and advisers. Nevertheless, mation on the same scale. Also, by promoting in the context of a landscape crowded with standardized reporting of development proj- other knowledge aggregators that have a spe- ects and providing an easy mechanism for con- cific topical, regional, or audience focus,42 tributing data, the program encourages Knowledge/Topics lacks strategic uniqueness.43 expanded participation and use. It has difficulty differentiating its offerings The Country Gateway is a centralized serv- from those of other development portals and ice that provides advisory services, technical in directing its limited resources to distinguish support, and help in mobilizing funds for the its value, and will need to sharpen its focus to establishment of country-based knowledge increase its usefulness. portals that are operated as independent DgMarketplace, which offers procurement organizations. The 44 individual Country listings from the World Bank and other devel- Gateways, now at varying stages of develop- opment agencies, provides value to buyers by ment, aim to support country-level knowledge making the tender process more competitive, aggregation and dialogue, improve access to 26 | S H A R I N G K N O W L E D G E skills and resources, and facilitate cooperation as of three seats on an 18-member board, has and partnerships at the local level.44 Some have fueled criticisms of undue influence. Second, all formed partnerships and or other forms of col- but one member on the current board are laboration with local comparators early in financial contributors. This composition has their start-up, and have thereby been able to limited board service by representatives from complement local efforts rather than compete the strategic partnership organizations whose with them. Still, as the Internet becomes more program involvement is valuable. But new gov- pervasive and information technology devel- ernance arrangements that would encompass ops in these countries, the Country Gateways partnership organization involvement would will have to continue to evolve and differenti- not help advance the third issue: establishing a ate their services. In addition, their financial financial strategy and attracting the funds to sustainability beyond an initial period of seed ensure continuity of programs over time. At financing is currently being tested. While present there is no endowment to generate financial requirements over time are likely to income, and the existing three-year drawdown require Country Gateways to make tradeoffs of most donors' funds ends in 2004. between commercial and development objec- Moreover, both the Bank and the founda- tives, it is too early to gauge the impact of tion have to resolve how to manage the Bank's those decisions on the role that the Country future role as a Development Gateway partner. Gateways will play in meeting the Develop- For Bank programs to derive benefit from the ment Gateway's overall development knowl- Development Gateway, the Bank has more to edge­sharing objective. do to integrate the innovation and learning The Bank has provided a total of US$15.5 provided by the Development Gateway into its million for the start-up of the Development own operations. At the same time, the Devel- Gateway, of which US$7 million was spent opment Gateway must strike a balance before its transfer to the Development Gate- between serving as an effective instrument of way Foundation in 2001. Following that the Bank's own operational activities and dis- transfer, the Bank, as a member of the board tancing itself from the Bank in order to con- of the foundation, agreed to provide an addi- solidate its credibility and encourage inclusive tional US$5 million over the next three years partnership and participation across the devel- through its Development Grants Facility. Also, opment community. Ensuring that the Devel- in FY01-FY02, the Bank provided some opment Gateway activities are demand driven US$3.5 million in grants through infoDev45 for at the country and global levels is essential to the Country Gateway program. It now oper- managing this relationship over time. ates the Development Gateway through a serv- ice agreement with the foundation that costs The Global Development Learning Network the foundation US$6 million (none of which The GDLN is a partnership of mostly inde- can come from the funds provided to the foun- pendent Distance Learning Centers (DLCs), dation from the Development Grant Facility). funders, and content providers that support The creation of the Development Gateway knowledge sharing and learning through dis- Foundation as an independent, not-for-profit tance learning courses, seminars, and cross- organization has attracted more than US$70 country dialogues among decisionmakers million in additional funding and has forged a dealing with development issues of common number of strategic partnerships. But both gov- concern (Romiszowski 2003). The core idea is ernance and funding arrangements pose issues that of a network linking DLCs around the for three reasons. First, while the Bank has globe that deliver programs in languages and been crucial to the mobilization of support for formats consistent with the information and the Development Gateway, its role in the gov- skill needs of targeted audiences, and that are ernance of the foundation, through control of operated independently under various busi- the positions of president and treasurer, as well ness models and by different agencies or insti- S H A R I N G K N O W L E D G E W I T H C L I E N T S A N D PA R T N E R S | 27 tutions in each country. The DLCs affiliated with the GDLN have the technology to run BOX 4.3: interactive knowledge-sharing and learning GDLN Programming Links City Officials events through a combination of videoconfer- East Asia: A total of 64 City Development Strategies were pre- encing across several sites, e-mail or Web- pared in China, Indonesia, Mongolia, the Philippines, Thai- based discussions, and face-to-face or self- land, and Vietnam. GDLN facilities provided access to techni- study instruction. This multimedia capacity is cal expertise, funding, and city-to-city knowledge sharing the GDLN's particular value. Examples of a within and across countries. GDLN global dialogue program and a train- Latin America: A course on open and participatory municipal ing course are provided in box 4.3. governance was delivered in partnership with Mexico and Spain Since the formal launch of the GDLN in to 60 sites in the region. June 2000, 61 DLCs have become fully func- tioning affiliates of the network.46 While this Source: GDLN documentation. expansion represents a rapid start-up of the initiative, this review finds a lack of precision in the monitoring and reporting of GDLN TABLE 4.2: World Bank Lending for GDLN Distance Learning Centers activities and related Bank expenditures. The (as of June 30, 2003) review also finds a substantial disparity across countries in the nature of the Bank's support Source/country US$ millions for centers, and the absence of a clearly artic- IDA credits ulated strategy for fostering the sustainability Benin (LIL) 1.8 Côte d'Ivoire (LIL) 2.0 of individual centers and the GDLN as a glob- Ethiopia (LIL) 4.9 al network. Ghana (part of larger project) 1.6 Total spending by the Bank for its support Mauritania (LIL) 3.3 of the GDLN's start-up has been estimated by Senegal (LIL) 2.1 the Bank's GDLN team to be on the order of Tanzania (part of larger project) 1.8 Uganda (part of larger project) 2.7 US$30-$45 million since June 2000.47 This Bolivia (part of larger project) 0.7(est'd) amount includes the actual administrative Sri Lanka (LIL) 2.0 budget of WBI's GDLN division, along with Total IDA credits 22.9 costs incurred by the Regions and by ISG in IBRD loans support of GDLN, which are only estimated. Dominican Republic 3.4 Centers in African countries and a few Thailand (part of larger project) 1.1 Total IBRD loans 4.5 other countries are financed differently from centers elsewhere. As shown in table 4.2, the Note: LIL = Learning and Innovation Loan. Bank has lent a total of US$23 million in Inter- national Development Association (IDA) cred- case of Vietnam, additional donor grants. its for the start-up DLCs in 11 poor countries. DLCs in other countries have received grants This financing covers initial capital costs and a from various bilateral and multilateral agen- declining share of operating costs over four-to- cies, or financed by a local agency out of five years (estimated in the case of the African recurrent costs, usually for limited upgrading centers to be about US$300,000-$400,000). or use of existing facilities. An additional two countries have received While the Bank's efforts over three years IBRD financing for GDLN DLCs. No equiva- have led to growth in the network of DLCs lent financial obligations to the Bank have and involvement of external partners, the sus- been incurred by other GDLN DLCs. Six tainability of the global network of DLCs is DLCs in China, Brazil, Mongolia, Nicaragua, not yet assured. As shown in table 4.3, the Timor Leste, and Vietnam, which have been amount of programming shows fairly modest co-located in Bank offices, have been financed growth as measured both in participant train- out of the Bank's capital budget and, in the ing days and number of sessions. For the inde- 28 | S H A R I N G K N O W L E D G E pendent DLCs that currently rely heavily on events organized by the Bank, or indirectly, by GDLN programming, facility utilization rates stimulating other content providers. But there remain, on average, below that required for is no clear plan in place for generating the vol- financial sustainability.48 And this low level of ume of content needed for meeting the rising utilization has called into question the busi- demand of a growing network. If it is to come ness strategy on which those DLCs have based from the Bank, more resources will be their planning.49 required. Ultimately, the centers need to generate and TABLE 4.3: market their own programs, but few are cur- Mixed GDLN Trends, FY01-midFY03 rently able to do so. In Latin America, GDLN has built on existing university or other dis- Output FY01 FY02 mid-FY03 Activities (total) 273 259 142 tance learning facilities and experience. Cen- WBI 184 155 52 ters that have chosen to affiliate with the Bank 46 35 12 GDLN to expand their offerings tend to gen- External partners 43 69 78 erate content themselves or obtain it from Participant training daysa 90,580 111,300 45,680 other organizations in their countries.51 They DLCs 26 37 4 are also largely independent of the GDLN for a. Estimate based on experience of courses of 5 days with 25 participants. revenue and for connectivity, and have increas- Source: GDLN data provided by staff. ingly strengthened Regional interactions. But in countries with less established distance Experience shows that DLCs must operate learning capacities and less-developed infor- at 50-60 percent capacity to break even. At mation technology (IT) capacities, the man- present, the GDLN centers average rates of agement skills and revenue-generating markets 10-20 percent, although some are at 40 per- do not yet exist. These weaknesses endanger cent or more. To achieve a break-even level of the GDLN's network-scale effect and its glob- utilization, many centers are looking to com- al knowledge-sharing function. bine GDLN decisionmaker dialogues (the The Bank's GDLN team has proposed that main focus of the original GDLN strategy) and the network become a vehicle for the sizable interactive learning events involving course amount of capacity-building assistance cur- offerings geared to broader audiences (who rently provided by donors. This assistance self-pay or are subsidized). This approach has could benefit from the interactive capacities of developed in response to expectations that GDLN facilities and be a ready-made source decisionmaker dialogues are unlikely to of programming on a level sufficient to meet account for more than a small percentage of a the DLCs' sustainability gap. Demand for such center's utilization capacity and that there is use of GDLN centers would, however, have to potential demand (based in some countries on be generated through a considerable market- actual needs assessments) for core skills train- ing effort, since it is not yet widespread. ing for lower-level officials and representatives Moreover, questions remain about GDLN from the private sector or civil society. governance and the future of the Bank's multi- To reach sustainable levels of operation fol- ple roles in the network. At present, centers lowing this approach, a large volume of con- are affiliated with the GDLN through individ- tent will be required.50 Moreover, because the ual memoranda of understanding. There is no value added of GDLN facilities lies in their council or advisory body of partners to pro- interactive capabilities, a structured content vide overall strategic guidance and oversight that relies on interaction among participants is of the network, no GDLN mechanism for required to make full use of the DLCs. But lit- quality assurance of content and program pro- tle development content in that format exists. vision, and no process for evaluating the per- So far, the Bank has been the main source-- formance of any of the partners of the GDLN. either directly, in the form of courses and Broadly stated, the Bank's role has been to S H A R I N G K N O W L E D G E W I T H C L I E N T S A N D PA R T N E R S | 29 create the global network of affiliated centers, US$4.7 million and an additional US$500,000 assist new centers in the start-up of operations, from the Bank's administrative budget). Mr. and facilitate program interactions among all Wolfensohn has also pledged that the Bank will centers in the network. Yet no one unit within underwrite up to US$1.8 million of GDN's the Bank has been assigned responsibility for annual administrative costs through 2006. oversight and management of GDLN, quality GDN's stated objectives are to support the assurance, and accounting for GDLN financial generation and sharing of knowledge for and operational data. Nor are GDLN-related development; strengthen the capacity of activities well coordinated among relevant research and policy institutions in developing Bank units. GDLN Services, which is a unit of countries and transition economies to under- WBI, has responsibility for the administration take high-quality, policy-relevant research; and of the network, but the extent of its responsi- help bridge the gap between the development bilities in relation to other units in the Bank of ideas and their practical implementation. In and in relation to affiliated DLCs is not well advancing these objectives, GDN mobilizes defined. ISG provides technical support and and provides funding to seven regional connectivity to DLCs, but these arrangements research networks52 for research grants to indi- are not fully transparent. The Regions pro- viduals and institutions, allocated through mote and, in some cases, finance the establish- regional research competitions. ment and affiliation of DLCs, but coordina- Each of these networks has its own man- tion with GDLN services in fostering the agement structure and program agenda, is business development and financial sustain- governed by an independent board, and ability of individual centers is inadequate. obtains funding from a variety of sources. All Although a draft business plan states that the seven existed prior to the establishment of Bank has a commitment, if implicit, to also GDN, and except for the Latin American and generate the input--that is, to gather together Eastern European networks, were previously the experts, practitioners, and trainers neces- supported by the Bank's Research Support sary to organize and deliver knowledge shar- Budget. GDN's interactions with these inde- ing activities in numbers and types sufficient pendent research networks focuses on sup- for the DLCs to raise enough revenue--the porting research in individual countries within nature and extent of that commitment have the Regions; fostering collaboration among not been made explicit. countries within the Regions; promoting exchange on such issues as research methods, The Global Development Network (GDN) quality control, and research findings; and The World Bank launched GDN in December ensuring a focus on policy impact. Regional 1999 at a first annual conference in Bonn. In hubs in Europe, North America, and Japan 2001 it was incorporated as an independent, have also been established to complement not-for-profit organization based in Washing- developing country networks, and a fourth ton, D.C., and its relocation to a developing network is being formed in the South Pacific country is now proposed for 2004. Region. Initiated in the World Bank's Development In FY02, this core Regional research sup- Economics Vice Presidency, GDN has received port function accounted for 32 percent of strong support from both the Vice Presidency GDN's annual activities expenses of US$6.8 and from Bank President James Wolfensohn. million (see figure 4.3). Since 1999, the Bank has provided a total of The other 68 percent of annual expendi- US$18.3 million to GDN. This represented tures were for GDN's additional global activi- almost all initial GDN funding, and in FY02 ties, which entail: an annual Global Develop- accounted for US$5.2 million, or some 58 per- ment Conference, of which there have been cent of total GDN financing (composed of a four since 1999; global research projects; other Development Grant Facility allocation of collaborative programs; and electronic knowl- 30 | S H A R I N G K N O W L E D G E FIGURE 4.3: what they view as GDN's "loyalty" to too GDN Expenses by Core Activities (FY02) narrow a set of ideas and institutes it knows (Stone 2003). One specific criticism, of the GDN's exclusive focus on economics, led to a Annual decision of the Board in 2002 to encourage Other collaborative Conference 20% programs 20% regional networks to adopt a more multidisci- plinary orientation to future research funding. The speed at which GDN would move to a broader social science focus remains a heav- ily debated issue, with officials of the region- GDNet 11% al networks, which were deliberately created to strengthen economic research, voicing concern that they were not consulted on this decision. Global research Regional projects 8% research In 2001, GDN management commissioned competitions a study of the regional research competitions 32% Global awards 9% in six networks (and an independent DGF- required evaluation is scheduled for FY04). The study concluded that GDN regional hubs Source: Financial Performance--Report of Independent Auditors, September 13, 2002. grant research awards in a competitive man- ner and that research has been of high quality edge-sharing tools--a monthly e-mail newslet- and policy-relevant, with the "potential to ter, online discussions, and a Web site contribute to development." The review (GDNet) (see box 4.4 for further detail on noted that transparency could be enhanced by these activities). providing detailed selection criteria, with each While GDN is still a program in formation, call for proposals and competition improved external views and ongoing evaluation point by making grants available to "researchers to a number of issues confronting its rele- from countries with weak economics scholar- vance, long-term sustainability, and relation- ship"; and capacity-building impact could be ship to the World Bank. GDN used a broadly increased through addition of a training and participatory, Web-based, consultation process mentoring program for researchers. It also in establishing its governance structure. The suggested that steps should be taken to process led to the creation of a 17-member improve the impact of research. governing board composed of 10 individuals Both that study and interviews of a large selected from each of the 10 participating number of actors involved in GDN, conducted regional networks; one individual selected as part of an ongoing case study of GDN, pro- from each of the three international economic, vide insights into a global program that is in a political science, and sociology associations; formative period. The case study, which is part two at-large; and one each from the World of phase 2 of OED's review of World Bank Bank and the UNDP. Nonetheless, GDN is still global programs (World Bank forthcoming), is seen by some "as a child of the World Bank" exploring questions of GDN's relevance and (Stone 2003, pp. 43-61). A possible reason for effectiveness, such as the program's relation- this is that initial legwork for establishing ship and value added vis-à-vis other regional GDN was carried out by a member of the research organizations; its prospects for long- Bank's staff, who later retired from the Bank term sustainability; and its relationship with to become GDN's director. the World Bank and the Bank's role in pro- A number of donors and some in the inter- gram oversight within the context of GDN's national research community further criticize "spin-off" from the Bank.53 S H A R I N G K N O W L E D G E W I T H C L I E N T S A N D PA R T N E R S | 31 BOX 4.4: GDN Major Activities since 1999 · Regional Research Competitions--Some US$13 million has been allocated to fund just over 400 projects. · Annual Development Conferences--the fourth conference, held in Cairo in early 2003, was attended by some 500 participants from 95 countries. · Global Research Project--Explaining Growth project, near completion, has two overview volumes and seven regional volumes. Two new projects are Bridging Research and Policy, and Understanding Reform. · GDN's Web site provides a monthly electronic newsletter to some 11,000 subscribers, access to some 2,000 researcher and organization profiles, and just over 6,000 publi- cations. · Other collaborative projects (with the World Bank, U.S.-based National Institutes of Health, and scholarship program with IMF). Crosscutting Issues effective strategic direction and oversight, Although still at early stages of implementa- commensurate with available resources. tion, all three initiatives--the Development · As the initiatives mature, they and the Bank Gateway, the Global Development Learning need to define and better manage the Bank's Network, and the Global Development Net- evolving roles and responsibilities. work--have demonstrated that they can Early experience suggests that integration of enhance knowledge sharing and client capaci- the Development Gateway and GDLN into the ty to acquire and use knowledge. Progress in Bank's country programs can facilitate the dis- the initial implementation of each initiative semination of knowledge and iterative knowl- has been quite rapid, and the Bank's weight in edge sharing that is important for program- the international development community has matic lending, participatory ESW, and poverty been significant in mobilizing the participation reduction strategy paper (PRSP) and CAS of partners. preparations aimed at building country owner- But four major challenges remain to be met ship of development strategies and processes. to ensure the long-term sustainability and dis- These processes require cost-effective means of tinct program value of each of the initiatives: holding dialogues, exchanging good practices, · The utility of each program--at the coun- and involving multiple stakeholders--and for try, Regional, and global levels--needs clar- all of these operational activities, the global ification and continual review in a rapidly knowledge initiatives can provide useful plat- changing technology and development forms. In addition, the research capacity build- environment. ing supported through GDN potentially rein- · The financial sustainability of each pro- forces the greater collaboration and use of gram and the independent, country/Region- local expertise that clients and international al components of each program need to be experts have recommended. secured. Anecdotal evidence from LCR indicates fur- · Consolidation of each program's gover- ther that events involving stakeholders in sim- nance arrangements is needed to ensure ilar Bank (or other donor) projects across 32 | S H A R I N G K N O W L E D G E countries that are carried out in conjunction seamless as possible. In interviews, some with existing knowledge networks and use operational staff emphasized that bureau- Information and Communication Technology cratic and funding hurdles to the use of (ICT) capacity in order to sustain and deepen these capacities (notably GDLN) are a dis- dialogue have more impact on clients than incentive. real-time GDLN learning activities conducted For each of the initiatives, as well as for the as stand-alone events with little follow-up. independently owned country partner organi- There is also some experience to suggest that, zations, there is the further complication that given the complementary nature of the initia- "involvement with the Bank may pose trade- tives, the reach and efficiency of knowledge- offs," given civil society organizations' con- sharing and capacity-building activities can be cerns about the directive nature of the Bank's extended by joint undertakings (for example, role (Walker 2003). While the examples cited of country-based GDLN and Development above indicate that such concerns need not Gateway organizations) built around Bank discourage partnerships, this view reinforces projects and knowledge networks within and the importance of consolidating governance across countries. For example, LCR has arrangements and management processes to organized collaboration between the GDLN effectively address the Bank's evolving role in and the Development Gateway, with support each initiative. from an internationally respected NGO, that has fostered problem solving among dispersed and marginal groups of indigenous people Integrating Knowledge Sharing across the Region. From the outset, the knowledge initiative envi- Despite their promise, use of these capaci- sioned a rapid transition from building a better ties in operations has, however, been very lim- knowledge management system to expanding ited, for the following four reasons: knowledge sharing with clients and partners. · Many country-based partner institutions This transition remains incomplete. are not yet fully established (especially Knowledge sharing is increasingly becom- Country Gateways and DLCs still in their ing a way of doing business, as indicated by planning or implementation phases). the growing number of innovations in the · Use of ICT for development "has not pene- Bank's interactions with clients. But most trated the agenda of Regional Bank staff" CASs still do not make explicit what their (Walker 2003). knowledge objectives are, or how they pro- · Clear articulation of country knowledge pose to help countries leverage knowledge to demands is essential to move the initiatives achieve overall development objectives. The from a supply-driven to a demand-driven knowledge dimensions of operations are only mode. With a clear handle on demand, gradually intensifying, and project teams only country staff should be in a position to gradually enhancing their role as knowledge identify the right instrument to build into brokers. Moreover, the three global knowl- programs and relevant units (WBI, net- edge initiatives reviewed, which have started works) and capacity (GDLN, Development up quickly, face issues of financial sustainabil- Gateway, thematic groups) to mobilize in ity, governance, and unclear partnership roles meeting the demand. and responsibilities. As a result, the impact of · But Bank processes, resource allocations, the initiative--as perceived by clients--is still and electronic systems are not yet in place limited, although much welcomed and seen as to make this response to country demand as moving in the right direction. 5. THE SUPPORTING INSTITUTIONAL INFRASTRUCTURE Over the past six years, the Bank has built up its information technology system; fostered a more open, knowledge-sharing culture for itself; and established new knowledge and learning staff positions at various levels across the institution. But it has not established adequate governance arrangements, nor adequate processes and incentives for institutionalizing knowledge sharing as a way of doing business. A main conclusion of this review is that these management shortfalls account in large part for the evaluation findings of uneven progress in the implementation of the initiative, and for its limited impact thus far on the Bank's service to its clients. Experience from a broad array of institutions within project budgets).55 Figures 5.1, 5.2, and outside of the Bank shows that, to be effective, 5.3 provide further elaboration. a knowledge-sharing system needs to be tight- Governance approach. From the outset, the ly linked to the institution's core business Bank has taken the stance that knowledge strategy. It also needs to take an integrated sharing is not a line of business, but a way of approach to technology, people, and process- doing business, for which all units have es. This chapter examines the adequacy and responsibility. This decentralized approach to efficiency of Bank performance in establishing the governance of the knowledge-sharing these core institutional support elements. process has allowed for considerable program innovation. But the approach has been accom- Resources and Governance panied by two governance shortcomings: a lack of clearly defined roles and responsibilities Resources. Bank expenditures on knowledge and an absence of systematic monitoring and sharing programs and activities over the evaluation. These shortcomings account in FY97-02 period total just under US$220 mil- large part for the unevenness in the pace of lion in combined budget and trust funds knowledge-related change across the institu- resources.54 For the three years the Strategic tion and for the weak linkage between knowl- Compact was in place (FY98-00), knowledge edge sharing and other core business processes. management budget expenditures (exclusive of trust funds) averaged about 3.4 percent of Roles and responsibilities. At the corporate total direct expenditures, declining to an aver- level, a senior-level Knowledge and Learning age of about 2.4 percent in FY01 and FY02. Council met regularly from 1998 to 2000, but Network knowledge expenditures over the its focus was on allocating centrally provided period have been more than double those of resources for staff learning. The council never the Regions (exclusive of those incorporated functioned as a significant governance mecha- 33 34 | S H A R I N G K N O W L E D G E FIGURE 5.1: FIGURE 5.2: Bankwide Expenditures on Knowledge Management Rose Network Expenditures on Knowledge Management Exceed through the Strategic Compact Period those of the Regions 60 60 Trust Funds Regions Bank Budget Networks 50 50 ion) ion) 40 40 ($mill ($mill es es 30 30 xpenditure xpenditure taloT20 taloT20 10 10 0 0 FY97 FY98 FY99 FY00 FY01 FY02 FY97 FY98 FY99 FY00 FY01 FY02 FIGURE 5.3: The Majority of FY02 Expenditures Were for General Knowledge Management Activities Knowledge Mgmt TA 5% Thematic Groups 19% Knowledge Mgmt Systems Dev 0% Knowledge Mgmt General Knowledge Regional Support 5% Mgmt 65% Advisory Systems 5% Maintain Databases 1% Source: SAP/Business Warehouse (BW). T H E S U P P O R T I N G I N S T I T U T I O N A L I N F R A S T R U C T U R E | 35 nism for knowledge sharing, and there has a steering committee has been created at the been limited corporate-level strategic direction vice-presidential level to provide corporate and oversight. From the start of the knowledge guidance and oversight of the knowledge strat- initiative, there has been a small coordinating egy's implementation. A new staff-level steer- unit that has served to promote the buildup of ing committee will foster coordination among tools and activities and their mainstreaming activities and bring issues to the vice presi- across the institution. This role as the champi- dents. Despite these organizational measures, on of knowledge sharing is different from the explicit clarification of roles and accountabili- monitoring and standard-setting role played ties is still lacking. And without clear roles and by other Bank units responsible for fully main- responsibilities, objectives needed for monitor- streamed activities--for example, units ing and evaluation are not likely to be well responsible for leveraging safeguards or assur- defined. ing financial accountability. Monitoring. In contrast to good practice in Responsibility for the networks' knowl- other organizations, knowledge sharing edge-sharing function rests with the sector remains a largely unmonitored process at the boards, which have provided minimal direc- Bank. The task force that reviewed the knowl- tion and oversight.56 At best, knowledge-shar- edge initiative's progress in 1999 emphasized ing strategies at the network or sector depart- the need to devise a system of metrics that ment level have been implicit, not explicit. would track future progress (Prusak 1999). Knowledge management activities are neither While subsequent management documents executed in the most efficient, cost effective have repeatedly made a commitment to carry way, nor do they achieve their full potential, out this recommendation, the evaluability of largely because there is a lack of management all new knowledge-sharing programs and and coordination. In such a decentralized activities remains limited--beyond its broad system, each group has a different perspec- statement of goals, the Bank has not set moni- tive, different methodologies, and uses differ- torable and time-bound goals at the corporate ent tools, which can lead to duplication of or vice-presidential levels. As noted above, the effort. most recent (2001) strategic framework for the Although the Regions' knowledge-sharing knowledge initiative emphasized three broadly responsibilities are largely client oriented, pol- directional objectives--improving the quality icy guidelines have not been established for of operations through internal knowledge inserting knowledge-sharing strategies into sharing; expanding knowledge sharing with CASs, analytical and advisory services (AAA), clients and partners; and enhancing client or lending operations. In the absence of both, capacity to acquire and use knowledge effec- clear guidelines on what's expected at Region- tively. But the framework did not set specific al and country levels and measurable Region- outcome objectives, intermediate targets, or al and country objectives, there is no way to milestones. Networks do not have common know if goals are being achieved. Moreover, standards or procedures for tracking, report- the respective roles of the Regions, networks, ing, and monitoring the progress of programs and WBI in enhancing external knowledge and activities. Nor have Regions developed a sharing have not been spelled out or systemat- systematic way to track knowledge-sharing ically coordinated. activities undertaken as part of country lend- Since 2002, management has taken steps to ing and nonlending operations within existing revise organizational responsibilities for operational reporting.57 knowledge sharing. WBI has been tasked with the catalytic role of ensuring effective integra- Evaluation. Self-evaluation of knowledge- tion of Network and Regional efforts, and sharing activities is much less systematic than with becoming the locus for strategic direction self-evaluation of other Bank work (OED for the Bank's knowledge agenda. In addition, 2003b). The one-off assessments of specific 36 | S H A R I N G K N O W L E D G E tools and activities, such as review of Bank sites; a consolidated store of quantitative busi- internal and external Web sites and network ness data; and electronic capture and storage and sector staff feedback surveys, do not make of qualitative data, with 24/7 access and cus- their assessments against stated outcome tomer support. People both inside and outside objectives nor allow for comparisons over the Bank now have faster access to institution- time or across units. For example, they do not al information and geographically broader ask comparable questions about usefulness of connectivity. activities. Existing self-evaluation frameworks But these discrete improvements do not and procedures at the country and project lev- add up to a fully articulated strategy for meet- els also do not adequately address knowledge- ing the Bank's core business objectives, and sharing processes. Nor do OED's independent there is need for additional technology country assistance and project evaluations; improvements in support of the Bank's opera- most fail to look beyond the quality of under- tional work. Information technology (IT) staff lying AAA work to the adequacy of knowl- need to work with operational staff to edge-sharing processes and their impact on enhance knowledge capture and sharing Bank programs. through the Bank's internal and external Web, Outside the Bank, most organizations with the Development Gateway, or other means. leading knowledge management systems intro- ISG's strategic framework for FY03-05, duce performance indicators not in the early which focuses on this issue, includes as main stages of implementing their systems, but as components e-business partnerships to facili- part of a second stage. Even though the Bank tate knowledge sharing with clients and IT has progressed beyond the tool-building early improvements in support of internal business stage to the second stage of institutionalizing simplification.58 knowledge-sharing activities, it lags in moni- Before moving forward, however, there toring and evaluation--not only in compari- should be a comprehensive external evaluation son to other public and private enterprise lead- of how well the initial strategy and its achieve- ers, but also in relation to practice typical of ments are aligned with core business needs and other areas of innovation in the Bank (for processes. The ISG's FY03-05 strategy extends example, the Comprehensive Development the steps taken so far and addresses outstand- Framework [CDF] and PRSP)--and indeed, to ing issues from the current strategy; yet there its own knowledge initiative plans. has been no validation of the overall strategic choices of the past. Most ongoing data track- ing and surveys are at the level of individual Technology initiatives such as e-mail, videoconferencing, From the start, the knowledge initiative has and discussion spaces, but they do not assess included a heavy emphasis on the Bank's infor- the effectiveness of the components taken mation technology, with the dual aims of together, in contributing to staff needs for col- improving Bankwide connectivity and speed- laborative tools and the Bank's overall mis- ing access to Bank information by staff clients sion. Before launching into new technology and the wider development community. The applications or other improvements, it would 1997 Strategic Compact envisioned an IT sys- be useful to consider to what extent all tools tem that would replace what had become anti- currently available have actually promoted quated systems--for example, no shared better team-based work practices and drives, a plethora of fragmented applications, increased efficiency. Given that ISG is now and inadequate connectivity between head- moving to a new set of objectives, this would quarters and country offices. seem to be an opportune moment for such an Many improvements have been made. The overarching assessment. Bank now has a high-speed global network The ultimate effectiveness of intended tech- with standardized applications in about 100 nological improvements does not depend on T H E S U P P O R T I N G I N S T I T U T I O N A L I N F R A S T R U C T U R E | 37 the information systems, however, but on bet- incentives do not exist for supporting the ter content management. The quality, timeli- knowledge initiative, and 60 percent of those ness, and tailoring of knowledge and informa- respondents pointed to time constraints as the tion on the Web were discussed above in major barrier. Because knowledge-sharing pro- Chapter 3. In addition, there is need for agree- grams are informal--that is, not given task ment across the institution on common poli- designations for the Time Recording System cies and practices for managing distributed (TRS)--staff participation in internal knowl- content. Questions to be answered include, edge-sharing activities is voluntary. It carries "What kinds of operational (lending and non- no explicit expectations for actively contribut- lending) information will be posted on the ing to the exchange of knowledge before, dur- Intranet?" And "How will Web-based knowl- ing, or after operations. Moreover, it is not edge be validated, how often will it be updat- enough for organizations to simply facilitate ed, and how and when will it be archived?" the collection and sharing of knowledge. An The point is not to standardize the content but outside study of Bank knowledge sharing, to clarify processes for disseminating and described in box 5.1, emphasizes the impor- maintaining it. For this, corporatewide tance of creating conditions--specifically, processes for ensuring higher-quality content team autonomy, appropriate workload, and a management are needed. combination of "cosmopolitan and local" team composition--in order to empower teams to apply and feed back operational People knowledge. The initiators of the Bank's knowledge initia- tive emphasized that the key to improved Summing up knowledge sharing was to change the internal culture of the Bank.59 This emphasis reflected The Bank has made more progress in establish- the widely expressed view, seen in the knowl- ing the architecture to support its knowledge edge management literature, that while tech- initiative than in creating the governance nology is an important enabler, people's arrangements and work processes for carrying behaviors are what make for effective knowl- it out. As a result, the strategic intent of mak- edge sharing and use. As a result, several steps were taken to create a work environment in which staff would be encouraged to share knowledge routinely. Positions of knowledge BOX 5.1: coordinator and information analyst were cre- Conditions for the Effective Use of Shared Knowledge ated in networks and Regions, with responsi- bilities for developing and championing new An in-depth study of Bank project preparation and supervision tools and activities. Knowledge sharing was shows that "accessing shared knowledge does not necessarily made part of the Bank's performance evalua- improve the quality of projects. Instead, the three conditions of tion criteria for all staff in 1998, and incorpo- team autonomy, workload, and team composition moderated rated into the Bank's first formal mission state- the effects of distributed knowledge on team performance. . . . ment in 1999. More autonomy enables teams to overcome political pressures Although staff feel a greater openness in the exerted by those who provide the knowledge. Lower overload Bank,60 the changes in culture have not been enables teams to manage the time demands involved in using enough to create the incentives and other con- distributed knowledge without incurring excessive opportunity ditions required for staff to contribute to the costs. And [the combination of] cosmopolitan and local team capture, exchange, and application of opera- members enable teams to interpret knowledge that they access tionally relevant knowledge and experience. more appropriately." Two out of three task team managers inter- Source: Haas 2002. viewed for this review indicated that adequate 38 | S H A R I N G K N O W L E D G E ing knowledge sharing a way of doing business sharing, and learning, and between knowledge- has been only partly realized. What remains to sharing activities' core operational processes of be done is to clarify roles and responsibilities; CAS preparation, lending, and nonlending develop stronger linkages among the three services; and establish frameworks for moni- knowledge processes of research, knowledge toring and evaluation. 6. CONCLUSIONS AND RECOMMENDATIONS T he Bank did the right thing in committing services; operational teams need to do more to itself to a major knowledge initiative. The capture and feed back more systematically les- rationale--to improve development outcomes sons and good practices from on-the-ground by fully exploiting both the revolution in work; and Bankwide efforts need to be made information technology and the Bank's com- to improve content management of distributed parative advantage as a source of development knowledge to ensure quality, timeliness, and-- knowledge--is highly relevant to changing most important--operational relevance. client needs, international development prac- The second shortcoming is the failure of the tices, and Bank interactions with its clients. Regional units, country teams, and task man- Since the start of the initiative in 1996, the agers to develop a more strategic approach to Bank has moved quickly to upgrade its infor- the knowledge dimensions of the Bank's serv- mation technology capacities, establish new ice to its clients. Knowledge strategies should knowledge-sharing programs and activities, be made more explicit in CASs and operations, launch several global knowledge partnership and task teams should be given the explicit programs, and create a more open and knowl- responsibility and accompanying incentives edge-sharing organizational culture. Both for implementing the strategic objectives. This staff and clients have gained faster and easier systematic integration of knowledge into access to knowledge and information as a country programs and projects includes the result. incorporation of the global knowledge initia- While the fundamental thrust of the knowl- tives, which needs to occur not as an end in edge initiative is on target, there are two short- itself but as an enhancement to CAS or project comings that, if left unaddressed, will prevent objectives aimed at building clients' capacities the initiative from achieving its purpose of to acquire and use knowledge effectively. So enhancing client use of knowledge to achieve far, no specific objectives or timetables have development goals. been set regarding this second--external The first is the failure to take the strategy's knowledge-sharing--phase of the Bank's knowledge aggregation and general dissemina- knowledge initiative. Nor has effective coordi- tion phase to the next level--providing direct nation in implementing the knowledge initia- support for task teams in their operational tive become routine among corporate, net- work. Or, in other words, to institutionalize work, and Regional units. the knowledge strategy by applying its tools Greater strategic direction by senior man- directly to the Bank's core business processes. agement across the Bank is needed to guide Three changes are needed to move to this this mainstreaming of the knowledge initiative higher level: network knowledge-sharing and to ensure that individual programs and activities (especially those of thematic groups activities, which are the responsibilities of the and advisory services) need to devote more various units, cohere in a way that achieves the time and attention to working with frontline Bank's intention to make knowledge sharing a staff in support of lending and nonlending way of doing business. 39 40 | S H A R I N G K N O W L E D G E To realize more fully the knowledge initia- ­ Networks should set clear objectives for tive's potential to enhance the operations of anchor, thematic group, and advisory the Bank and empower clients to meet their service support of operational teams. development goals, this review recommends ­ Regional and country units should make three sets of actions: explicit the knowledge objectives of · While recognizing that all units across the CASs and projects and the strategies to Bank have responsibilities for making be used in leveraging knowledge in sup- knowledge sharing a way of doing business, port of the achievement of overall devel- management should exercise greater strate- opment goals. gic direction and oversight over the institu- ­ Networks and Regions should strength- tion's knowledge processes. More specifi- en their respective responsibilities in the cally, management should: capture, validation, and application of lessons learned and good practices. ­ Define clear responsibilities and account- abilities of corporate, network, and · Frameworks should be established for mon- Regional units for integrating knowledge itoring and evaluating network, Regional/ sharing into the Bank's core business country, and global knowledge-sharing pro- processes and better integrating knowl- grams and activities. This involves: edge and learning programs. ­ Setting outcome objectives and support- ­ Ensure that incentives are aligned with ing indicators (baselines, monitorable responsibilities, especially at the task targets, and benchmarks of progress). manager level. ­ Defining procedures, roles, and responsi- ­ The proposed Global Programs and bilities for monitoring progress and eval- Partnerships Council should articulate a uating achievements against the stated strategic approach to the Bank's role in objectives. existing and in any new global knowl- ­ Also, depending on the outcome of its edge initiatives, and the Bank should ongoing baseline assessment of the per- prepare and institute a long-term plan formance of sector boards, the Quality for the GDLN--including a financial Assurance Group (QAG) should contin- plan as well as clarification of the Bank's ue to review at appropriate intervals the forward involvement and the responsi- quality of the networks' knowledge shar- bilities of different Bank units for man- ing. OED country assistance, sector and aging that involvement and the integra- thematic, and project evaluations should tion of the GDLN into Bank country look beyond the quality of underlying programs. analytical work to the adequacy of the · Network and Regional knowledge-sharing knowledge transfer processes and their activities should tightly link to core busi- impact on the development effectiveness ness processes and move beyond knowledge of Bank-supported programs. And both aggregation and access to increased focus the Quality Assurance Group and OED on knowledge adoption, adaptation, and should build client feedback into their use. For example: assessments. ANNEX A. KNOWLEDGE-SHARING CHRONOLOGY (1996­02) 1996 1997 1998 1999 2000 2001 2002 Strategy President's Strategic World Development Knowledge Strategic Knowledge- milestones Annual Meeting Compact Report focuses on management Forum sharing framework speech features focuses on knowledge for included in identifies proposed "Knowledge retooling development World Bank knowledge Bank" knowledge mission sharing as key base statement action area Individual Thematic groups & 100 thematic groups Global Global Development programs & advisory services and 16 advisory Development Development Gateway launched activities launched services established: Network Learning 4 of 6 Regions have launched Network databases and launched knowledge-sharing activities Institutional infrastructure Governance Networks established Knowledge Knowledge Strategic Compact VP with knowledge and learning management assessment of Knowledge management council unit moved progress; knowledge Steering mandate; knowledge established from ISG to management unit Committee management unit Operational moved to World established established in Core Services Bank Institute Information Solution Group Technology External Web Field offices Intranet site posted linked to global revamped; communications Lotus Notes system launched People/ Budget for Knowledge sharing resources knowledge becomes indicator in management staff Overall allocated Performance Evaluations 41 ANNEX B. INVENTORY OF BANK KNOWLEDGE- SHARING ACTIVITIES (Activities in italics are included in the evalu- Global Knowledge Initiatives ation) Development Gateway Global Development Learning Network Regional/Country Activities Global Development Network Infodev Publications, newsletters, and Web sites WorldLinks Databases African Virtual University Knowledge events (conferences, workshops, continuous learning events) Country global knowledge initiatives Corporate Knowledge-sharing components of OED CAS/PRSP processes and country DEC operations WBI World Bank Internet, databases, Network/Sector Activities public information Publication sales and dissemination Thematic groups World Bank libraries Publications, newsletters, and Web sites Databases Networking and knowledge-sharing events (Brown bag lunches, workshops) Direct support to task teams (informal advice, peer reviews) Advisory services/Help desks Request response Knowledge aggregation Direct support of task teams, Network/sector management Anchors Publications, newsletters, and Web sites Databases Direct support to task teams Knowledge events (sector weeks, other) Knowledge partnerships 43 ANNEX C. LESSONS FROM THE LITERATURE ON KNOWLEDGE MANAGEMENT Knowledge management was just emerging as human capital into the structural capital of an a major organizational strategy when the Bank organization" (Skyme 2000, p. 4). announced its knowledge initiative in 1996. Over the past 10 years, experience with Many corporations and a number of large knowledge management programs has led to a public organizations had recently established, shift away from this view of knowledge as an or were in the process of establishing, knowl- entity that exists independent of people or con- edge management programs based on the view text and that can be moved about and manip- that organizations needed to be smarter, faster, ulated for organizational advantage. The core more innovative, and more agile in the emerg- concept is increasingly seen as "a process for ing global knowledge economy. optimizing the effective application of intellec- Since then, a substantial literature has built tual capital to achieve organizational objec- up that documents experiences in implement- tives" (U.S. Department of the Navy 2001, p. ing knowledge management programs. In 7). Greater attention is being paid to tacit addition, there has been a growing body of knowledge and how it can be effectively research on the determinants of successful shared. Also, studies have shown that pro- knowledge transfer. Five key messages from grams cannot assume that the transfer of that literature provided useful insights for knowledge will come automatically with OED's evaluation of the implementation of the knowledge sharing. Rather, tools and activities Bank's knowledge-sharing initiative. need to make sure that what is shared can be adopted, adapted, and used. With the change in thinking has come a change in terminolo- Knowledge sharingis a process, gy--with the designation knowledge sharing not a line of business tending to replace knowledge management. Much of the early practice and writing on Organizations have begun, therefore, to managing knowledge focused on explicit reframe the way they approach the practice of knowledge--"knowledge one can see and knowledge sharing, the key change being a document"--and on how to capture, organ- move away from building knowledge reposito- ize, and disseminate it. The core aim was ries to integrating knowledge sharing into core defined as "knowing what we know, captur- business processes. In other words, knowledge- ing and organizing it, and using it to produce sharing programs are being focused squarely returns" (Stewart 1997, p. 112). Knowledge on the core processes of an organization, with was understood as a substance to be engi- those processes taken as the target points for neered--produced, catalogued, warehoused, applying knowledge-sharing tools and activi- and shipped. And knowledge management ties. And the purpose is being defined as was seen as an activity that could be assigned empowering process managers with the means to a designated few in an organization that to achieve performance objectives by embed- would be responsible for "decanting the ding knowledge-sharing tools and capacities 45 46 | S H A R I N G K N O W L E D G E into their work processes and practices. This, 2. The appropriateness of the arrangements-- in turn, requires that knowledge-sharing pro- or rules of engagement--by which the grams be built on a clear view of the points in knowledge is exchanged: in particular, the the business processes where knowledge can be extent of agreement on the goals of the leveraged for greatest value. With this change knowledge exchange, differences in capaci- to a more integrated or operational approach, ties of the parties involved and how those knowledge sharing is increasingly being seen differences are taken into account, and not as a function of an organization but as a understanding of the circumstances in competency of its people. which the knowledge sharing takes place 3. The applicability of the knowledge-sharing Success liesin the application-- activities to the kind of knowledge needed not just the aggregation and sharing-- and the context within which it is of knowledge exchanged (Cummings 2002). Studies that deal with what makes for the suc- cessful transfer of knowledge emphasize that Implementation of knowledge-sharing knowledge has to be "internalized" if it is to initiatives requires a planned transition from be used effectively. While it can be disseminat- advocacy toinstitutionalization, and typical- ed globally at great speed, it has to be adapted ly occurs over a period of about three years locally by "incumbent firms" or the "local In a benchmarking study of how various doers of development." And that adaptation organizations (including the World Bank) have has been shown to involve active, extended implemented knowledge-sharing programs, learning rather than simple communication the American Productivity and Quality Center processes. identified five stages that organizations typi- A survey of the literature indicates that the cally go through, and the key steps in each of extent to which knowledge is effectively trans- those stages (APQC 2000a). The progression ferred, absorbed, and applied depends on how is illustrated in the following diagram. well knowledge-sharing efforts handle three One of the overarching findings from that dimensions of the knowledge-sharing process: study was that the most successful efforts have 1. Accurate assessment of the type of knowl- involved a plan for how to transition from edge that is needed--whether explicit or building new knowledge-sharing tools and tacit and whether embedded in tools, rou- activities and creating awareness of their value tines, or networks of people for core business processes to actually embed- Design and Launch Get Started Develop Strategy Expand and Support KM Initiatives Institutionalize KM STAGE 1 STAGE 2 STAGE 3 STAGE 4 STAGE 5 · Advocate · Explore and experiment · Pilot · Develop organization- · Integrate strategy into · Establish cross-functional group · Measure results wide strategy management process and oversight body · Identify roles and · Shift implementation from · Outline objectives responsibilities KM team to operational staff · Establish measures, formal · Codify expected outcomes budgeting, and reporting · Monitor value to the organization A N N E X C | 47 ding knowledge sharing into those processes While all may exist in an organization at the throughout the organization. While experi- same time, recent studies of practitioner com- ence has shown that this transition tends not munities emphasize the importance of clearly to be strictly linear, the evolution from "advo- differentiating among their potential contribu- cacy to institutionalization" in many reported tions to organizational objectives and the ways cases has taken some three to five years. they will, therefore, be managed, supported, and assessed. In at least one corporation, this evolution toward more rigorous expectations Communities of practice can play a variety of the role and performance of communities of roles within organizations, but all need to has led to the establishment of general guide- demonstrate value and be accountable lines and "community charters." Typically, a Increasingly, private and public sector organi- charter explains what type of community has zations are providing resources, time, and col- been formed and why; identifies how the com- laborative tools to formal and informal net- munity will support long-term business objec- works of workers--often called communities tives; states what the primary and secondary of practice--to increase the sharing of lessons objectives of the community are and how learned and experience. While the value of progress toward those objectives will be meas- these communities to organizations was ini- ured; outlines what the principle activities will tially taken largely on faith, expectations be for achieving the objectives; and indicates about their roles and scrutiny of their per- the expected benefits of those activities for formance have tended to increase commensu- each category of community participant. rate with organizations' investments of staff Although early efforts to monitor commu- time and funding for community activities. nity performance attempted only to measure Over the past decade, at least three different activities (for example, by counting Web site types of communities have emerged within hits, attendance at meetings, and responses to large organizations. The first is communities e-mail queries), a more recent approach starts of interest, which are groups of individuals from the proposition that communities typi- with a common interest that may or may not cally aim to help individual members, relate to their day-to-day work, but that share strengthen the practices of a community (or ideals and communicate or collaborate. The other business units or teams), and build second is communities of practice, which are broader organizational capacity. The assess- groups of individuals who share a common ment framework is, therefore, built on exam- work practice over time. These groups tend to ining outcomes and value added in terms of cut across traditional organizational bound- which of these objectives a specific community aries and enable individuals to acquire new aims to achieve and the relevance of its stated knowledge or new skills at an accelerated rate. objective(s) to specified measures of business For the most part, they build and apply the performance (Fontaine and Millen 2002; same practices, and negotiate which methods McDermott 2002). work best and when they are most useful. They are guardians of competence in that Effective knowledge-sharing programs practice within an organization and they help require comprehensive approaches to each other to develop the competence to con- content management tribute individually within their business units. The third is intentional communities of prac- To be effective, knowledge-sharing programs tice, which are groups with some specific busi- need to ensure that the knowledge and infor- ness goal that the community is collectively mation that are made available are not only accountable to deliver and which have a fixed easily accessible but also readily applicable. lifespan--until what they are aiming for has For this, programs need to take an active and been delivered. comprehensive approach to managing their 48 | S H A R I N G K N O W L E D G E content, putting knowledge or information tions of who is responsible for keeping knowl- together in such a way that it can be immedi- edge and information up to date and at what ately adapted and used. intervals, taking into account that information Although many knowledge management loses value over time and storage is not cheap. programs, in their early phases, have tended to Corresponding criteria that some organiza- view this process of content management in a tions now use for assessing the effectiveness by narrow, technological perspective of online which they aggregate, store, and disseminate presentation and navigation, experience has their knowledge are: shown that more is needed if the right knowl- · Relevance: systematic, ongoing knowledge edge is going to get to the right place in the assessment and inventory of knowledge right time to contribute to intended outcomes. assets Specifically, knowledge-sharing programs · Accuracy: determined by peer validation, need to encompass processes for assuring quality, freshness, and applicability of distrib- not on an ad hoc basis, but anchored in the uted knowledge, with the aim of ensuring the regular workflow of staff availability of operationally relevant knowl- · Access: based on ability of intended users to edge and information at the point and time of obtain the knowledge or information they need. This entails processes of content assess- need to make decisions or complete tasks ment, or identification of knowledge gaps; relevant to their work content aggregation, including the validation · Currency: judged according to policies and and packaging of knowledge and information procedures for retiring and archiving out- in ways that best enable their adaptation and of-date content and for maintaining a deep customization to specific users' needs; and understanding of users' changing knowl- content maintenance, based on clear indica- edge needs. ANNEX D. EVALUATION FRAMEWORK FOR KNOWLEDGE-SHARING PROGRAMS AND ACTIVITIES: CRITERIA, INDICATORS, AND EVIDENCE The evaluation framework used by this review in assessing the Bank's expanded internal and external knowledge- sharing activities is summarized in the following table. Criteria Indicators (illustrative) Evidence (illustrative) Relevance · Programs and activities anchored in corporate and · Knowledge-sharing strategic objectives, country priorities including client and audience, well defined · Programs and activities built on Bank comparative and linked to corporate, sector and thematic, advantage country assistance strategies, and core · Institutional support aligned with knowledge- business processes sharing strategy Quality and timeliness of · Aggregated knowledge is tailored and timed to client · Staff, client, expert reviews and surveys knowledge products and needs, clearly presented, technically sound, and state-of- · Bank content management processes services the-art Accessibility and reach of · Intended users/clients have ready access to up-to- · Dissemination tracking, usability testing, tacit and codified knowledge date knowledge and expertise needed to do their jobs usage monitoring of published and on-line knowledge and information and knowledge services · Staff, client participation in knowledge- sharing events · Staff, client feedback surveys, focus groups, reviews Utility · Knowledge products and knowledge-sharing activities · Knowledge strategy articulated in CAS/Poverty incorporated into core business processes Reduction Strategy Paper and operations · Shared knowledge adapted and applied by clients in · Knowledge products and activities built into policies, programs, and institutional developments staff and client (team) learning activities · Lessons learned and good practices captured and · Staff and client feedback surveys feedback to Bank and client · Program and activity self-assessments · QAG and OED assessments of knowledge- sharing process in country programs, and lending and nonlending services Likely impact · Bank and client knowledge bases and capacities · Ongoing knowledge assessments enhanced · Self-, independent assessments of program · Targeted improvements in Bank portfolio performance and activity outcomes relative to stated · Interim and longer-term development objectives objectives achieved Cost effectiveness · Programs and activities carried out without more · Tracking and benchmarking of the costs resources than necessary to achieve objectives of programs or activities Source: OED 2002a. 49 ANNEX E. METHODOLOGY NOTE This note describes the approach followed for bias is that the interviewers were identified each of the evaluation surveys and back- and a list of interviewees constructed with the ground reviews undertaken for the knowl- help of country offices. But this bias is miti- edge-sharing evaluation. The reliance on sur- gated by OED's role in the final selection of veys, which is greater than usual in OED the interviewers, specification of categories of evaluations, reflects in part the scarcity of interviewees, and final selection of names from reporting, monitoring, and self-assessment lists provided, as well as by the preparation of that currently characterizes the individual the interview protocol and synthesis report by knowledge-sharing programs and activities. an outside consultant. A follow-up workshop was held at Bank headquarters, November 25-26, 2002, to Client Survey and Workshop deepen understanding of the results obtained A survey was conducted to seek client views from the client survey and to seek recommen- on the relevance, quality, and usefulness of dations on ways to strengthen the Bank's Bank knowledge-sharing efforts. A total of knowledge products and services. Participants 121 individuals participated in survey inter- included nine survey participants and four views--in roughly equal numbers from each of external experts, as well as OED and other five countries--over the period June-Septem- World Bank staff. ber 2002. Participants were from government, the private sector, research and academia, Advisory Services Survey NGOs, and media. The countries were Bangladesh, Brazil, Poland, Senegal, and Tan- OED and the World Bank Institute conducted zania. a joint survey of 19 Bank advisory services/ Each survey respondent participated in an help desks in February-March 2002 to obtain in-person interview conducted by a resident information on their activities, outputs, and expert commissioned by OED. Interviews outcomes. Fifteen services responded, for a were based on a common guideline prepared response rate of 75 percent (see list of respon- by OED with the help of an external survey dents below). expert. The guideline included open-ended, The e-mail­based survey covered five main qualitative questions and quantitative ques- topics: (1) descriptive information (such as tions that asked respondents to rate specific objectives, reporting structure, resources, serv- attributes of Bank knowledge products and ices, client definition, and operating princi- services. Verbatim reports from all interviews ples); (2) number, sources, and nature of provided the input for a synthesis report pre- queries; (3) areas of growing demand and pared by the external consultant to OED. expansion plans; (4) quality assurance proce- Aggregate quantitative ratings presented in the dures; and (5) challenges. synthesis report were based on unweighted Analyses of the responses were separately averages of responses. A possible source of prepared by OED and WBI and compared, 51 52 | S H A R I N G K N O W L E D G E and a report of survey findings was presented Community-Based Rural Development; Debt and discussed at a retreat of Advisory Services Markets; Disability; Early Child Development; in March 2002. E-Government; Gender and Rural Develop- The participating Advisory Services were: ment; Growth; Involuntary Settlement; Nutri- Data Help Desk, Education Advisory Service, tion; Payment Systems; Population and Repro- Energy Help Desk, ESSD Advisory Service, ductive Health; Ports, Rails, Aviation and Web Feedback, Financial Sector Knowledge Logistics; Poverty Impact Analysis, Monitor- and Information Service, OED Help Desk, ing and Evaluation; Private Delivery of Public OPCPS Advisory Service, PREM Advisory Ser- Services; Public Expenditure Management; vice, Rapid Response Unit (Private Sector Quality of Fiscal Adjustment; Rural Microfi- Advisory Service), Safeguard Policies Help nance and Small and Medium-Size Enterpris- Desk, Social Protection Advisory Service, es; Rural Private Sector, Markets, Finance and Transport Help Desk, Urban Help Desk, and Infrastructure; Rural Water and Sanitation; Water Help Desk. School Health; Secondary Education; Social Analysis and Policy; Urban Economics and Development Strategy; and Urban Environ- Thematic Group Survey ment. OED surveyed 28 thematic groups, one-third of the Bank's FY02 total, in May-June 2002 Staff User Survey (see list below). Participating groups were selected for the survey by means of a stratified OED conducted an interview survey of a sam- random sample. The sample was drawn from ple of Bank operational staff to elicit informa- strata representing each Bank sector, with the tion on how staff obtain information needed to number of thematic groups selected from a do their jobs and how lessons from experience sector reflecting its proportion of the Bank's are built into and captured from operations. total number, with no sector having less than The staff surveyed were selected from the task one group selected. teams of a random sample of 21 projects (18 The survey was conducted by means of investment and 3 adjustment) approved in structured interviews with the thematic group FY01, representing 10 percent of the total coordinators of all 28 groups, based on a com- approvals of that year. A total of 30 staff were mon interview protocol. The interviews, which surveyed, including the task manager of each lasted approximately 45 minutes, covered four of 19 projects and a second team member in main topics: (1) descriptive information (such half of all investment projects. In addition, the as date of origin, size and composition of mem- Appraisal Documents or the Memorandum bership, objectives); (2) activities and their and Recommendations of the President were effectiveness; (3) management processes for reviewed for information on the knowledge programming, budgeting, reporting, and deter- dimensions of the respective projects. mining group knowledge needs; and (4) proce- A common protocol was used for the inter- dures for monitoring and evaluation. Informa- views and document review that addresses as tion from the surveys was supplemented by four sets of questions: (1) sources of informa- materials provided by thematic group coordi- tion used in the design of the projects (Bank nators or sector managers, or available on the and external analysis and documentation, Web sites of individual groups. databases, thematic groups, advisory services, A draft synthesis of survey findings was dis- Web sites, peer reviews, informal contacts); (2) tributed to all survey participants for com- knowledge-sharing activities undertaken in the ment. course of project implementation; (3) knowl- The participating thematic groups were: edge capacity-building components of proj- Biodiversity; Child Labor; Civic Engagement; ects; and (4) views on overall improvements in Coastal and Marine Resource Management; Bank knowledge sharing, utility of different A N N E X E | 53 knowledge-sharing activities, and adequacy of Each item reviewed was rated separately as incentives for staff participation. input into an overall synthesis assessment. A draft synthesis of survey findings was dis- Draft reviews were shared with Bank staff and tributed to all survey participants for com- revised to take account of their comments. ment. Consultant Reviews of the Development Expert Reviews Gateway and the Global Development Learn- ing Network External experts commissioned by OED reviewed the quality, relevance, and innova- Independent consultants were commissioned tiveness of the Bank's shared knowledge in by OED to prepare desk reviews of the design four issue areas: power sector reform, primary and start-up of the two knowledge initiatives. and secondary education, public expenditure The reviews, conducted within limited consul- management, and rural and urban water sup- tancy periods of three weeks, were based on ply. Two criteria were used in selecting the examination of program documents, inter- issues: (1) diversity across sectors and themes views with relevant Bank staff, and selected and (2) extensive Bank involvement across a telephone interviews with partners in each wide range of both IBRD and IDA countries. program. Three of the four experts have served as con- Using standard OED evaluation criteria sultants to the Bank, but none has been a Bank (relevance, efficacy, and efficiency), each pro- employee. gram was reviewed in terms of its: (1) clarity of The materials reviewed, selected by OED in objective and the importance of the objective consultation with the relevant sector units of to achieving the Bank's overall mission; (2) the Bank, included research and analytical early evidence of extent to which it is likely to reports, country-specific economic and sector achieve its intended objectives; and (3) the work, and supporting materials for major con- cost-effectiveness of the program as an instru- ferences or workshops in each field. The ment for knowledge sharing and the sustain- experts were also asked to review the quality ability of the program, given the existing busi- and accessibility of material on related World ness model and the level of support currently Bank Web sites. In addition, they were asked being provided. to look at a sample of projects to determine Specifically, the review of the Development how adequately projects were building in cut- Gateway covered the four Gateway services ting-edge knowledge. (Knowledge/Topics, AiDA, dgMarket, and the The experts were asked to use a common Country Gateways), the integration of the pro- set of six criteria in making their evaluations. gram into Bank operations, the governance of These were (1) relevance: was the material rel- the program, and the role of the Bank. Con- evant to clients' needs? (2) comprehensiveness: sultants for this review were staff of an inde- was the material appropriately comprehensive pendent knowledge and an information servic- in its treatment of the subject matter? (3) level es consulting firm in Canada. The consultants of knowledge: did the material provide clients were previously part of a team that had pre- with the best and most up-to-date knowledge pared a report for the Bank on the manage- on the issue? (4) clarity: were the ideas and ment of its portfolio of global knowledge pro- recommendations in the materials stated clear- grams. ly? (5) objectivity: did the materials present The GDLN review, which covered the pro- information in an objective manner (indicat- gram's affiliated Distance Learning Centers, ing, where pertinent, existing differences of central GDLN Services, and integration of the viewpoint)? and (6) practicality: did the mate- program into Bank operations, was conducted rials provide information of practical useful- by a distance learning expert on the teaching ness to decisionmakers? staff of Syracuse University Training Systems 54 | S H A R I N G K N O W L E D G E Institute and staff of a technology and training Draft reviews were shared with each pro- systems institute in Brazil. The reviewer has gram's core Bank staff and revised to take worked as a consultant on Bank education account of their comments. projects. ANNEX F. EXAMPLES OF COMPARATOR KNOWLEDGE INITIATIVES Mission/ objectives Activities/ services Governance/ funding Stakeholders Key programs British Council · To connect people · Link UK schools with · Incorporated as a · Funding partner: UK · Montage worldwide with partner schools in charity in the UK, Foreign and Common- · Windows on the learning opportunities developing countries operates as an wealth Office World school- and creative ideas · Provide venue for executive nondepart- · Program partner: linking database from the UK, and to shared curriculum mental public body World Bank · Knowledge and build lasting relation- development for · Overseen by a board Learning Centers ships between the teachers of trustees Project UK and other countries · Provide Internet access, · UK Foreign and Com- · Global Library and · To leverage UK distance learning, monwealth Office Information experiences for global videoconferencing provides annual grant; Network development facilities additional funding · Development priorities: · Maintain global net- from UK government education, arts, science, work of libraries and agencies, international governance, information information resource bodies, private sector; sharing centers approx. 40% revenue (£170M) from selling services Virtual Colombo · To create opportunities · Teacher education, · Joint initiative of · Target audience: edu- · Teacher training Plan (AusAID) to improve education including 200 scholar- AusAID and the cators, students, · Content and access to knowledge ships for trainee World Bank policy-makers, development across the developing teachers · $1.5 billion partnership; researchers · Teacher scholarship world for everything · Plan to establish eight Australian government · Partners: IBRD (GDLN, · Distance Learning from primary school multipurpose IT pledged AUS$400 M-- Gateway), Australian Centers teacher training to ad- teacher training $38 M to teacher train- universities, Australian vanced courses for centers ing, $22M to improve Ministry of Foreign policymakers · Extend access to access to knowledge, Affairs (Australian · Use Australian expertise information in under- $10M for Development government sponsor) to deliver distance served regions and Gateways, $18M to · Target countries: Asia education programs contribute to policy promote policy Pacific, Africa (12 development development countries targeted) Bellanet · International initiative · Serves as a vehicle for · Budget is approximate- · Target audience: · IDML (markup working with the devel- governments and dev- ly US$1.5M donors, development language initiative) opment community to elopment agencies to · International nonprofit agencies, end · WAVE Web- increase collaboration cooperatively experi- initiative governed by beneficiaries browsing through and provide advice and ment with ICT a steering committee · Funding partners: e-mail assistance on effective initiatives representing several CIDA, Danish Interna- · Global Knowledge use of ICTs · Monitors and supports donor institutions and tional Development Partnership (GKP) · Development priori- technology innovations hosted by IRDC Assistance, IRDC, Virtual Activities ties: Capacity building, with applications for · Functions as a Swedish International · iTrain Online technology enablement development secretariat with Development Cooper- · Provides hosting and 10 full-time staff ation Agency, UNDP, online workspace Rockefeller & applications for more MacArthur than 20 organizations Foundations 55 56 | S H A R I N G K N O W L E D G E Mission/ objectives Activities/ services Governance/ funding Stakeholders Key programs Leland Initiative (USAID) · 5-year effort to extend · Work with governments · Budget is US$15M over · Target audience: · Leland subinitia- Internet connectivity to design a national 5 years African governments tives: Leland to 20+ African countries, gateway; define an e- · USAID and host benefit from readiness School-to-School building on existing readiness strategy; and country sign an MOU assessments and connectivity capacity procure, partner, and for collaboration and implementation program · Objectives: enabling implement connectivity codevelopment of a coordination · Other USAID policy environment; solutions plan of action · End beneficiaries and initiatives include sustainable supply of · Hopes to train 5,000 country telecommun- the Global Internet services; regulatory staff over ication providers Technology enhanced Internet use the next 3 years benefit from increased Network for sustainable · Leland School-to- connectivity development School Program focuses on connectivity for education Imfundo (DFID) · DFID initiative to create · Resource Bank to assist · Budget is £7 million · Target audience: DFID · Other programs and manage ICT partner- governments and staff over 5 years country staff, govern- from DFID include ships for education in to access resources and · Initiative is an integral ments and policy- "Significance of ICT Africa expertise from the part of DFID's Africa makers for Reducing · A resource for DFID's private sector Division · Three categories of Poverty"study and country offices to identi- · Knowledge Bank, · DFID created compre- partners: "Building Digital fy and broker private focused on sharing hensive ethical guide- ­Resource Bank Opportunities" sector partnerships DFID experiences with lines for partnerships partners (private sector initiative · DFID launched 13 ICT education initiatives and requires shared suppliers) programs in 2001 with and identifying and understanding and ­Liaison partners budgets over £100,000; addressing gaps in MOU (shared objectives) 39 have been launched applied research or ­Local delivery in total since 1998 analysis partners Markle Foundation · Foundation devoted · Programs: Policy for a · A private not-for-profit · Target audience: NGOs, · Global Digital entirely to the develop- Networked Society; philanthropy and development agencies, Opportunity ment of ICTs, taking a Interactive Media for grant-making organi- policymakers, donors Program leadership role in Children, Information zation, at times actively · GDOP partners: broad- · Digital Opportunity establishing and main- Technologies for manages ongoing based memberships Summit taining partnerships Better Health projects including approx. 50 · Founding partner, · Commitment to use · Sample investments: · Foundation has US$200 private and public Global Network capital funds in order Joint project with million in assets; plan sector organizations Readiness & to increase impact of Oxygen Media, PBS to invest US$100 · Grant recipients and Resource Initiative ICT investments curriculum-based million over 3-5 years their beneficiaries · Priorities: health, Internet project; · Opportunity Fund: children, policy for a Chinese Academy of Grants have ranged networked society, Social Sciences from US$1,000 to public engagement US$3.5 million UN Development · UN's principal provider · Acts as a coordinating · Governed by a 36- · Target audience: · ICT for Develop- Programme (UNDP) of development advice mechanism at the member executive governments and ment Programme and grant support global level (DOI, board, representing policymakers, end to increase · ICT for Development Dotforce) both developing and beneficiaries awareness and agenda is focused on · Partners to provide developed countries · Initiative partners: pilot telecenters stimulating the enabl- country-level assistance · Voluntarily funded Bellanet, Cisco, Markle · WIDE platform ing environment, acting in defining ICT strate- organization Foundation, IBM for south-south as a catalyst for the gies, as well as to · US$2.2 billion in total cooperation application of ICT, inno- support programs resources from OECD · UN Volunteers vating new projects to donors, multilateral UNITeS (United leverage ICT partners or program nations Information country governments Technology Service) · NetAid (see below) · SDNP A N N E X F | 57 Mission/ objectives Activities/ services Governance/ funding Stakeholders Key programs Netaid (Cisco, UNDP) · To mobilize direct · Provides transaction · NetAid Foundation · Target audience: · NetAid World support to local develop- marketplace for functions as an donors, development Schoolhouse ment projects to fight donations (individual independent nonprofit agencies, end · Launched with a extreme poverty and or corporate) to directly company, based in NY beneficiaries high-profile rock provide a connection support local develop- · Overseen by a board · Delivery partners: concert between (nontradition- ment organizations of directors Cisco, UNDP joint al) donors and delivery · Evaluates and monitors · Majority of core venture organizations project quality and operating and develop- · Also: UN Volunteers, accountability ment costs are funded UNICEF, CARE, Save · "Dollar in=dollar out" by original partners the Children, Inter- donation policy (Cisco, UNDP) national Rescue Committee Global Information · To foster private sector · Provide a vehicle for · An independent, non- · Target audience: · GIIC Africa Infrastructure leadership and private- participation for the governmental initiative private sector · GIIC Asia Commission (GIIC) public sector coopera- private sector and a involving leaders from (particularly IT), tion in the development discussion forum for a developing as well as policymakers, of information networks range of international industrialized countries donors and services to advance organizations · 11-member board of · Members: 50 CEOs global economic · Develop a research directors sets overall and presidents of growth, education agenda on information direction, including major international and quality of life infrastructure and budget for operations; corporations, policy- · Development priorities: hold consultative Managing Director is makers, and academics global electronic meetings on regional responsible for imple- from around the world commerce, GII develop- infrastructure issues menting ment, education Source: Digital 4Sight 2002. ANNEX G. CLIENT SURVEY REPORT Perspectives on World Bank Knowledge Products and Services: Findings from a Five-Country Survey Prepared by David Shirley, Consultant November 12, 2002 Background B. Methodology This report summarizes the findings from a A total of 121 individuals participated in sur- five-country client survey conducted by the vey interviews, in roughly equal numbers from Operations Evaluation Department (OED) of each of the five countries. They included repre- the World Bank in mid-2002. The survey, sentatives from government, the private sector, which was part of a broad-ranging OED eval- research and academic communities, NGOs, uation of World Bank knowledge products and media--all with significant development and services, examines clients' views on the expertise and familiarity with the World Bank. quality and usefulness of knowledge and infor- Each survey respondent participated in an mation provided by the Bank. The five partic- in-person interview conducted by individuals ipating countries were Bangladesh, Brazil, from within the respective country. Interviews Poland, Senegal, and Tanzania. were based on a common guideline prepared by OED, which included open-ended, qualita- A. Objectives tive questions and quantitative questions that The aim of the survey was to gain understand- asked respondents to rate specific attributes of ing of how diverse constituencies view the rel- Bank knowledge products and services. Verba- evance, accessibility, and applicability of tim reports from all interviews have provided development knowledge and expertise that the the input for this synthesis report. Table G.1 World Bank has helped them to acquire and presents the number of respondents by coun- use, and the impact that exchange of knowl- try, profession, and sector. edge has had on their work. The survey exam- ined the following questions: Summary of Main Findings · The current use of World Bank information The majority of respondents from all coun- and advice, and how it is obtained tries, professions, and sectors spoke favorably · Overall views on the quality and applicabil- of the World Bank information that they use in ity of information and analysis provided by their work. In spite of their generally positive the Bank assessment of Bank information overall, how- · The effectiveness of different modes of ever, respondents were critical of specific exchanging knowledge supported by the aspects of Bank information and delivery (for Bank, including not only written reports example, adaptation to local circumstances, and advisory services, but also training and dissemination, collaboration, and support for exchanges across countries institutional capacity building). Predictably, · The key factors that contribute to or inhib- respondents' overall views of and previous it effective knowledge sharing experience with the Bank played an important · The perceived impact of Bank knowledge role in their assessment and use of Bank infor- products and services. mation. 59 60 | S H A R I N G K N O W L E D G E TABLE G.1: Number of Respondents By Country By Profession By Sector Bangladesh 23 Academic 22 Econ./General 85 Brazil 27 Government 69 Education 13 Poland 25 Media 9 Power 13 Senegal 25 NGO 7 Water 10 Tanzania 21 Private Sector 11 Other 3 Main findings included: of the Bank and substantially decreases receptivity to Bank information overall. · Tension between Technical Soundness and · Poor Record in Dissemination. Virtually all Applicability to Local Circumstances. The respondents criticized the Bank's weakness majority of respondents from all countries, in disseminating information to people professions, and sectors agreed that the who need it. The majority of respondents information provided by the World Bank is blamed the Bank's poor dissemination on of unquestionably high quality--as long as its traditional practice of communicating quality is defined strictly in terms of techni- primarily or exclusively with government cal soundness and sophistication. When discussing the quality of Bank information, officials. Governments lack either the insti- however, respondents were often critical of tutional capacity to disseminate informa- the Bank's ability to provide information tion throughout the country or fail to share that is realistic in light of local circum- it with other groups that could benefit stances and responsive to local needs. Many from it. Respondents agreed that, in order insisted that technical soundness is mean- to optimize the impact of its information ingless apart from applicability to the local on a countrywide level, the Bank must context. Figure G.2 indicates the degree to abandon its "top-down," government- which respondents were more critical of the focused approach to information dissemi- realism of Bank information in light of local nation and be more proactive in its efforts circumstances than of other specific attrib- to reach previously underserved individu- utes of Bank information ("relevance," als and groups. "timeliness," and "technical soundness") · Mixed Reviews on Collaboration. Respon- and the quality of the information overall. dents' views diverged about the degree to · Frustration with the Bank's Failure to Con- which the Bank collaborates--or fails to sider Alternative Perspectives. The majority collaborate--with local expertise in their of respondents complained that the Bank is countries. A substantial number of respon- too narrowly focused in the analyses and dents criticized the Bank's poor record of "best practices" that it presents, with little collaboration, along with its condescension or no attention to alternative perspectives. toward their own knowledge and experi- Many respondents expressed frustration ence. A comparable number of respondents about the Bank's insistence that its models spoke favorably of the Bank's collaboration and solutions represent the only viable and use of local expertise within their coun- approach to solving economic and social tries. Many respondents--even some of problems in their country. Several respon- those who were highly critical of the Bank's dents complained that the Bank's insistence poor record in the past--observed that the that its approach is the only correct Bank's level of collaboration has improved approach generates mistrust and suspicion in recent years. A N N E X G | 61 · Distinction between Analytical and Institu- ple, sector-specific expert analyses, compar- tional Capacity Enhancement. Respondents ative data from the World Bank Institute, were also divided in their assessment of the and so forth). Bank's contributions to analytical and insti- tutional capacity in their countries, as well B. Varied access to Bank information as the importance of Bank efforts in these Reading formal publications (ranging from areas in the future. Many credited the Bank annual reports to sector-specific and compara- with having made substantial contributions tive analyses) was by far the most common to their personal analytical capacities, but method of gaining access to Bank information. viewed the Bank's contributions to institu- The majority of respondents also identified a tional capacity as negligible, particularly variety of other, often interrelated methods outside governmental institutions. through which they actively pursued the Bank · Overall Perception of Positive Change. The information used in their work (depending on majority of respondents, including many their availability). The following is a list of the who were openly critical of Bank informa- most commonly used sources of Bank infor- tion overall, described positive changes in mation in order of frequency of mention: the Bank's recent knowledge products and · Bank publications (studies, reports, surveys, services. Respondents named a variety of comparative analyses) factors (for example, increased transparen- · Bank Web sites cy, increased reliance on the Internet, · Direct contact with Bank staff (interactions increased collaboration with and use of with task managers, advisory consultations) local expertise, overall shift in focus from a · Attending Bank training events (work- lending to a knowledge institution) that shops, seminars, conferences, study tours) have contributed to this change. Virtually · Visiting Bank facilities (country office, all respondents, however, agreed that the library, resource center, teleconference cen- Bank still has a long way to go in meeting ter, sales outlet). their information needs. C. Growing impact of the Internet Use and Acquisition of Bank Information The majority of respondents reported that the increased availability of Bank information A. Wide variety of information used online through the Bank Web site has dra- Respondents described a wide variety of Bank matically improved the overall availability, information that they use in their work, includ- relevance, and timeliness of Bank informa- ing: tion. Overall, respondents praised the poten- · Annual reports (for example, the World tial of the Internet, both in providing a more Development Report and appendices) comprehensive body of information and in · Procedural guidelines (for example, credit enhancing their capacity to compare and inte- agreement, procurement, donor coordina- grate information from related areas and sec- tion, employment, and so forth) tors. A number of respondents worried, how- · Sector-specific reports and analyses ever, about the potentially negative impact of · Policy briefs and discussions the Bank's increasing emphasis on online · Specific Bank reports and publications (for products and services, including the potential example, "Taming the Leviathan," "Global for: Economic Prospects and the Developing · Decline in the availability and responsive- Countries") ness of Bank personnel · Information and analyses available exclu- · Decline in the quality of print publications sively through the Bank Web site (for exam- · Neglect of populations who currently lack 62 | S H A R I N G K N O W L E D G E easy access or are not responsive to new describing the unrivaled comprehensiveness digital technology. and sophistication of Bank information mate- rials and expertise. D. Timely Bank response to specific requests Respondents reported an extensive list of "The Bank is the institution which we reports and other information that they had address when we need some kind of recently requested from the Bank for use in information or advice. Don't underesti- their work. Of those respondents who had mate this fact. If you really need an made specific requests to the Bank, an over- expert on a certain issue related to whelming majority reported that they received development, the WB is where you go." the information in a timely and helpful manner. [Government respondent from Brazil] "Whenever I need a document, I give a The three figures below illustrate respon- phone call saying that I need such ele- dents' overall assessment of Bank information ment, and generally I receive it the same according to country, professional, and sector day." [Government respondent from affiliation. Senegal] Assessment of Quality of Bank E. Additional information needs Information Respondents identified several areas in which The majority of respondents from all coun- they would like to receive additional informa- tries, professions, and sectors praised the qual- tion from the Bank, including: ity of Bank information, expressed in terms of · Additional reports and analyses on sector- "relevance," "timeliness," and "technical specific information soundness." · Assistance in tracking changes (for exam- ple, trends, evaluations of programs and "In terms of quality, the reports are strategies, and so forth) over time always 100 percent--well founded, · Ability to correlate information on related detailed, focused, always concerned topics (for example, women and poverty). with consistency, always the best quali- ty." [Government representative from Other respondents, however, complained Brazil] about the overabundance of information that "The technical soundness of World they receive from the Bank and other Bank information is unquestionable." sources--and the lack of adequate guidelines, [Journalist from Tanzania] summaries, or search tools with which to quickly identify the information that they need to do their work. A. Technical soundness commended As shown in figure G.4, aggregate ratings for "I just can't digest everything that the attributes of relevance, timeliness, and comes into my office. Maybe five technical soundness were substantially higher percent." [Government respondent than the overall rating, with technical sound- from Brazil] ness receiving the highest rating of all (4.82, compared to 4.81 for relevance, 4.6 for timeli- ness, and 4.34 for the overall assessment). Overall Assessment Respondents repeatedly used the following The majority of respondents praised the qual- interrelated factors to describe the quality of ity of Bank information overall, with many World Bank information: A N N E X G | 63 FIGURE G.1: Overall, the Bank Effectively Meets Client Knowledge Needs (by Country) Strongly agree 6 5 4.63 4.59 4.32 4.13 4.00 4 3 2 1 Strongly disagree 0 Senegal Brazil Tanzania Poland Bangladesh FIGURE G.2: Overall, the Bank Effectively Meets Client Knowledge Needs (by Profession) Strongly agree 6 5 4.52 4.30 4.00 4 3.64 3.50 3 2 1 Strongly disagree 0 Government Academic Media Private Sector NGO FIGURE G.3: Overall, the Bank Effectively Meets Client Knowledge Needs (by Sector) Strongly agree 6 5.00 5 4.28 4.25 4.08 4 3 2 1 Strongly disagree 0 Power Economy/General Water Education 64 | S H A R I N G K N O W L E D G E · Comprehensiveness become guidelines. They could bring a · Detail very positive effect, but also a negative · Expertise (including the knowledge and one. If I'm in Brazil, living with Brazil- expertise displayed by Bank team leaders ian reality, implementing projects, and and staff in their direct interactions with I observe that there's not a correct respondents) vision of the situation, then I begin to · Consistency and reliability question this best practice. Is it really · Theoretical soundness true that this "best practice" is the · Analytical strength. exact, correct, and real vision?" [Government respondent from Brazil] B. Inadequate attention to local circumstances In contrast, respondents criticized the quality C. Different views among respondent groups of Bank information when it was expressed in The different respondent groups were remark- terms of the degree to which Bank information ably consistent in their assessments of each is "realistic in the light of local circum- attribute of the Bank's information, with the stances." Respondents rated the "realism" of following exceptions: Bank information in relation to local circum- · Respondents from Bangladesh were less stances substantially lower than the effective- likely to rate Bank information as "techni- ness of Bank information overall (figure G.4). cally sound" than were other respondents (4.18 compared to 4.82 overall), "It's one thing for the presentation to · Respondents from Bangladesh and NGO be theoretically sound, but it cannot be respondents were less likely to regard Bank considered "best practice" until it is information as "realistic in the light of local tested for relevance, implementability, circumstances" than were other respon- and benefit to people on the ground." dents (2.82 and 2.17 respectively, compared [Government respondent from to 3.63 overall), Bangladesh] · Private sector respondents were much "What happens is that these "best more critical of the "timeliness" with practices" come to us as norms; they which Bank information is provided than FIGURE G.4: The Bank providesinformation thatis: Strongly agree 6 4.82 5 4.81 4.60 4.34 4 3.63 3 2 1 Strongly disagree 0 Focusing Strengthening Providing Incorporating Disseminating information capacity links local information knowledge A N N E X G | 65 were other respondents (3.40 compared to respondents, is to make the best case possible 4.60 overall). for models and solutions that conform to its promarket approach to economic and social development. It is the responsibility of end Failure to Consider Alternative Perspectives users to adapt the information and models The majority of respondents described the provided by the Bank to local circumstances Bank as narrowly focused in the analyses and and local needs. "best practices" that it presents, with little or no attention to alternative perspectives. "[The Bank's solutions] are not always Respondents tended to evaluate the Bank's appropriate, but they always make you approach differently, depending on their coun- think. Even if a given idea wouldn't try affiliation. work in Polish circumstances, it makes you think; it's a kind of fertilizer." A. Majority/minority views [Government respondent from The majority of respondents were critical of Poland] the Bank's unwillingness to consider alterna- "I think [adapting the information to tive perspectives in providing models and solu- local needs] is our task. If you want to tions. For many respondents, the Bank's nar- have a good adaptation to fit specific row focus reveals its bias in favor of Brazilian conditions, this is something macroeconomic solutions and general inflexi- that we have to do." [Government bility in considering local circumstances--as respondent from Brazil] well as its condescension toward the experi- ence and expertise of country institutions and personnel. By failing to present alternative per- B. Presumption of correctness spectives, they believe the Bank precludes Still, both respondent groups criticized the access to potentially useful models and solu- Bank's insistence that the models and solutions tions that fail to conform to its overall devel- it provides represent the only viable approach opment objectives. to solving economic and social problems in their countries. For a majority of respondents, "My general impression is that the this insistence that the Bank's way is the only Bank presents information and perspec- way underlies most reports, strategic models, tives that support its own policy agen- and policy analyses. Many respondents from da." [Government respondent from Poland and Brazil, however, described the Bangladesh] Bank's singular approach as a typical and "The Bank often ignores potentially understandable characteristic of international useful solutions from other parts of the lending and knowledge institutions--and one world which were not sponsored or that does not ultimately undermine the infor- supported by the Bank (for example, mation's usefulness. rural electrification in Morocco)." [Journalist from Senegal] C. Varying degree of flexibility by topic Many respondents reported that the Bank's For a number of respondents in Brazil and flexibility in considering alternative approach- Poland, however, the Bank's singular orienta- es tends to vary according to topic. For tion to models and solutions represents a con- instance, the Bank was described as flexible in structive approach to sharing information-- its treatment of international trade and global- and one that is consistent with the Bank's ization, but utterly inflexible in its treatment of legitimate objectives in their countries. The resettlement, the environment, and poverty in legitimate role of the Bank, argue these developing countries. 66 | S H A R I N G K N O W L E D G E D. Distinction between Bank publications quality of Bank information. Overall, respon- and personnel dents provided similar assessments of the In discussing the Bank's willingness to consid- World Bank's performance in delivering er alternative perspectives, a substantial num- knowledge and information to users in their ber of respondents distinguished between the countries, along with the importance of information provided through Bank publica- increased effectiveness in the future. tions (both in print and digital) and the infor- As figure G.5 illustrates, respondents were mation presented directly by Bank personnel. most critical of the Bank's performance in the Respondents were inconsistent, however, in areas of dissemination, incorporation of local their evaluation of these respective resources. knowledge and collaboration, and providing Some respondents described Bank personnel links to counterparts in other countries. as more open and less doctrinaire than official Respondents spoke more positively of the Bank publications, while others complained Bank's performance in focusing knowledge about imperious pronouncements of staff where it is most needed and strengthening ana- members who were out of touch with the less lytical capacity. Still, respondents stressed the rigid and dogmatic approach of the Bank's need for improvements in all of these aspects recently published materials. There was no of Bank knowledge sharing. discernible correlation, however, between Consistent with their overall assessment of respondents' perceptions in this area and Bank information, respondents from Bangla- country, professional, or sector affiliation. desh and NGO respondents were consistently more critical of the Bank's performance in each area. Effectiveness of Bank Knowledge Sharing · NGO respondents were most critical of the Generally speaking, respondents were more Bank's weaknesses in providing "links to critical of the Bank's performance in delivering information from other countries" (3.17, information than they were with the general compared to 4.01 overall). FIGURE G.5: The Bank does an effective job of: Strongly agree 6 5 4.44 4.24 4.01 3.96 3.89 4 3 2 1 Strongly disagree 0 Focusing Strengthening Providing Incorporating Disseminating information capacity links local information knowledge A N N E X G | 67 · Respondents from Bangladesh were most The majority of respondents blamed the critical of the Bank's efforts to "incorporate Bank's poor record in disseminating informa- local knowledge" (3.13, compared to 3.96 tion on the institution's traditional practice of overall). communicating primarily or exclusively with government officials. In far too many cases, The following sections provide detailed they said governments lack the institutional views on four of these dimensions of Bank capacity to disseminate information through- knowledge sharing: dissemination, collabora- out the country or in some cases deliberately tion, providing links, and capacity enhance- withhold it from those who could benefit ment. from it. A. Poor record on dissemination "In order to have a transfer of knowl- Virtually all respondents were critical of the edge between more capable and less Bank's weaknesses in disseminating the infor- capable, it's necessary to have interac- mation that it provides to people throughout tion. It's the opposite of water ­ if you the countries who need it most. have a big tank and a small tank, then you have a difference of pressure and "No, it doesn't [do enough to dissemi- water flows. But in order to have nate its information]. That's for sure." knowledge flow, the difference can't be [Academic respondent from Poland] too high, too big." [Government "The Bank communicates very poorly respondent from Brazil] with the public; in fact it doesn't com- "Information should be made accessible municate with the public." [Journalist at ALL levels. It is currently accessible from Brazil] at only the highest levels." [Government respondent from Tanzania] Figure G.6 illustrates respondents' assess- ments of the Bank's dissemination in their In addition to the Bank's excessive reliance countries. on the government as an information conduit, FIGURE G.6: The Bank does an effective job of disseminatinginformation: Strongly agree 6 5 4.15 4.11 3.95 3.92 3.89 4 3.23 3 2 1 Strongly disagree 0 Poland Senegal Tanzania Brazil Bangladesh Overall 68 | S H A R I N G K N O W L E D G E respondents repeatedly identified a litany of "Few people realize that the Bank has a factors that currently inhibit the Bank's infor- wealth of information [because] few mation dissemination, including: people have access to the Internet with which to access the Bank's Web site." · Excessive reliance on the Internet in coun- [Academic respondent from Tanzania] tries and regions where the majority of peo- ple still lack easy access to the Internet Poor dissemination to the general public is · Lack of transparency and open discussions seen by many respondents as the root of pop- with stakeholders ular misperceptions and mistrust of the Bank · Lack of proactive educational initiatives and its objectives and programs. (for example, workshops, seminars, and public forums) with the general public "Most people associate the Bank with · Lack of adequate translations privatization and with the poor results · Overcentralization and the lack of distribu- achieved by privatization practices in tion and sales venues on the local level Brazil. That makes things difficult. So · Lack of adequate mailing and distribution the Bank should work to change that lists view, and the way to do this is through · Lack of comprehensive listings and year- more communication." [Government end summaries of information provided by respondent in Brazil] the Bank · Poor dissemination of Bank information to Respondents identified several ways in libraries and universities which the Bank might enhance its ability to · Lack of collaboration with the national and provide information throughout their coun- local media tries, including: · Absence of proactivity and motivation on the receiving end · Abandon the current "top-down," govern- · Excessive costs (including any cost at all in ment-focused approach to dissemination less developed countries such as Senegal (assuming that government officials will and Bangladesh). forward information to individuals and groups throughout the country). "If you are concerned with reducing · Work with government, academic, media, infant mortality and [increasing] water NGO and private sector representatives to supply/sanitation, but you don't dissem- identify exhaustive lists (for example, inate these works [on the topics] in postal, e-mail, and institutional) of the peo- Portuguese, you will not reach a broad ple who could benefit from Bank informa- audience." [Government respondent tion. from Brazil] · Publish and distribute periodic (annual, biyearly, quarterly) listings and summaries "First of all, it should prepare a sound of newly available Bank information. mailing list of people and institutions · Provide more translations of Bank informa- and send them relevant information reg- tion. ularly. This is such a simple thing but · Work collaboratively with the media to they don't do this." [Academic respon- translate important information into lan- dent from Poland] guage that is accessible to the general pub- "But this is sometimes OUR problem; lic. we do not always exploit the informa- · Provide information to schools, universi- tion that we do receive." [Government ties, and libraries, with links to curricula respondent from Senegal] where possible. A N N E X G | 69 · Actively support the development of Plan Studies, I had even asked the Bank regional information relays throughout the to propose [an outside] consulting firm, country (for example, local documentation but it refused, insisting it was my and resource centers). responsibility. They were completely · Provide clearer links to or suggestions for open to the fact that a local consulting applicability at the national or local level. firm was the successful bidder." [NGO leader from Senegal] B. Mixed views on collaboration Respondents often differed in their evaluation Conversely, respondents who feel as if they of the degree to which the Bank collaborates-- have been snubbed or ignored in the past are or fails to collaborate--with local expertise in more likely to speak critically of the Bank. their countries. A substantial number of respondents--rep- "We know this country, we know the resenting all countries, professions, and inter- enterprises, we have a contribution [to est sectors--were highly critical of the Bank's make] for the economy, and for this poor record of collaboration and resentful of kind of relationship. Not to consider its implications about their own knowledge [our contribution] as at least a first and experience. approximation, I think is a loss." [Government respondent from "Some Bank experts are biased, based Brazil] on their experiences of what has worked elsewhere in the world, with Many respondents indicated that Bank col- little attention to local needs or ideas." laboration needs to expand beyond the hiring [Government respondent from of local staff and consultants. Generally speak- Tanzania] ing, respondents were more likely to speak favorably--and provide positive anecdotes-- However, a comparable number of respon- of the Bank's hiring practices on the local level dents--again representing the full range of (both in its use of consultants and its staffing countries, professions, and interest sectors of country offices) than of the institution's included in the study--spoke favorably of the willingness to collaborate on other levels. Bank's collaboration and use of local expertise Many respondents complained about the within their countries. Bank's reluctance to listen and respond to local expertise on the policy, project implementa- tion, and educational levels. Additionally, "Yes, it does, and I think that it's quite respondents suggested that the effectiveness, good. The World Bank applies or bases accessibility, and credibility of Bank informa- itself on the knowledge of our experts." tion and training could be strengthened [Government respondent from Poland] through active collaborations with national and local institutions, including: These contrasting views of Bank collabora- tion seem to be substantially informed by the · Academic institutions respondents' direct, first-hand experience, or · NGOs lack of experience, with the Bank. Respon- · Libraries and learning centers. dents who have had positive first-hand experi- ences with the Bank tend to speak favorably of "Its interaction with local experts the Bank's practices in their country. through dialogues on various policy prescriptions is almost nil." "When I was making the Invitation to [Government respondent from Tender for the Local Electrification Bangladesh] 70 | S H A R I N G K N O W L E D G E "Yes, the Bank does a good job of "For instance, Chilean insurance recruitment in the country office and in solutions have been recommended to consulting assignments, but to what Poland. Now Polish solutions are being extent local perspectives are incorporat- recommended to other countries. Thus, ed in development programs, it is diffi- they encourage the flow of good ideas, cult to tell." [Government respondent like for example the way of assessing from Poland] inflation in Brazil or Chile. The Bank recommends solutions to different Respondents who criticized the Bank's poor countries and if their governments record of collaboration were quick to name a accept those solutions, then they can variety of international institutions whose cur- only benefit." [Government respondent rent level of collaboration exceeds that of the from Poland] Bank, such as: In the quantitative evaluations, however, · United Nations Development Programme respondents provided comparatively low rat- · The European Union ings (4.01, compared to 4.34 for Bank informa- · Asian Development Bank tion overall) for the Bank's ability to provide · Bilateral aid agencies (for example, France, links to other countries. This apparent contra- United States). diction may be linked to respondents' criticisms Many respondents--even some of those of the Bank's failure to present alternative who were highly critical of the Bank's poor approaches in presenting examples and solu- record in the past--indicated that the Bank tions (described above). For many respondents, has begun to improve its efforts to collaborate the usefulness of the Bank's expansive reper- in their countries. All agreed, however, that toire of comparative examples and expertise is there is still plenty of room to do more. Some compromised by the Bank's tendency to present respondents viewed the Bank's willingness to "best practices" and comparatives as the only collaborate skeptically, however--as a result valid way of confronting a problem. of bowing to pressure and competition at the local level, rather than internal policy changes "[The Bank's comparative recommen- on the institutional level. dations] should come to us as advice, rather than as compulsion." C. Providing links to other countries [Government respondent from Poland] Respondents as a group had inconsistent assessments of the Bank's ability to provide Respondents provided the following sug- useful links with other countries. During the gestions for increasing the usefulness of the qualitative portion of the interview, the international linkages and comparative infor- majority of respondents described the Bank's mation provided by the Bank: unrivaled position as a source of comparative · A less dogmatic style of presenting compar- information and expertise, and its ability to ative examples and solutions (including provide useful examples and solutions from open forums for discussing and criticizing other countries. Several respondents from Bank solutions) Poland and Brazil also described the Bank's · Increased opportunities for travel to and role in providing examples and expertise from exchange with other countries (through their own country to users in other countries. which local representatives can make their own assessments and comparisons) "No other organization provides so · A more proactive role for local users in much comparative information from so adapting and testing the validity of com- many different settings." [Government parative recommendations provided by the respondent from Bangladesh] Bank. A N N E X G | 71 D. Analytical vs. institutional capacity provided to representatives of other Respondents also disagreed in their assess- institutions, not just the state." ments of the World Bank's contributions to [NGO leader from Senegal] analytical and institutional capacity in their "They should also provide training to countries, as well as the importance of Bank teachers and public servants--not just efforts in these areas in the future. economists and managers-- using Many respondents drew a sharp distinction television, the Internet, and other between Bank support for analytical and insti- media." [Government respondent from tutional capacity enhancement. Generally Brazil] speaking, respondents were much more likely to applaud the Bank's contributions to local However, many respondents warned that analytical capacity and knowledge than the even the most aggressive initiatives by the Bank's efforts to strengthen actual institutional Bank may fall short of the existing need, given capacity. the absence of local staff and infrastructure to benefit from the Bank's contributions. "Yes, on the level of analytical insights and techniques; no, on the institutional "Yes [they do what they can], but too level, where the Bank doesn't go often there is a dearth of local capacity beyond project frameworks." to make use of the Bank's contribu- [Government respondent from tion." [Government respondent from Bangladesh] Bangladesh] "The World Bank provides excellent training [on the analytical level]. The Respondents consistently identified train- workshops (on rural sanitation) allowed ing as a critical area for strengthening analyt- us to bring together small, very well ical and institutional capacity. Specific inno- trained groups to advance on conceptu- vations in training suggested by respondents al matters, on strategies for carrying out included: certain actions. The reports are able to · More use of e-learning for training and systematize a set of important reflec- retraining tions." [Government respondent from · Short-term exchange programs with the Brazil] Bank · Increased opportunities for attending work- A substantial number of respondents com- shops and seminars sponsored outside the plained about the Bank's heavy concentration country on the capacity enhancement of government · Increased training opportunities for NGO officials and institutions. Many who spoke leaders, journalists, academics, and other positively of the Bank's current efforts in nongovernmental personnel (including capacity enhancement cautioned that the Bank train-the-trainer workshops) continues to place too much emphasis on the · Collaborating with other institutions to training and support of government officials sponsor public awareness-building seminars and institutions. More emphasis should be (to encourage reliance on the private sector, placed on training and advisory support to instead of the state, in achieving social and strengthen capacity on the local level and economic goals) across a wider range of institutions. · Shifting the focus of training from main- stream economics to areas of more critical "Yes, progress is definitely being made concern locally (for example, the environ- through the work of the Distance ment, nonprofit economy, and institutional Learning Center, but training should be economics) 72 | S H A R I N G K N O W L E D G E · Specialized training events (in databases, · Assistance in developing indicators for eval- policy formulation, elaboration and plan- uation of the performance of the national ning, and so forth) information system (through direct dia- · Longer training periods (at least every two- logue with Bank staff) three months, in addition to the three-four · Development of simulation models (for events now used). example, for anticipating the impact of shock on the terms of trade) · Assistance in the development of bench- Impact of World Bank Information marks for evaluating projects and institu- The majority of respondents acknowledged an tional progress. ongoing impact of World Bank information in their work. Many struggled to identify the spe- Respondents also described other areas in cific area (or areas) in which Bank information which Bank information has had an impact on had been most useful, however, suggesting that their work, including: their interaction with the Bank over time had · Specific operational procedures and know- influenced their work in ways that were subtle how (for example, completing surveys and and difficult to describe. designing terms of reference) The most commonly cited impact by far · Changing views on both the policy and was that of increased individual capacity. operational levels (for example, toward Overall, respondents were most likely to road management and financing, institu- acknowledge the ways in which Bank infor- tional setup, contracting methods, and mation had enhanced their individual analyti- project financing) cal skills and capacity, and the increased con- · Enhancement of institutional capacity (for fidence and effectiveness that Bank data and example, organizing and systematizing analytical models bring to their work. information systems, computerizing libraries and information facilities, develop- "It's definitely had an impact on my ing Web sites, and so forth) work, especially in the area of analysis, · Broad innovations in policy and infrastruc- but it's extremely difficult to measure ture development (for example, decentraliz- [how and in response to what World ing and modernizing urban transportation Bank input] I've changed." [Academic in Brazil and achieving universal schooling respondent from Tanzania] in underpopulated areas of Senegal). "The quality of statistical information provided by the Bank allows me to do "Revamping its information/documen- my work much more efficiently than in tation into a systematic documentary the past." [Government respondent network [has been a tremendous from Senegal] improvement]." [Academic respondent from Senegal] Respondents provided the following exam- "The World Bank was the pioneer on ples of ways in which Bank information has the idea that it's not enough to build increased their individual analytical skills and new infrastructure; it's necessary to capacity: have the social and institutional · Help in identifying specific analytical apparatus to operate this infrastructure resources (for example, using Bank reports in a sustainable way. This led to new to identify rate of returns on investments) laws, new institutions, a new way to · Assistance in defining and identifying spe- deal with water supply and water cific populations (for example, the poor, quality problems." underserved rural populations, and so [Government respondent from forth) Brazil] A N N E X G | 73 Key Emerging Issues local circumstances, whereas for others, adap- tation is a job for countries themselves. Three overarching issues emerge from these findings as needing the greatest attention. 2. Expanding dissemination and dialogue 1. Adapting global knowledge to local Dissemination was by far the most commonly circumstances mentioned area in which the Bank must improve its knowledge services. Respondents As described above, the findings reveal a identified numerous ways in which the Bank widespread ambivalence toward World Bank might enhance its ability to provide informa- information--with respondents struggling to tion and analyses throughout their countries, balance their appreciation of the technical including: soundness of Bank information with their frustrations regarding the Bank's record in · Abandon the current top-down, govern- adapting its knowledge to local circum- ment-focused approach to dissemination, stances. which assumes that government officials Respondents repeatedly identified a wide will disseminate information to individuals variety of factors that inhibit the usefulness and groups throughout the country. and applicability of the Bank's information · Work with government, academic, media, and analysis. NGO, and private sector representatives to identify wider lists of people and · Overgeneralization groups who could benefit from Bank infor- · Exclusive focus on macro--to the exclusion mation. of micro--economic issues · Publish and distribute periodic (quarterly, · Single-mindedness and selectivity in pre- annual, biannual) listings and summaries of senting models and strategies newly available country- and sector-rele- · Lack of understanding and adaptation to vant knowledge products. the needs and concerns of local circum- · Provide more local language translations of stances Bank information and analyses. · Emphasis on extremes (in the presentation · Work collaboratively with the media to and evaluation of solutions). translate important information into lan- guage accessible to the general public. "[The Bank's presentation] is too · Provide information to schools, universities, generalized to be useful. Like other and libraries, with links to curricula where international bodies, the Bank uses possible. standardized, stereotyped methods to · Actively support the development of region- solve specific problems." al information relays within countries (for [NGO leader from Senegal] example, local documentation and resource "[The Bank places too much emphasis centers). on showing] something that has gone Respondents also suggested that receptivity badly wrong, or something which was to and use of Bank information in their coun- very successful. For one country to tries would be dramatically enhanced by more learn from another, there has to be a opportunities for open, critical discussions of reflection in the middle." [Government Bank information and the format and context respondent from Brazil] in which it is provided. This could be achieved both through public seminars and discussion For some respondents, the Bank needs to do groups sponsored by the Bank itself and a better job of adapting global knowledge to through the external monitoring and evalua- 74 | S H A R I N G K N O W L E D G E tion of Bank information and activities on the ical of the ability of Bank information to national level. deliver specific benefits (technical sound- ness, relevance, realism in light of local cir- "[The Bank should identify and] pay cumstances, timeliness). representatives from nongovernmental · Limited Access to and Use of the Internet. organizations to conduct regular moni- The availability of the information provid- toring and criticism of the Bank. ed through the Bank Web site has had little The other day in the workshop with effect in Bangladesh. The most common the Bank I mentioned that, in the way of accessing Bank information in Netherlands, they pay people all over Bangladesh continues to be through hard the world to criticize them. This would copies. The present generation of senior be a good experience for the World officials and researchers exhibit limited Bank, to add some resources so that exposure and receptivity to the use of elec- institutions like Rede Brazil can tronic media in their work. continue to monitor and criticize the · Negligible Impact of Bank Information on Bank, recognizing that the most Respondents' Work. Consistent with the important information is about what is negative attitudes expressed toward Bank wrong." [NGO leader from Brazil] information overall, respondents from Bangladesh reported the lowest level of 3. Capacity enhancement impact of any of the country respondent groups. The majority of respondents Many respondents urged the Bank to place reported either that information from the more emphasis on enhancing the capacity of Bank has had no discernible impact on their clients to acquire and use knowledge and work or that they were unable to identify information--especially in the area of institu- the precise ways in which Bank information tional capacity. has helped them. Specific suggestions included: more and · Specific Limitations to Country Office better on-the-job training; linkages with coun- Facilities. A number of respondents were terparts from other countries; and more active critical of the limited resources available at institutional collaboration in the generation the Bank's country office in Bangladesh. and sharing of knowledge. Many respon- Though the office's library is accessible dents, however, did not expect the Bank to during weekdays to anyone wishing to use play as important a role in this area as other it, the resource is poorly advertised and international organizations already more few people outside the government are invested in support for capacity building. aware of its availability. The library pro- Most agreed that considerable dialogue is vides a broad variety of Bank and non- needed between the Bank and end users in Bank resources--all in English. There is their countries before real progress can be only one computer terminal with online made in this area. links in the entire facility. Country-Specific Findings B. Brazil: Key Findings · Comparatively Favorable Assessment of A. Bangladesh: Key Findings Bank Information. Respondents from · Highly Critical of Bank Information. Brazil provided comparatively favorable Respondents from Bangladesh were more evaluations of both the Bank's overall mis- critical of Bank information than were sion in their country and the information other country respondents included in the that it provides for use in their work (a 4.59 study (a 4.00 rating, compared to a 4.34 rating for the effectiveness of Bank infor- rating by respondents overall). Respon- mation, compared to a 4.34 rating for dents from Bangladesh were also more crit- respondents overall). A N N E X G | 75 · Less Need for Improvement in Bank Infor- · Importance of a Region-Specific Approach. mation Delivery. Respondents from Brazil The lack of adequate regional studies and were less likely than were respondents from analyses emerged as a key theme among other countries to indicate the need for Brazilian respondents, given the size and improved effectiveness in all areas of the complexity of the country, in comparison to Bank's information delivery. For instance, other countries served by the Bank. As one Brazilian respondents provided a 4.33 need respondent explained: "With all respect for rating for "strengthening institutional and Costa Rica, you can't treat Brazil like Costa analytical capacity," compared to 5.16 for Rica. You have to be much more regional- respondents overall. ized. If poverty is concentrated in the north- Bangladesh: Assessments of Bank Information and Delivery FIGURE G.7: The Bank providesinformation thatis: Strongly agree 6 5 4.57 4.45 4.18 4.00 4 2.82 3 2 1 Strongly disagree 0 Relevant Timely Technically Effective Realistic sound overall FIGURE G.8: The Bank does an effective job of: Strongly agree 6 5 4.00 4 3.50 3.48 3.33 3.13 3 2 1 Strongly disagree 0 Focusing Links to Enhancing Disseminating Incorporating on needed other capacity local areas countries knowledge 76 | S H A R I N G K N O W L E D G E east of Brazil, you need to have studies of of communications on a regional basis in private economic sectors, state by state, with favor of a centralized, government-focused this information." approach to information sharing; and the · Specific Problems with Bank Dissemina- conspicuous absence of adequate transla- tion. While Brazilian respondents also tions of documents and reports (both expressed less concern than did other online and in print) into Portuguese and respondents about Bank dissemination Spanish. (4.44, compared to 5.10 overall), they were nevertheless consistently critical of the C. Poland: Key Findings Bank's poor dissemination policies in sev- · Critical of Bank Information Overall. eral specific areas, particularly: the absence Respondents from Poland were more criti- Brazil: Assessments of Bank Information and Delivery FIGURE G.9: The Bank providesinformation thatis: Strongly agree 6 5.19 5.11 5 4.59 4.44 3.89 4 3 2 1 Strongly disagree 0 Technically Relevant Effective Timely Realistic sound overall FIGURE G.10: The Bank does an effective job of: Strongly agree 6 5 4.64 4.38 4.31 3.92 4 3.85 3 2 1 Strongly disagree 0 Enhancing Focusing Links to Disseminating Incorporating capacity on needed other local areas countries knowledge A N N E X G | 77 cal of Bank information overall than were Bank objectives and Bank information in other respondents (a 4.13 effectiveness rat- their country. Senegalese respondents pro- ing for Bank information, compared to vided the highest general rating of the effec- 4.34 for respondents overall). tiveness of Bank information (4.63, com- · Comparatively Low Assessment of Bank pared to 4.34 overall), and also provided Information Delivery. Consistent with their moderate or high ratings to each of the ben- critical views of Bank information overall, efits and attributes of Bank information Polish respondents also provided compara- examined by the research. tively low need ratings in the areas of "dis- · Comparatively High Need for Improved semination" (4.67, compared to 5.10 over- Bank Effectiveness. Senegalese respondents all), "incorporating local knowledge" (4.52, also provided the highest need ratings for compared to 5.10 overall), and "providing improving the effectiveness of Bank infor- links to information from other countries" mation in the future in relation to every (4.15, compared to 4.94 overall). attribute included in the study, with a par- · Perceptions of Improvement. In spite of ticular emphasis on "incorporating local their comparatively low rating of Bank knowledge" and "strengthening capacity" information overall, the majority of Polish (5.96 and 5.92, respectively on a 6.0 scale). respondents reported positive change in the · Perceptions of Improvement. Respondents ways that they seek and receive information from Senegal were more likely than were from the Bank. Most respondents were other respondents to speak of positive highly appreciative of the Bank Web site change in the way the Bank provides infor- and the increased role of the Internet as a mation in their country--and to provide way to access information about the Bank's specific illustrations of improvements made activities, both in Poland and in other coun- by the Bank. These included: increased tries. A number of Polish respondents also availability of the Bank Web site; increased described the increased responsiveness of training (for example, restitution and vali- the Bank mission in Warsaw in providing dation seminars); videotape conferences information. and teleconferencing; upgrades at the · Self-Criticism. Respondents from Poland Bank's Documentation Center; and the were much more likely than were other Bank's ongoing collaboration with the Dis- country respondents to speak critically-- tance Learning Center. and pessimistically--of their own country's · Self-Criticism. Along with Polish respon- role in the perceived failure to access Bank dents, Senegalese respondents were much information, contribute local knowledge to more likely than were other country respon- Bank reports and analyses, and work col- dents to speak critically of their own failure laboratively on Bank-sponsored projects. As to identify and use available Bank informa- one respondent explained: "There is this tion and expertise in an effective manner, English saying that you can take a horse to including the failure of government offices the water but you can't make it drink. This to share information with one another. A is a huge problem in Poland. The World number of respondents described the need Bank should make more effort to get closer to increase proactivity as a two-way to various communities in Poland. But if street--a responsibility that must be accept- those communities are not willing to ed both by the Bank (in its strategies for dis- absorb, it is not going to work." semination and collaboration) and end- users in Senegal (in taking the initiative to D. Senegal: Key Findings identify and exploit existing resources). · Comparatively High Rating of Bank Infor- E. Tanzania: Key Findings mation Overall. Respondents from Senegal were somewhat more likely than were other · Moderate Assessment of Bank Information country respondents to speak favorably of and Benefits. Respondents from Tanzania 78 | S H A R I N G K N O W L E D G E Poland: Assessments of Bank Information and Delivery FIGURE G.11: The Bank providesinformation thatis: Strongly agree 6 4.98 5 4.72 4.58 4.13 3.93 4 3 2 1 Strongly disagree 0 Technically Relevant Timely Effective Realistic sound overall FIGURE G.12: The Bank does an effective job of: Strongly agree 6 5 4.65 4.44 4.33 4.15 4 3.82 3 2 1 Strongly disagree 0 Focusing Incorporating Enhancing Disseminating Links to on needed local capacity other areas knowledge countries represented the most moderate group in ment in the availability and timeliness of their attitudes toward the information pro- information from the Bank. Outside of vided by the Bank. Tanzanian respondents the government, however--and even provided ratings that were virtually identi- among several government respondents-- cal to the average ratings overall--in their the impact of the Internet was perceived as evaluations of both the general effectiveness limited almost exclusively to the govern- of Bank information and the specific infor- mental level. Some feared that the Bank's mation benefits examined by the study. increasing reliance on the Internet might · Divided Assessment of the Internet. For actually be reducing the amount of infor- many government respondents, the Bank mation available to those without Internet Web site provides a significant improve- access. A N N E X G | 79 Senegal: Assessments of Bank Information and Delivery FIGURE G.13: The Bank providesinformation thatis: Strongly agree 6 5.00 4.83 5 4.78 4.63 4 3.77 3 2 1 Strongly disagree 0 Timely Technically Relevant Effective Realistic sound overall FIGURE G.14: The Bank does an effective job of: Strongly agree 6 4.94 5 4.61 4.50 4.11 4.05 4 3 2 1 Strongly disagree 0 Focusing Incorporating Enhancing Disseminating Links to on needed local capacity other areas knowledge countries · Divided in Perceptions of Change. Many the Bank's control on developmental government respondents praised the policy. Bank's increased responsiveness, the avail- · Comparative Benefits of Bank Informa- ability of more and better information tion. Respondents from Tanzania were through the Internet and a more collabo- more likely than were other respondents rative, knowledge-oriented approach from to criticize shortcomings of Bank informa- the Bank overall. Other respondents tion in relation to the comparative benefits viewed these changes with suspicion-- provided by other international institu- arguing either that the recent improve- tions. Organizations whose information ments in Bank information were cosmetic services compared favorably to the Bank's and nonsubstantive or aimed at increasing included: UNDP, KFW, and EU (more 80 | S H A R I N G K N O W L E D G E effective collaboration at the local level); and the UN (more open to alternate CIDA (more effective capacity building); approaches). Tanzania: Assessments of Bank Information and Delivery FIGURE G.15: The Bank providesinformation thatis: Strongly agree 6 5 4.84 4.84 4.56 4.32 4 3.65 3 2 1 Strongly disagree 0 Technically Relevant Timely Effective Realistic sound overall FIGURE G.16: The Bank does an effective job of: Strongly agree 6 5 4.35 4.30 4.20 3.95 3.90 4 3 2 1 Strongly disagree 0 Links to Focusing Enhancing Disseminating Incorporating other on needed capacity local countries areas knowledge ANNEX H. MANAGEMENT RESPONSE TO THE OED REVIEW SHARING KNOWLEDGE: INNOVATIONS AND REMAINING CHALLENGES I. INTRODUCTION from five client countries. In the design stage, OED decided not to look at the activities of World Bank Management welcomes the Oper- the World Bank Institute (WBI). While that ations Evaluation Department (OED) review decision was probably necessary to narrow the of the knowledge initiative, and it commends scope of the work, it means that the review OED for completing an ambitious and diffi- does not examine the critical work of WBI in cult analysis. As OED notes, the Bank has supporting the strengthening of knowledge made enormous progress since 1996 and is transfer systems in borrowing countries, an now recognized as a leading knowledge man- area where WBI has taken and will continue to agement institution. It really has no peers take a lead role. among public sector institutions supporting developing countries. Management also recog- nizes that the knowledge initiative involves a Conclusions. The review concludes that the large and complex array of specific activities Bank's efforts to improve development out- and broader issues, such as organizational cul- comes, by fully exploiting both the revolution ture, that make it a challenge to evaluate. Not in information technology and the Bank's surprisingly, in spite of the progress to date, comparative advantage as a source and aggre- more can be done to strengthen the Bank's gator of development knowledge, are highly knowledge sharing initiative. Management is relevant to client needs, international develop- working to strengthen the strategic direction ment practices, and Bank interactions with its of its knowledge work and to incorporate it clients. It concludes that the Bank has made more explicitly into operations. The review good progress in establishing the tools and will contribute to this effort. activities to support the knowledge initiative. However, the review notes two areas where the Bank could improve its support to developing II. OED FINDINGS countries. First, the Bank could do more to This review assesses the relevance of the apply its knowledge-sharing tools directly to Bank's knowledge initiative as well as the the Bank's core business processes (direct sup- effectiveness of new knowledge-sharing activi- port for task teams in their operational work). ties and global knowledge programs undertak- Second, according to the review, Regional en to implement the initiative. The review also units, country teams and task managers could examines the supporting institutional infra- usefully develop a more strategic approach to structure that has been built over the past six the knowledge dimensions of the Bank's serv- years. The methodology used for this evalua- ice to its clients (notably, CASs could be more tion was wide in scope, including surveys of explicit about knowledge strategies). Advisory Services (ASs) and Thematic Groups (TGs), structured interviews with 25 task team Recommendations. The review recommends members and 30 additional interviews, expert that Senior Management provide greater reviews, and a survey of stakeholder views strategic direction and oversight. The review 81 82 | S H A R I N G K N O W L E D G E also recommends that Network and Regional implementing combines the three objectives knowledge-sharing initiatives be closely linked outlined in a progress report made to an infor- to core business processes, and that they move mal meeting of Executive Directors in October beyond knowledge aggregation and access to 2001: improved internal knowledge sharing; increased focus on knowledge adoption, adap- greater sharing of knowledge between clients tation, and use. Finally, the review recom- and partners; and enhanced client capacity for mends the establishment of frameworks for knowledge use. monitoring and evaluation of Network, Regional/country, and global knowledge-shar- The Strategy's Strengths. Management agrees ing programs and activities. with the review's assessment of the knowledge strategy's strengths, which emphasize engaging staff, clients, and partners to learn from each III. MANAGEMENT COMMENTS other, with the overarching goal of improving This section sets out Management comments our work in poverty reduction. Although pos- on the Bank's knowledge-sharing strategy, its itive ratings from outside experts and organi- institutional infrastructure, and on internal zations are important, the favorable ratings and external knowledge sharing. (The last two staff offer on their access to knowledge are linked--sharing knowledge internally has required to do their work are equally impor- little value unless this knowledge is passed on tant. As the OED report notes, almost 90 per- to clients and partners.) cent of staff concluded in the 2002 staff survey that they have access to adequate knowledge A. The Bank's Knowledge Strategy and information to do their work, a sharp The review describes the Bank's knowledge- increase from 1997 (OED 2003a). It is note- sharing strategy as innovative, broad ranging, worthy that the Strategic Compact Assess- and responsive to the challenges of develop- ment, acknowledged by Executive Directors to ment. It also identifies areas where the initia- be very thorough and frank, concluded that tive could be strengthened. Management the main knowledge-sharing elements had agrees that the Bank's global knowledge and been implemented. its ability to facilitate the sharing of knowl- edge among its clients and partners have enor- A More Open Organization. Management mous potential in enhancing development also wants to highlight a major benefit of the impact and welcomes OED's evidence of knowledge strategy not emphasized by the major progress in a relatively short time, while review, that of helping to make the Bank a sig- also recognizing that achieving the full poten- nificantly more open and transparent organi- tial of this initiative will require continued zation. This transformation has been influ- Management attention. enced by a number of knowledge-sharing programs launched by the Networks, Regions, The Strategy's Continuing Evolution. Man- the WBI, and others. These programs provide agement agrees that the Bank's strategy has Bank clients and partners access to a wealth of evolved, from an initial focus on enhancing the knowledge resources, and encourage them to Bank's internal knowledge capture and dis- provide substantive comments and suggestions semination capacities, to expanding knowl- on specific Bank policies, strategies, and docu- edge sharing with clients and other develop- ments, as well as other development topics. ment partners. This evolution has been largely They have also fostered greater participation driven internally: the Bank has been a pioneer by key clients and partners, resulting in an in public sector knowledge management, and improved environment for operational work in it has had few models or examples to follow or the Bank's client countries. The underlying from which to learn. Management notes that premise is that the Bank does not possess a to maximize overall impact, the strategy it is monopoly on knowledge. In this light, and as A N N E X H | 83 the review notes, its knowledge-sharing strate- Thematic Groups. Management welcomes the gy supports development changes that puts review's comments on Thematic Groups, and increased emphasis on ownership, partnership, it agrees that they comprise an extremely het- and results--all processes that place a premi- erogeneous group with regard to funding, um on the exchange of ideas and information. membership, and functions. It also concurs with OED's assessment regarding the value of Further Strengthening the Design of the Strat- stronger systems to validate lessons learned egy. Management agrees that the Bank can and good practices. Management also agrees usefully put more emphasis on better integrat- that there are a variety of budgeting, program- ing knowledge sharing in the Bank's core busi- ming, and reporting models and more stan- ness processes, and as noted above this is part dardization, based on best practice, could be in of the Management agenda going forward. order. In addition, TGs have a challenge to However, integrating knowledge-sharing pro- better engage staff outside the Washington grams into Bank operational work has been office (including through enhanced use of tech- only one part of the overall strategy. Another nology) and to provide more direct support to key element has been sharing knowledge with task teams where there is demand. However, external clients, civil society organizations, the indirect support offered by TGs to task and the public at large, notably in the context teams should not be underestimated. Most TG of global public goods (and knowledge itself is leaders see "support to operations" as their a global public good) and, in particular, pro- most important objective. Management notes viding a unique platform for that knowledge that, even though institutional incentives for sharing. Management also recognizes the need operating TGs have declined since the end of to put into place specific and time-bound the Strategic Compact, nearly 80 still exist and objectives for its knowledge-sharing activities many are very active and productive. In fact, in a manner consistent with its other activities. the review cites examples of TGs that have However, institutionalizing a comprehensive demonstrated real success. The review also knowledge sharing strategy generally takes states that only about one-third of TG mem- longer than the 3-to-5 year period cited by the bers describe their participation as active. review. Most knowledge sharing initiatives Judged by most research on communities of have been launched in the private sector, practice, this level of participation would be where they are usually easier to implement and considered high, particularly when participa- measure because of clearer incentives, concrete tion is voluntary. Further, the review does not outputs, and measurable contributions to the mention other communities of practice, both bottom-line in terms of speed to market or internal and client-focused, that are not classi- reduction in cost of production or delivery. fied as TGs. Some of these are communities of Although there are lessons to be drawn from practice facilitated and supported by WBI private sector companies, there is a limited (including Knowledge for Development, Com- body of experience of public institutions that munity Empowerment, and Social Inclusion). measure the impact of knowledge sharing in These groups have been instrumental in link- terms that go beyond financial metrics. ing hundreds of practitioners in developing countries. (As noted above, for good reason, B. Internal Knowledge Sharing to Improve the OED review limited itself to knowledge Bank Operations sharing activities outside those supported by The Bank has made great strides in internal WBI.) knowledge sharing, notably through Thematic Groups, Advisory Services, and Web sites. Advisory Services. The report states that the However, even more can be done, including majority of Advisory Service (AS) "queries strengthening linkages among knowledge, might have been handled by previously exist- learning, and research. ing structures," and that ASs "serve a wider 84 | S H A R I N G K N O W L E D G E range of audiences and purposes than is languages other than English. The Bank's intended by their primary objective of enhanc- intranet for staff use was a 2002 Intranet ing operational effectiveness." Management Design Winner in the Nielson Norman Group agrees that the Bank should continue efforts to contest. Going forward, further strengthening direct clients to Web sites to meet their basic of the "management" element of content man- information needs rather than make use of agement (notably work processes and editori- valuable AS time, and that efforts should be al value-added) remains a priority. Since the made to encourage staff and clients to make launch of the Internet Services Program in use of ASs to find answers to substantive prob- April 2001, implementation of a new Content lems. However, the role that ASs can be Management System has begun, and a signifi- expected to play in support of operational cant number of Intranet, and a smaller number work should not be exaggerated, and the value of external Web sites, have been migrated to a and cost effectiveness of ASs as information more robust web publishing platform. A next intermediaries for a broader clientele should step is to design work processes that enhance not be underestimated. Efforts to enhance the the capture and dissemination of demand- effectiveness and efficiency of ASs are under driven content. An important step toward way--WBI regularly convenes meetings of an enhanced support for operational work is the Advisory Service Community (comprised of piloting of "role-based portals," including a AS staff from across the Bank), and a Unified planned Knowledge and Learning Portal, as Case Management system that will speed the well as portals for Executive Directors, clients, delivery of responses and significantly improve and project teams, that will allow users to reporting capabilities is being developed. A "pull" content that they need for their day-to- next step will be to develop standards to mon- day work rather than having it "pushed" to itor AS performance and evaluate their contri- them by web content suppliers. Management bution. Supported by Management, these col- would like to note, however, that since the ini- lective efforts should also begin to address tiation of the knowledge-sharing initiative, the some misperceptions as to the value and pur- Web has become an absolutely critical com- pose of ASs, making it clear that their purpose munications and operational tool for the is to go beyond a limited referral service and to Bank, and that it has already transformed the serve as "integrators" for the Bank's knowl- way Bank staff work. edge communities. Linking Knowledge Sharing and Learning. Internal and External Web sites. There have Management agrees that the links between been huge leaps in the use of the web by staff knowledge sharing and learning activities and clients over the past few years (from 2 mil- could be strengthened in the Regions and Net- lion page views in 1998 to 18 million in 2002) works, and that although several units are and in efforts to research and respond to user making important efforts to coordinate the demands. Management is working to steadily two dimensions, more can be done. The inte- improve web content for both internal and gration of these two functions is being led external audiences, including better targeting, jointly by the Director of WBI's Global with positive results. For example, the Bank Knowledge and Learning Pillar and the Chief recently launched an Arabic version of the Learning Officer, who has a dual reporting line external Web site, to complement English, to the Vice President for Human Resources Spanish, French, and Russian versions. The (HR) and the WBI Vice President. World Bank external Web site in Spanish was named site of the year for 2002 in the eco- Linking Knowledge Sharing and Research. nomic and business category by Yahoo's Span- Management also agrees that it is too early to ish service. Fifteen country Web sites (includ- assess the impact on the quality of operations ing India and Japan) offer information in of efforts to link the Bank's research and A N N E X H | 85 knowledge-sharing activities, but the new rela- nerships with local institutions to leverage tionship between WBI, Development Econom- local and global knowledge and establish the ics (DEC), and Poverty Reduction and Eco- basis for long-term institutional development. nomic Management (PREM) appears It is important to note that almost half of all promising. This initiative spans policy, Bank Economic and Sector Work (ESW) is research, and country-level capacity enhance- done jointly with client countries, providing an ment services, and aims to ensure more important basis for linking global and local responsive and effective upstream contribu- knowledge, and that the trend is toward even tions to key country tasks based on a compre- more joint ESW. hensive view of country needs. Collaboration has also been extended in the form of a joint Development and Integration of Global DEC/PREM/WBI trade department. Areas of Knowledge Initiatives. Management wel- intense collaboration are being extended to comes the review's comments on three of the poverty and governance. Bank's innovative global knowledge initia- tives. Although the Global Development C. Sharing Knowledge with Clients and Learning Network (GDLN), Global Develop- Partners ment Network (GDN), and the Global Devel- In addition to the responses that Management opment Gateway are key initiatives, it is offers below regarding the specific points important to note that they represent only a raised in the review on this topic, it is worth part of the broad spectrum of global knowl- noting again that the OED review, for good edge initiatives the Bank has initiated and sup- reason, did not cover the activities of the WBI. ported. Therefore, there was no specific attention to the activities of the WBI in sharing knowledge Global Development Gateway. Management with clients and partners and coordinating notes the importance of the creation through knowledge-sharing activities across the Bank. the Gateway of new and unique Internet serv- ices for the development community. In the Enhancing the Effectiveness of Bank Knowl- early days of its inception, there was much edge Sharing with Clients. Management com- debate concerning the Bank's possible domi- mends the OED for its efforts to survey client nation of development content on the Inter- perceptions and agrees with the conclusions net. While this issue and several others have that emerged from this effort. The most recent been successfully resolved, some issues Implementation Forum for Bank Management remain, especially with regard to the inde- drew similar conclusions regarding the critical pendence of the new Development Gateway importance of local institutional capacity for Foundation, given the Bank's prominent role the long-term effectiveness of development in its governance mechanisms. Equally critical assistance. As a result, WBI has been asked to is the issue of financial sustainability. Both the take the lead in supporting the scaling-up of Country Gateways, which have been created capacity enhancement efforts across the Bank. with significant funding from the Bank, and This will ensure that the best available tools the central Development Gateway team, have are put to use, lessons learned are incorporat- limited time horizons for their current funding ed into project design, and the impact of these streams, so it is important that long-term efforts is monitored and strengthened over funding be secured. Securing this funding will time. In cooperation with country teams and depend heavily on demonstrating to potential other appropriate Bank units, WBI's new sponsors the important value added of the Regional and Country Pillar has begun to Development Gateway's services in a dynamic develop more comprehensive and focused global technology market. The Bank will need capacity enhancement strategies for priority to ensure that the Development Gateway countries, which draw on medium-term part- serves staff and clients even more effectively, 86 | S H A R I N G K N O W L E D G E without exercising undue influence over its GDN. Considerable progress has been made governance structures. In that regard, the in establishing the credentials of GDN as an recent addition of the Country Analytic Web entity that is fully independent from the World site (CAW) to the Gateway and the work Bank. However, it may take some time for underway on a harmonization Web site for many of those familiar with the GDN to over- the Gateway are important expansions. A come their tendency to think of it as a World wide set of development partners created the Bank unit. The decision by GDN's multina- CAW to facilitate coordination and coopera- tional, multidisciplinary, governing body to tion among countries and donors with the relocate the GDN Secretariat to India is goal of improving development impact and expected to change perceptions about GDN's cost-effectiveness for both capacity building independence. Management is also aware of and knowledge sharing. The goal of the inter- donor concerns regarding GDN's position as active harmonization Web site will be to pro- an intermediary funding channel between the vide development practitioners ready access regional networks and themselves. However, to the emerging stock of good practices on aid since its launch, the GDN has made a consid- harmonization at the donor and country lev- erable effort to consult on a regular basis with els. Facilitating access to information across a donors to match GDN offerings with the inter- range of subject areas will support harmo- ests of individual donors. Lastly, the signifi- nization efforts by identifying projects where cant contributions from developing country resources are being shared and alignment of governments are a good indication that the policies are taking place. GDN is responding to client demand. GDLN. Management agrees that GDLN's Cross-Cutting Issues. Management agrees that major strength is in its multimedia capacity to the future directions of the three global knowl- "deliver programs in languages and formats edge initiatives referred to in the review consistent with the information and skill needs require careful attention, notwithstanding the of targeted audiences." Another strength is its early accomplishments of the initiatives. ability to reduce the amount of travel for staff Working with local partners that are institu- and clients, with major cost savings. In some tionally and financially independent while cases, such as the events of September 11, increasing Bank staff awareness of these tools 2001 and the recent SARS outbreak in East for use in support of Bank operational work is Asia, the network of Distance Learning Cen- a sensitive task, one that requires diligent ters has allowed projects and events to proceed attention by Management and active aware- despite travel restrictions. The power of this ness on the part of Development Gateway, virtual convening format is compelling. GDLN, and GDN staff. Although demand for its services has increased, GDLN faces a number of chal- D. Supporting Institutional Infrastructure lenges, in terms of utilization, governance, Work is under way to strengthen the support quality assurance, and financial stability. An for knowledge-sharing activities, including the effort is already under way to address these governance structure, monitoring and evalua- issues. Discussions have already been held at tion, and technology. the Vice Presidential-level Knowledge Steering Committee and with Managing Directors, and Resources and Governance. The governance terms of reference have been issued for an of the knowledge initiative has evolved along external consultant to make recommendations with the strategy. The Vice Presidential-level on the strategy and business model for GDLN. Knowledge Steering Committee was estab- Moreover, in spring FY04 there will be a con- lished to fulfill this role, with WBI acting as sultative meeting held with the participating the Secretariat. In addition, WBI holds regular Distance Learning Centers. meetings of a committee of key knowledge A N N E X H | 87 actors from across the Bank to discuss issues much collaboration is already taking place: relating to knowledge sharing and to make many information technology staff are now recommendations on these issues as they move located in the Regions, working closely with to the Knowledge Steering Committee for operational staff. In addition to the positive decision. changes made in the area of content manage- ment noted above, the ongoing development of Monitoring and Evaluation. Clearly another a Knowledge and Learning Portal will also important focus of reinvigorated Bankwide improve standards for Web content, without guidance and coordination mechanisms will imposing undue costs and constraints. be monitoring and evaluation (M&E). There have been notable efforts to track knowledge People. Management concurs that peoples' activities effectively, and there is much to learn behaviors are the most important part of effec- from these efforts. However, they have been tive knowledge sharing and use. It should carried out in a nonstandard, highly decentral- be noted that the link between improved ized way within Networks, Regions, and other knowledge sharing and a change in the Bank's Bank units. This has made it difficult to assess internal culture is not causal; instead, they go impact across the institution as a whole. The hand-in-hand. Knowledge-sharing tools and recent initiative to evaluate Sector Board per- approaches can affect organizational culture, formance, notably with regard to knowledge but the culture must be conducive to main- sharing efforts, by the Quality Assurance streaming knowledge sharing. As the review Group (QAG) is a positive development and states, several steps have been taken to culti- will contribute to increased performance by vate a knowledge-sharing environment. Most knowledge communities. Recognizing the importantly, staff are evaluated on their con- need for appropriate metrics to assist the Bank tribution to learning and knowledge sharing to put in place a sound evaluation mechanism and managers are evaluated on their success in for knowledge sharing, WBI participated with creating an environment conducive to learning the American Productivity and Quality Center and knowledge sharing. Other steps, including in a recent Knowledge Management Measure- the creation of knowledge management job ment Benchmarking Study. This allowed the profiles by HR, have followed. However, chal- Bank to learn from the experience of global lenges remain that largely reflect the informal leaders in the field, and it will be followed up nature of the knowledge-sharing program. by intensive work with key actors across the Management acknowledges this issue, and it Bank to devise an M&E strategy to address will examine specific measures to further some of the issues highlighted in the OED encourage contributions to the knowledge ini- review. tiative in ways that do not add to the net bur- den on operational staff. Technology. Management agrees that improve- ments have been achieved in broadening geo- IV. CONCLUSIONS graphical connectivity and accelerating access to institutional information, for people inside Bank Management is fully committed to and outside the Bank. There have been many improving development outcomes by exploit- other improvements, including the develop- ing both the revolution in information tech- ment and mainstreaming of e-learning plat- nology and the Bank's comparative advantage forms, collaboration workspaces, e-discussion as a source and aggregator of development applications, and video services, to name just a knowledge. Management welcomes OED's few. Although collaboration between informa- finding that the Bank has made good progress tion technology staff and operational staff can since 1996 in providing staff, clients, and part- be strengthened, and the use of specific tools ners with faster access to Bank knowledge and analyzed under the M&E work cited above, expertise and that this progress is recognized 88 | S H A R I N G K N O W L E D G E by both Bank staff and client countries. It rec- Committee, facilitated by WBI, provides a ognizes the difficulty of taking the knowledge mechanism to better coordinate policies and strategy to the next level in terms of enhancing programs across the Bank. Second, QAG has the knowledge dimensions of the Bank's serv- started to assess the performance of Sector ice to its clients and has taken on this chal- Boards with regard to their core functions, lenge. At the same time, Bank Management including knowledge management and learn- realizes that it takes more than seven years to ing. These assessments provide an excellent effectuate the changes in organizational cul- mechanism to monitor and improve the quali- ture, processes, and practices that will fully ty of knowledge work performed by Thematic mainstream knowledge sharing. Groups. Third, there is a concrete focus on bringing together the staff learning and knowl- Recent Steps. A number of very significant edge-sharing agendas of the Bank. This is steps has been taken recently to improve the reflected in the joint areas of responsibility of implementation of the knowledge strategy. the Chief Learning Officer (CLO) and WBI's First, WBI has been given an explicit mandate Director of Global Knowledge and Learning. to support and champion the implementation Management will rely on these mechanisms to of the Knowledge and Learning Bank. The address the recommendations of the OED Vice Presidential-level Knowledge Steering Review. A N N E X H | 89 Summary of OED Recommendations with Management's Response OED Recommendation Management Response 1. While recognizing that all units across the Bank have Management agrees that greater strategic direction and oversight should be exercised responsibilities for making knowledge sharing a way over the Bank's knowledge processes. To undertake these and other functions of doing business, management should exercise greater recommended in this review, the role of the VP-level Knowledge Steering Committee will strategic direction and oversight over the institution's be strengthened in FY04. The Committee has been active throughout FY03, reviewing knowledge processes. issues ranging from the business model for GDLN to the role of knowledge products and services in the context of current and future lending scenarios. By fourth quarter FY04 More specifically, management should: the VP Knowledge Committee will have defined a strategy for implementation. Define clear responsibilities and accountabilities of Management agrees on the fundamental need to define responsibilities and Corporate, Network, and Regional units for integrating accountabilities across all Bank units with the goal of more fully integrating knowledge knowledge sharing into the Bank's core business processes sharing into the Bank's core business. The recent creation of Knowledge Management and better integrating knowledge and learning programs. job profiles by HR will aid in this effort. WBI, in cooperation with HR and the CLO, will also continue to play a lead role in integrating knowledge and learning programs. The development in FY04 of a Knowledge and Learning Portal is intended to help address this challenge. Ensure that incentives are aligned with responsibilities, Management notes that including knowledge sharing as a key aspect of staff and especially at the task manager level. manager evaluations has had a positive impact on incentives. Management agrees that incentives for participating actively in knowledge sharing can be further improved. The VP-level Knowledge Steering Committee will review this issue. The proposed Partnership Council should articulate a Management notes that the role of the new Global Programs and Partnership Council strategic approach to the Bank's role in existing and any (GPPC) will apply chiefly to knowledge initiatives created with external partners. In such new global knowledge initiatives; and the Bank should cases, the GPPC will rely on the referral of issues from the VP-level Knowledge Steering prepare and institute a long-term plan for the GDLN-- Committee. The formulation of a long-term plan for GDLN is already under way. Terms including a financial plan as well as clarification of the of reference have been issued for a consultant to develop a strategic plan and a business Bank's forward involvement and the responsibilities of model that would address the sustainability of GDLN, and the Bank's future role, and a different Bank units for managing that involvement and potential exit strategy. the integration of the GDLN into Bank country programs. 2. Network and Regional knowledge-sharing activities Management agrees that the Bank should focus more on knowledge adoption, adaptation, should tightly link to core business processes, and move and application. Specifically, efforts will be made to further mainstream existing best beyond knowledge aggregation and access to increased practices, which seek to better integrate global knowledge to local conditions (including focus on knowledge adoption, adaptation, and use. indigenous knowledge, participatory ESW--which is rapidly becoming the standard, and communities of practice). This issue will be on the work program of the VP-level For example: Knowledge Steering Committee in FY04. Networks should set clear objectives for anchor, thematic Management agrees that Networks should play a more active role in setting clear group, and advisory services support of operational teams. objectives for supporting operational teams. The results of the upcoming QAG assessment of Sector Board performance will be reviewed to determine ways in which the lessons learned and methodology can be utilized by Networks. Regional and country units should make explicit the Management agrees that more work should be done to make explicit the role of knowledge objectives of CASs and projects, and the knowledge in specific activities as well as in the Bank's overall development goals. As a strategies to be used in leveraging knowledge in support part of its capacity enhancement efforts, WBI will work to identify the knowledge of the achievement of overall development goals. components of CASs and projects in cooperation with Regional units. However, no new requirements will be imposed on staff for this purpose and efforts will be made to ensure that this work is client-driven and not imposed by the Bank. Network and Regions should strengthen their respective Management agrees that the Networks and Regions should coordinate their responsibilities in the capture, validation, and application knowledge-sharing responsibilities more effectively and scale up successful programs. of lessons learned and good practices. The VP-level Knowledge Steering Committee will provide guidance in this area. 90 | S H A R I N G K N O W L E D G E OED Recommendation Management Response 3. Frameworks should be established for monitoring Management agrees that monitoring and evaluation frameworks should be an integral and evaluating Network, Regional/country, and global part of knowledge-sharing activities across the board. In FY04, WBI will initiate a process knowledge sharing programs and activities. of developing a Bankwide framework for measuring the impact of knowledge-sharing activities drawing on good practices from other organizations, and engaging stakeholders This involves: from across the Bank. An evaluation framework will be completed by fourth quarter of FY04 with the first steps toward implementation starting in FY05. Setting outcome objectives and supporting indicators Management agrees that the Bank should establish specific outcome objectives and (baselines, monitorable targets, and benchmarks of supporting indicators. These will be explored through the above mentioned progress). measurement framework. Defining procedures, roles and responsibilities for Management agrees that it is imperative to clearly define roles and responsibilities in monitoring progress and evaluating achievements against monitoring and evaluating progress in achieving knowledge-sharing goals. These will be the stated objectives. explored through the above mentioned measurement framework. Also, depending on the outcome of its ongoing baseline As outlined above, Management notes that QAG will review its experience with Sector assessment of the performance of sector boards, QAG Board reviews; Management will formulate its work program in this area based on should continue to review at appropriate intervals the that review. quality of the Networks'knowledge sharing. OED country assistance, sector/thematic, and project evaluations should look beyond the quality of underlying analytical work to the adequacy of the knowledge transfer processes and their impact on the development effectiveness of Bank-supported programs. And both QAG and OED should build client feedback into their assessments. ANNEX I. CHAIRMAN'S SUMMARY: COMMITTEE ON DEVELOPMENT EFFECTIVENESS Main Findings of the Report. This evaluation There is broad agreement between OED assesses the relevance and effectiveness of the and Management on the scope and focus of Bank's Knowledge Initiative (KI), launched by the recommendations, including with respect President Wolfensohn in 1996. The KI com- to the need for more fully integrating knowl- mitted the Bank to become a knowledge bank edge sharing into the Bank's core business, that would work to enhance its ability to learn including strengthening strategic direction from its own experience in development, as and oversight, and monitoring and evalua- well as that of others and utilize this knowl- tion. edge to improve the quality of Bank operations as well as sharing the information and experi- Main Outcomes of the Committee's Discus- ence directly with clients. The OED evaluation sion. The committee welcomed the thought- finds that the Bank's commitment to a knowl- provoking report and reiterated the impor- edge initiative was timely and appropriate. The tance of the subject, not only because it is one strategy responded to challenges facing the of the four priorities identified in the Strate- developing countries in trying to reduce the gic Compact but also because of its intrinsic information technology gap. The Bank is now salience for all of the Bank's business. The recognized as a leading knowledge-manage- committee noted the complexity of the topic ment institution. Knowledge is easier to access. and the challenges in evaluating knowledge Good progress has been made in establishing functions, and generally felt that OED had the internal architecture to support the Bank done a commendable job. The committee knowledge initiative but more needs to be done also noted the progress that had been made to make the initiative more operationally by management but also acknowledged that relevant and to strengthen the governance much remained to be done, especially with arrangements and work processes for its imple- regard to developing a monitoring and evalu- mentation. The evaluation makes three recom- ation framework to specifically include mendations for management's consideration: knowledge work in CASs, and get a better understanding of what the Bank is accom- · That management exercise more strategic plishing with its capacity building work. The direction and oversight over the Bank's committee was overall supportive of the rec- knowledge processes ommendations. It noted the need to revisit · That networks and regions strengthen the the issue of global public programs (GPPs) link between knowledge sharing activities separately. It asked that management inform and non-lending activities it of progress in the knowledge initiative in · That the Bank establish frameworks for about two years time. CODE will issue a monitoring and evaluating Bank knowledge report to the Board on its discussion of the programs and activities. report. 91 92 | S H A R I N G K N O W L E D G E Among the issues discussed were the implementation issues; undertaking experi- following: mentation on support to clients to make OED recommendations and management knowledge more operationally effective; and response. Several members noted that the ensuring that knowledge issues are more sys- Management Response alluded to several tematically addressed in the CAS. recent initiatives and ongoing programs in the Bank, especially the role of the WBI in coordi- Strengthening the link between the Bank's nating knowledge activities in the Bank-- knowledge activities and core operational which had not been assessed in the OED processes. The committee urged management review. Management noted that WBI supports to do more to integrate knowledge products and champions the Knowledge Bank but that strategically into the routine core business of implementation of knowledge management the Bank at the country level and wanted to and knowledge sharing is at the level of the understand better the role of Sector Boards regions and networks. In crafting the manage- and Networks in this regard. They underlined ment response, based on comments and input the need to be less-top down, more demand from the relevant VPUs, WBI was reflecting its driven and relevant at the country-level, more recently assumed role as the Knowledge Bank supportive of south-south learning and to bal- Secretariat. OED noted that at the time the ance a focus on knowledge sharing with a evaluation was launched WBI did not have the focus on knowledge creation. One speaker overarching responsibility it has today for proposed that the present arrangements for many of the learning initiatives. knowledge sharing be revamped with clear definitions of institutional roles and responsi- Strategic Directions and oversight of Bank bilities to ensure more accountability. Man- Knowledge processes within the Bank. The agement noted that it has taken a number of committee felt that there were more questions steps to ensure that the knowledge created is raised than answers provided by the OED demand-responsive, based on local knowledge report and the Management Response, espe- and relevant to client needs, as have other cially as regards going forward and what groups such as the Global Development Net- improvements could be reasonably expected work (GDN). For example, the GDN funds within the short- to medium-term. Members proposals generated from local researchers. In would have liked to see a more concrete, time- some cases, clients pay for knowledge services. bound response with proposals from manage- WBI already supports south-south learning ment on what the Bank wanted to achieve in programs, notably in connection with PRSPs. the immediate- to medium-term and how this could be implemented and monitored in a Need for a M&E framework and the demon- pragmatic way, within a decentralized matrix stration of value for money. The committee system. The committee noted that main- underlined that after seven years it was imper- streaming knowledge management "as a way ative that the initiative put in place a function- of doing business" would require full manage- ing monitoring and evaluation (M&E) frame- ment commitment at all levels to address work, including QAG reviews of knowledge incentive and organizational culture issues. content in quality reviews. Speakers cautioned Management noted that it had already under- against complacency about the Bank's compe- taken a number of actions, which responded tence and urged the Bank to draw on best to the OED recommendations, including: practice examples from private sector knowl- establishing a Vice Presidential-level Steering edge management services. Management Committee, chaired by the Managing Director informed the committee that for the past four in charge of Human Development, comprising years the Bank had received high ratings from the Regions, Networks, Information and Tech- external evaluators on its practices in the area nology Services (ISG), and WBI to focus on of knowledge management. It also noted in its A N N E X I | 93 Response that, as a leading knowledge institu- the Global Development Network (GDN). tion in the international development arena, Some suggested exploring the causes of under- there is a limit to the experience of other insti- utilization of the GDLN while others empha- tutions that the Bank can draw on, notably for sized the need to avoid duplication and pro- measuring the impact of knowledge sharing. posed an exit strategy be developed for the The committee noted that it would be impor- Development Gateway to free-up resources of tant to hold knowledge products fully the DGF for other uses. Management noted accountable in the CASs and to address issues that the Development Gateway was now under of cost-benefit with regard to various elements the direction of the independent Development of knowledge sharing (e.g. research, capacity Gateway Foundation and was no longer building, and training). Management noted housed within the Bank. The new CEO of the that QAG was already in the process of evalu- Foundation will make a presentation to Execu- ating Sector Board performance, including in tive Directors on the Foundation's business the area of knowledge sharing. QAG is also plan on October 23, 2003. Management working with WBI to improve its project qual- informed the committee that a number of ity at entry evaluations with respect to knowl- actions have already been taken to address the edge sharing. In addition, WBI participated issues raised in the report, including establish- with the American Productivity and Quality ing the GDN in Cairo and New Delhi as an Center in a recent Knowledge Management international organization. It informed the Measurement Benchmarking Study, allowing committee that the majority of the GDLN cen- the Bank to learn from the experience of glob- ters were performing well, but some have just al private sector and public sector leaders in started their activities and still need to develop the field and will be working intensively with a viable business model. Management noted key actors across the Bank to devise an appro- that a consultant has been hired to review the priate M&E strategy for knowledge sharing GDLN system and make recommendations on activities. Based on this ongoing work, an a strategy and business model. Management evaluation framework is planned for comple- also noted that it was exploring how to address tion by the end of the FY with the initial steps the issue of rationalization of user fees for its toward implementation in FY05. services across regions. The committee will have an opportunity to discuss issues related to Long-term plans for the global knowledge pro- GPP when OED submits its review of GPPs in grams. The committee raised questions about this fiscal year. the future plans and issues of sustainability related to the Development Gateway, Global Pietro Veglio Development Learning Network (GDLN) and Acting Chairman ENDNOTES Chapter 1 1. See Annex A for a detailed chronology of the implementation of the knowledge bank initiative from 1996 through 2002. 2. For three consecutive years beginning in 2000, the Bank was recognized as one of the top 20 "Most Admired Knowledge Enterprises," all but one of the other 19 being private sector corporations. Awards are made by an international panel of corporate executives and knowledge management experts, based on an enterprise's knowledge culture, knowledge leaders, knowledge-based products and solutions, intellectual capital, collab- orative knowledge sharing, organizational learning, focus on customer knowledge, and transformation of knowledge into shareholder value. 3. This was one of 10 main recommendations of an external advisory panel enlisted by the Bank to review progress on implementing the knowledge initiative. Prusak (2001) identified the need for "meaningful met- rics to evaluate progress, keep knowledge activities focused on the bottom line, and enable the development of criteria to guide future investments." Also, since 2001, the Operations Evaluation Department (OED) Annual Reports on Operations Evaluation have highlighted the absence of adequate monitoring and evalu- ation in this area. See, for example, OED 2003b. 4. The shift in terminology coincided with the move of the Bankwide knowledge-sharing coordinating unit out of the Information Solutions Group (ISG) to the former Operational Core Services (OCS), both changes reflecting a broadening of emphasis from information technology tools to direct interactions among practi- tioners as a central feature in the implementation of the knowledge initiative. 5. See Annex B for an inventory of the Bank's main knowledge-sharing activities, with an indication of the net- work, Regional, and global programs and activities that are the specific focus of this review. 6. Cummings 2002, a background paper to this study, available on request. Additional advice was provided by Gabriele McLaughlin, Director, National Knowledge Practice, Xerox Connect, Inc; and advice sought from Dorothy Leonard, Harvard Business School, a leading scholar on the application of knowledge in public and private enterprise. 7. A fuller discussion of these lessons drawn from the literature is presented in Annex C. 8. See Annex D for a more detailed outline of the indicators used in this assessment. 9. Annex E provides a more detailed description of the evaluation methodology. Chapter 2 10. Key documents are World Bank 2002/2001/2000; 2001b; 1997. 11. Those challenges are the focus of the 1998/99 World Development Report on knowledge for develop- ment (World Bank 1998a). In its concluding Part Three, on policy priorities, there is a discussion of what international institutions and developing country governments can do to "close knowledge gaps and overcome information problems" inhibiting the achievement of countries' development goals. 12. See Chapter 5 for more detail on these expenditures. 13. See Annex F for a description of a sample of players in the knowledge for development field. 95 96 | S H A R I N G K N O W L E D G E Chapter 3 14. Experience shows that, to be effective, communities of practice need to foster a high level of interac- tion among members on an ongoing basis. Though they may form out of informal work groups that begin by simply exchanging ideas, those that endure tend to evolve toward more purposeful interac- tions and more rigorous expectations and accounting of outcomes. See Annex C for further discus- sion of lessons from studies on communities of practice. 15. In FY02, total expenditures amounted to some US$8 million, of which some US$7 million was from Bank budget and close to US$1 million from trust funds (See Chapter 5 for a more detailed description of budget expenditures). 16. Fifteen of 23 respondents indicated that they spend 1-5 staff weeks per year on thematic group coor- dination. Others who reported spending more time did not distinguish between thematic group coor- dination and anchor knowledge-sharing work (OED survey of thematic group coordinators, 2002). 17. While these surveys have response rates ranging from 12-60 percent and do not ask identical ques- tions, some findings emerge with enough similarity and frequency to allow for certain broad obser- vations to be made about the efficacy of activities, supplemented by more qualitative evidence from interviews with staff. Bank thematic group and broader network knowledge management surveys, including: Financial and Private Sector Investment Network; Environmentally and Socially Sustainable Development Network 1999; the- matic group member survey, September 2000; Human Development Network Education Sector Sur- vey March 2001; Poverty Reduction and Economic Management Network KM Survey, January 2002; Governance and Public Sector Reform, thematic groups survey, 2002. 18. These findings are from an OED survey conducted separately from this review. See OED 2003a. 19. Though some 11 of 28 thematic group leaders surveyed for this review report recent efforts to engage field staff more, these efforts are largely intermittent--involving workshops held in field offices, videoconferences, and thematic group meetings during learning weeks (OED thematic group survey). 20. In response to the various thematic group surveys that have been conducted in the last few years, including the one conducted for this review, staff specifically cite three factors as constraints to their participation: (1) thematic group budget cuts of the last three years; (2) participation on their own time, not as part of their Time Recording System (TRS); and (3) lack of recognition of thematic group work in Overall Performance Evaluations. 21. Joint OED/WBI survey of 15 advisory services/help desks, March 2002, with 12 of 15 participating services responding to question on staff size. 22. Bankwide, the total expenditure for all advisory services in FY02 was US$2.2 million, as indicated in the data presented in Chapter 5. 23. OED task team survey and Project Appraisal Document review of a sample of projects; Bank sur- veys; OED thematic group survey. 24. This section covers Web use for internal and external knowledge sharing because the Bank's external site is important for both and there are crosscutting issues of audience focus and content manage- ment that pertain to both the internal and external sites. Most of the section draws on data from Forum One Communications (World Bank 2003b, 2002b). 25. In the last five years, the Bank has also increased the use of e-mail newsletters as a vehicle for dis- semination of information, knowledge, and ideas. According to the Forum One report (World Bank 2003b), by mid-FY03, the Bank had 50 e-mail newsletters with some 90,000 subscribers, at least 20 percent in developing countries. Many of these are maintained by thematic groups and network or Regional knowledge coordinators as a way to create communities of practice around the world relat- ed to a specific development issue. No evaluation of their impact was made as a part of this review. The growth in the number of subscribers as well as the specificity of their subject matter suggest that they are an important additional vehicle for Bank knowledge dissemination. And, as the Forum One E N D N O T E S | 97 report indicates, e-mail newsletters serve as a method of disseminating World Bank resources espe- cially to audiences who have limited Internet access. 26. Available in English, French, Russian, and Spanish, the external sites are used with far more frequen- cy than the external Web sites of the International Monetary Fund and other multilateral develop- ment banks, though surpassed by those of the United Nations and European Union (World Bank 2003b, 2002b). 27. See Annex C for further discussion of the requirements of effective content management. 28. The Quality Assurance Group reported improvements in quality at entry from 82 percent satisfacto- ry in FY96 to 94 percent satisfactory in FY01 and a drop in FY02 to 86 percent satisfactory. Quality of supervision increased from 63 percent in FY96 to 89 percent in FY02. The Quality Assurance Group's quality ratings on ESW also showed steady improvement, from 72 percent satisfactory in FY96 to 94 percent satisfactory in FY02 (World Bank 2003e, table 3.4, p. 1). Chapter 4 29. Outside the scope of this review, various country programs have also given increased attention to helping countries build up their information and communication technology (ICT) capabilities and become more competitive in the global knowledge economy. This growing knowledge-economy work involves both analytical work and lending that aims to increase the capacity of countries to acquire and use knowledge effectively. 30. See Annex E for a description of the methodology used in this review and interview survey. 31. Based on regular Quality Assurance Group reviews of samples of ESW, this type of highly participa- tory ESW increased from about 30 percent in FY00 to nearly 45 percent in FY02. 32. This same point is made by implication in the Strategy Update Paper for FY03-05 (World Bank 2003c), which emphasizes the need for more integrated packages of knowledge and lending in coun- try programs. 33. See Annex G of this study. Survey included 121 respondents from 5 countries: Bangladesh, Brazil, Poland, Senegal, and Tanzania; and OED Workshop on Leveraging Knowledge for Development. See Annex G for the summary report of the five-country client survey. A summary report of the work- shop is available on request. 34. The issue of translation has been recently addressed in a management proposal for a World Bank Group translation framework, discussed by the Board on July 3, 2003 (World Bank 2003f). 35. This same point is highlighted in the recent OED Brazil Country Assistance Evaluation (OED 2003c), which notes that the government sees an important role for the Bank "in informing society about long-term structural issues, their potential solutions, and the trade-offs involved. To play this role effectively, however, the Bank must make a greater effort to disseminate its work among the several potential audiences and to the population at large." 36. While in-country respondents may have vested interests in this issue, it should be noted that interna- tional experts participating in the OED workshop emphasized the same point, noting that knowledge is local and contextual, and needs to be actively absorbed to be used effectively. 37. Tanzanian academic respondent to OED client survey. Respondents' views regarding the weaknesses in dissemination, adaptation, and institutional capacity building are consistent with findings from across the Bank's 30 client- feedback surveys conducted over FY99-02. 38. See Annex B, which includes a list of the Bank's six global knowledge initiatives. 39. Currently under review are recommendations to: (1) establish a "gated decision process" for Bank innovation and development of ICT and knowledge initiatives; (2) concentrate on the Bank's role as a "content aggregator;" and (3) create an innovation fund to support new start-ups. See Digital 4Sight 2002. 98 | S H A R I N G K N O W L E D G E 40. OED staff participated in the design of the Development Gateway, and two OED staff currently serve as guides for the Aid Effectiveness topic section, which involves review and approval of materi- al submitted for publication, development of policies and guidelines for what will be covered in the section, and work to stimulate interest in and use of the material by development practitioners and researchers. 41. One of the most vocal critics of the Bank's role in the Development Gateway has been the Bretton Woods Project 42. For example, Eldis, an online gateway managed by the Institute of Development Studies, offers con- tent for the research and academic communities; and OneWorld, an independent Web portal, has created a global communications network for NGOs and has found an innovative means of address- ing inequalities in Internet access by partnering to provide local radio programming. 43. It has been most effective when a topic has a well-defined application or purpose (as in the case of the Afghanistan Reconstruction Gateway) or has created an interactive community. 44. A strong feature of the start-up of the Country Gateways has been the transparent and phased approach by which they have been supported. This has involved an open call for proposals for small-scale grant funding, and seed financing provided through a two-stage planning and implemen- tation process. The approach contrasts with the nontransparent approach to establishing GDLN- affiliated centers, which has involved the Bank in direct negotiations rather than an open proposal process. 45. InfoDev, which receives World Bank funding from the Development Grant Facility, is a global pro- gram that provides grants and technical assistance to encourage policies that increase connectivity and to support innovative use of ICTs for development at global, Regional, national, and local levels. 46. This number, provided by WBI's GDLN division, excludes four centers listed in information submit- ted to OED by Regional units (one each in AFR, EAP, ECA, and South Asia [SAR]). The discrepan- cy appears to derive from the absence of a common definition of what constitutes a GDLN center. Differences in definition relate to issues of whether a center should achieve a number of successful activities and have a minimum number of activities per year before being regarded as a full GDLN center. 47. This figure was provided to OED in the third quarter of FY03 by the Bank's GDLN team. 48. Though programmed events declined in number from approximately 273 in FY01 to 259 in FY02, they were at a level of 142 by mid-FY03, indicating an upward turn. Moreover, despite fewer dis- crete events in FY02, they involved multiple sessions and, therefore, an estimated increase in number of participant training days. Yet the latter trend appears to have reversed in the first part of FY03. 49. This is the finding of the background report on GDLN prepared for this evaluation and it is a main point in a GDLN presentation (World Bank 2003a). 50. If DLC use were scheduled at 50 percent maximum use (that is, sustainability level according to the current business model), an estimated sixfold increase in the number of events fielded in FY02 would be needed in FY03 (World Bank 2003a, p. 18). 51. An innovative approach to supporting them is the LCR Content Development Fund. With resources from the Spanish government, this fund finances content development from sources of expertise in the Region. 52. Africa Economic Research Consortium (AERC), Center for Economic Research and Graduate Edu- cation (CERGE-EI), East Asian Development Network (EADN), Economic Education and Research Consortium (EERC), Economic Research Forum-Middle East and North Africa (ERF), Latin Ameri- can and Caribbean Economic Association (LACEA), and South Asian Network of Economic Insti- tutes (SANEI). 53. This summary provides preliminary observations from a GDN review being conducted as part of OED's evaluation of the Bank's global programs. E N D N O T E S | 99 Chapter 5 54. Expenditures cited in this report are based on data from the Bank's Business Warehouse RM 2.3-2.6 and 5.1 (Supervision) and (Lending) reports. These figures retroactively incorporate budget reform changes made in FY02 to include direct costs only, and the FY03 changes in business process coding, which alter the way knowledge management expenditures are reported. Comparator data from a benchmarking study conducted by the American Productivity and Quality Center suggest that the level of dedicated staffing for knowledge-sharing activities in the Bank is about twice that of the other best practice organizations (APQC 2002a, Annex B-1). A different set of comparators cited in early Bank documentation for the knowledge initiative reported staff levels similar to or larger than the Bank's current level, but the basis of selection of this second set of comparators was not indicated. 55. Not included in these numbers are the Bank's substantial investment in its information and commu- nication systems that support the knowledge-sharing activities as well as other Bank processes. Over the period FY00-02 alone, total ICT expenditures amounted to just over US$183 million, of which information and communication technology accounted for some US$111 million. 56. The Quality Assurance Group is currently carrying out a first assessment of sector boards and their contribution to the quality of Bank operations, including the contribution of their knowledge man- agement function, which may lead to movement toward certain common standards, but none cur- rently exist as benchmarks for the assessment. 57. An attempt to develop and use knowledge flags as a way to accomplish this tracking has stalled because of lack of agreement on how to devise a system that serves the purposes of enhancing per- formance but that is not excessively onerous. 58. In July the Bank launched a Web-based application for project teams, the "Project Portal," to facili- tate team efforts to access and create documents. In particular, the Project Concept Note (PCN) and Project Appraisal Document will be launched in Microsoft Word, so document sharing and editing will be easier. 59. As stated in a 1998 paper explaining the concept of the Knowledge Bank: "While communications will be facilitated by electronic means, the knowledge management system is about people, not about machines. The challenge is to harness the technology to link people together and to leverage its impact for development.... The technology is the relatively easy part.... The more difficult part of the Knowledge Bank will be the necessary organizational culture shift away from an individualistic mode of working and storing knowledge, towards a sharing team-based mode of work." See World Bank 1998c, pp. 3-4). 60. See figure 2.1 in Chapter 2. BIBLIOGRAPHY Background Papers prepared for the evaluation, signified by an asterisk*, are available from OED on request. APQC (American Productivity and Quality Center). 2002a. "Building and Sustaining Communities of Practice." Houston, TX. ------. 2002b. "Retaining Valuable Knowledge: Proactive Strategies to Deal with a Shifting Work Force." Houston, TX. ------. 2000a. "Successfully Implementing Knowledge Management." Houston, TX. ------. 2000b. "Sharing Best Practices in Knowledge Management." Houston, TX. Bair, J., and R. Hunter. 1998. "Introducing the KM Project Viability Assessment." Gartner Group Research Note. Stamford, CT. *Bevan, David. 2002. "Public Expenditure Management." OED Background Paper, Washington, D.C. Bezanson, Keith, and D. Stone. 2003. "Linking Research, Monitoring and Advocacy or Research and Policy Makers: Are they Worlds Apart or Worlds Together?" Presented at the Global Policy Workshop on the Development Impact of Rich Countries' Policies, Cairo, 16-17 January 2003. Carayannis, Elias G., and Bruno Laporte. 2002. "By Decree or By Choice? A Case Study, Implementing Knowledge Management and Sharing at the Education Sector of the World Bank." WBI Working Papers, World Bank, Washington, D.C. Collison, V., and Geoff Parcell. 2002. Learning to Fly: Practical Lessons from One of the World's Lead- ing Knowledge Companies. Oxford, U.K.: Capstone. *Cummings, Jeffrey. 2002. "Knowledge Sharing: A Review of the Literature." OED Background Paper, Washington, D.C. Davenport, T. H., and John Glaser. 2002. "Just-in-Time Delivery Comes to Knowledge Management." Harvard Business Review, July:107-11. Davenport, T. H., and Morten T. Hansen. 2002. Knowledge Management at Anderson Consulting. Watertown, MA: Harvard Business School Press. Davenport, T. H., and Gilbert Probst. 2000. Knowledge Management Case Book. London: John Wiley & Sons. Davenport, T. H., and Larry Prusak. 1998. Working Knowledge: How Organizations Manage What They Know. Watertown, MA: Harvard Business School Press. Davenport, T. H., D.W. De Long, and Michael C. Beers. 1998. "Successful Knowledge Management Pro- jects." Sloan Management Review: 43-47. Digital 4Sight. 2002. "Using Technology to Scale Up the World Bank's Impact." Toronto. Dixon, Nancy, 2000. Common Knowledge: How Companies Thrive by Sharing What They Know. Watertown, MA: Harvard Business School Press. Ellerman, David, Stephen Denning, and Nagy Hanna. 2001. "Active Learning and Development Assis- tance." Journal of Knowledge Management 5 (2): 171-79. 101 102 | S H A R I N G K N O W L E D G E Ellis, M. Sheldon, and Melissie Rumizen. 2002. "The Evolution of KM at Buckman Laboratories." Knowledge Management Review 5 (1): 12-15. Fonseca, Flavia. 2003. "The Effect of an Integrated Knowledge Management Architecture on Organiza- tional Performance and Impact: The Case of the World Bank." Ph.D. dissertation, University of Mary- land, Washington, D.C. Fontaine, Michael A., and David R. Millen. 2002. "Understanding the Value of Communities of Practice: A Look at Both Sides of the Cost/Benefit Equation." IBM Institute for Knowledge-Based Organiza- tions, Armonk, N.Y. *Goldwyn, David L. 2002. "Power Sector Reform Review." OED Background Paper, Washington, D.C. Haas, Martine. 2002. "Acting on What Others Know: Distributed Knowledge and Team Performance." Ph.D. Dissertation, Harvard University, Cambridge, MA. Hansen, Morten, Nitin Nohria, and Thomas Tierney. 1999. "What's Your Strategy for Managing Knowl- edge?" Harvard Business Review, March. *King, Kenneth, and Simon McGrath. 2002. "Knowledge Sharing in Development Agencies: Lessons from Four Cases." OED Background Paper, Washington, D.C. *Kleemeier, Elizabeth, and Keith Stallard. 2002. "Water Supply Sector." OED Background Paper, Wash- ington, D.C. Lev, Baruch. 2001. Intangibles: Management, Measurement, and Reporting. Washington, D.C: Brookings Institution. McDermott, Richard. 2002. "Measuring the Impact of Communities." Knowledge Management Review 5 (2) 26-29. Morey, Daryl, Mark Maybury, and Bhavani Thuraisingham, eds. 2000. Knowledge Management: Classic and Contemporary Works. Cambridge, MA: MIT Press. OED (Operations Evaluation Department). Forthcoming. "The World Bank's Approach to Global Pro- grams: An Independent Evaluation--Phase II." Washington, D.C. ------. 2003a. "Lessons and Best Practices at the World Bank: A Survey Examining Formulation, Valida- tion, and Utilization Processes." Washington, D.C. ------. 2003b. Annual Report on Operations Evaluation. Washington, D.C. ------. 2003c. Brazil Country Assistance Evaluation. Washington, D.C. ------. 2002a. "An Evaluation Framework for the Bank's Knowledge-Sharing Process: An Interim Report of the OED Review." Washington, D.C. ------. 2002b. "The World Bank's Approach to Global Programs: An Independent Evaluation--Phase I." Washington, D.C. Prusak, Larry. 1999. "Action Review of Knowledge Management: Report and Recommendations." IBM Institute for Knowledge Management, Armonk, NY. *Romiszowski, Alex. 2003. "The Global Development Learning Network: A Review of the First Two Years of Operation." OED Background Paper, Washington, D.C. Roos, Johan, J. Roos, N.C. Dragonetti, and L. Edvinsson. 1998. Intellectual Capital: Navigating the New Business Landscape. New York: New York University Press. Seeley, Charles P. 2002. "Igniting Knowledge in Your Business Processes: How to Connect Knowledge Activities with Your Business Processes." Knowledge Management Review 5 (4): 171-79. Skyme, David. 2000. Cited in Knowledge Management: Classic and Contemporary Works, D. Morey and others, eds. Cambridge, MA: MIT Press. Stone, Diane. 2003. "The `Knowledge Bank' and the Global Development Network." Global Gover- nance (9) 43-61. ------, ed. 2000. Banking on Knowledge: The Genesis of the Global Development Network. London: Routledge. B I B L I O G R A P H Y | 103 Stewart, Thomas. 1997. Intellectual Capital: The New Wealth of Organizations. New York: Doubleday. U.S. Department of the Navy. 2001. Metrics Guide for Knowledge Management Initiatives. www.don- imit.navy.mil *Walker, Louise, Heidi Hay, and Helen Walsh. 2003. "Start-Up of the Development Gateway." OED Background Paper, Washington, D.C. Wolfensohn, James D. 1996. "People and Development," Annual Meetings Address, October 1, Washing- ton, D.C. *Woodall, Maureen. 2002."Primary and Secondary Education." OED Background Paper, Washington, D.C. World Bank. 2003a. "The World Bank's Commitment to GDLN: From Start-up Towards Consolidation." GDLN, Washington, D.C. ------. 2003b. Forum One Communications. World Bank Web in Review 2002: Overview of Usage of Online Services. Washington, D.C. ------. 2003c. "Strategy Update Paper for FY03-05: Implementing the World Bank's Strategic Frame- work." Washington, D.C. ------. 2003d. "Medium-Term IT Strategy for the Bank." Information Solutions Group. Washington, D.C. ------. 2003e. Annual Report on Portfolio Performance: Fiscal Year 2002. Quality Assurance Group. Washington, D.C. ------. 2003f. "A Document Translation Framework for the World Bank Group." Washington, D.C. ------. 2002, 2001, 2000, 1997. "Strategic Framework." ESSD,Washington, D.C. ------. 2002a. China Country Assistance Strategy 2002. Washington, D.C. ------. 2002b. Forum One Communications: World Bank Web in Review: Overview of the Status and Usage of Online Services. Washington, D.C. ------. 2001a. "Assessment of the Strategic Compact." World Bank External Relations Department, Washington, D.C. ------. 2001b. "Management's Assessment of the Strategic Compact," 3 (3) Washington, D.C. ------. 1998a. World Development Report 1998/99: Knowledge for Development. New York: Oxford University Press for the World Bank. ------. 1998b. The Knowledge Bank. Washington, D.C. ------. 1997. The Strategic Compact: Reviewing the Bank's Effectiveness to Fight Poverty. Washington, D.C. ------. Various years. Bank Staff Survey, 1997, 1999, 2002. Washington, D.C. OPERATIONS EVALUATION DEPARTMENT ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Operations Evaluation Department (OED) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. OED assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achieve- ment of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. THE WORLD BANK 1818 H Street, NW Washington, DC 20433 USA Tel: 202 473 1000 Fax: 202 477 6391 www.worldbank.org TMxHSKIMBy357125zv":;:=:":# ISBN : 0-8213-5712-3