96638 Devising an Appropriate Strategy for Capacity Building of a National Monitoring and Evaluation System: Lessons from Selected African Countries Robert Lahey April 2015 This note examines key stages of national monitoring and evaluation (M&E) systems’ development in different country contexts and argues for tailored approaches to building M&E capacities. The development of national monitoring and evaluation systems (NMESs) rests on four building blocks—vision of leadership, an enabling environment, capacity to supply and analyze M&E information, and capacity to demand and use M&E information. Developing countries can differ significantly in their performance along these four dimensions—some have virtually no NMES, while others are at a much more advanced stage where reliable and timely M&E information is generated and used. Most developing countries likely fall in between. Using the example of five African countries, this note discusses the differences and similarities in capacity-building needs for countries at different levels of NMES development. In recent years, emphasis is shifting to a new paradigm This note1 examines NMES capacity building in regarding national monitoring and evaluation (M&E) the broader context of the new paradigm, examining capacity development—from a historical pattern of the state of NMES development in five countries in M&E being carried out primarily in the context of Africa—Benin, Botswana, Ethiopia, Mauritania, and cooperation and development aid, to a new concept Senegal—to explore and identify patterns and lessons centered around national ownership and M&E capac- that could help inform future NMES capacity develop- ity that is linked to the national vision of the country, ment in general. accountability, and good governance (see, for example, Picciotto [2007] and Menon [2010]). As the frame Background: A Framework of reference for national monitoring and evaluation for Developing an NMES system (NMES) development changes, the nature of If an M&E system is to be owned by a country, it needs the capacity-building strategy, including the support to be linked to the national development plan of the that may be given by the international community to country and integrated into the operations and culture NMES development, will need to be revisited. NMES of government institutions. To be sustainable though, development, including M&E capacity gaps, needs to be governments must believe in the utility of the NMES considered in a broader context and built around more and understand its benefits to them. And, to do that, than simply traditional training initiatives. To work they must own the system. In other words, national toward the goal of a sustainable, effective, national, ownership implies a particular cultural, social, and and country-owned NMES, key foundation pieces such political context (Segone 2010). as infrastructure and supporting institutions need to With a broad goal of good governance as a driver be developed, along with training of country officials. behind developing an NMES, its development could be thought of as resting on two overarching influences: (i) the Historically, efforts in many countries have been political will for change within a country and (ii) technical directed at improving the supply of M&E information, factors that will influence the pace of M&E infrastructure though most often in the context of individual projects development. Upon these two foundation pieces sit four or priority sectors such as health or education, but gener- essential building blocks (figure 1; Lahey 2013). ally not in a comprehensive fashion inherent in an NMES. i. vision of the leadership within the country; Further, the demand for or use of M&E information has ii. an enabling environment for an M&E system to develop often been linked primarily to funding or reporting require- and function; ments from donors or international agencies—for example, iii. the capacity to supply and analyze M&E information— requirements of Poverty Reduction Strategy Paper (PRSP) the technical capacity to measure and analyze perfor- funding or Millennium Development Goal (MDG) prog- mance and provide credible and timely information; ress reporting. In other words, M&E systems, where they and exist, are often piecemeal and not necessarily aligned with iv. the capacity within the system to demand and use the broad set of the country’s national development goals. M&E information—key users include government Additionally, the NMESs quite likely are missing many key institutions, ministries, citizens, media, and other institutional components and the infrastructure needed to stakeholders. make them both national in scope and sustainable. Political support is an essential driver to launch and While each country is unique and faces its own chal- fund the NMES exercise; lead the change in organiza- lenges in “growing” its NMES, implicit in the framework tional culture that may be needed; provide the NMES of figure 1 are some important considerations for NMES champion(s); ensure an enabling environment; deflect development: resistance to M&E and the changes it might imply; and help • A broad set of players needs to be involved in NMES ensure long-term sustainability of the NMES. development for it to be both effective and sustainable. However, the successful development of an NMES • The goal is not simply to create an M&E capability, but takes more than political will. Even with a resource com- to use performance information to improve public sec- mitment to invest in M&E development, there may be tor management and governance. As such, a capability technical hurdles that require a lengthy process to put in within government is being created to both generate place and develop credible data systems. In addition, it (that is, supply) performance information as well as takes time to train M&E specialists and educate managers to use performance information in decision making throughout the system on how and where M&E informa- by government managers. tion will be used. This is generally a lengthy and iterative • The use of (or demand for) M&E information will process, as the experiences of most countries using M&E function if there are effective incentives built into systems confirm, and one where allowance for continuous the system. learning and improvement through oversight mechanisms • Training and development is required for both tech- is particularly beneficial to the improvement of the NMES. nical analysts as well as nontechnical managers in government—these are the eventual users of the M&E Figure 1. Four Essential Building Blocks for an information who will need to understand how and Effective NMES where M&E information can help them in the man- agement of their programs and policies. Senior and Vision of Enabling political officials need sufficient M&E knowledge so Leadership Environment that they grasp the importance of the NMES to achiev- ing the high-order goals of accountability, results-based management, and sound governance for the country. Political Will for Change The Importance of Understanding Capacity to Supply Capacity to Current Level of NMES and Analyze M&E Demand and Use Development in a Country Information M&E Information The four building blocks of figure 1 represent the founda- tion pieces of a country’s NMES. Countries are at different Pace of Development of stages/levels of development of what might be considered M&E Infrastructure their national M&E system—everything from essentially no NMES to a point of having an effective and sustainable Source: Author’s illustration. 2 NMES (regardless of what it looks like), and everything in shows five levels of NMES development. In between the between. two end states—that is, level 1 with no or low M&E, and International experience has shown that national level 5 with a “mature state”—there could be even more M&E capacity development is an iterative process, gener- than three levels or stages of NMES development. And, as ally developed incrementally, sometimes in a piecemeal noted above, movement along the continuum, from one manner, and, not uncommonly, with false starts (Mackay level to another, is not necessarily linear. It is still useful 2007). For illustrative purposes, figure 2 shows what though to adopt a framework to assess where a country might be considered a continuum in the development of might be situated in terms of its NMES development and, the four building blocks that comprise a nationally owned from the perspective of the international community, M&E system. In broad (and simplistic) terms, figure 2 consider what this might imply in terms of developing Figure 2. Key Stages and Drivers to Reach Mature NMES Status High Level 5: Mature state • Political will Level 4: translates into Growing M&E commitment to system fund M&E to • Political will and needed levels commitment to • Country-owned M&E and perfor- NMES providing Level 3: needed results Committed mance manage- ment measurement development and reporting • Apparent po- • Funding and detailed plan to • Credible, Level 2: Early litical will to build reliable, and results-oriented develop country- initiatives timely informa- Country M&E Readiness national M&E owned NMES • Apparent • Institutional tion generated political will, but system from country • Increased structures, poli- no resource cies, and opera- systems, includ- commitment commitment to ing at subna- country owner- tional guidelines • Some (lim- in place tional level ited) initiatives ship and use of • Evaluation a Level 1: M&E beyond • Capacity-build- by country to ing strategy for key part of the No/low M&E international ac- develop M&E human resourc- NMES • Little/no countability structures es and data • Self-sustaining commitment • Institutional • M&E focused development • M&E part of the to M&E—not a structures being largely on moni- • Evaluation policy, planning, priority established toring imple- recognized as and budgetary • Minimal M&E • Limited mentation rather important part of cycle capacity and evaluation than measuring M&E system, but • Focus on experience • Some M&E results still underutilized accountability, • Any existing capacity build- • Little/no • Capacity gaps good gover- M&E focused on ing, but contin- evaluation in implementa- nance, and accountability to ued capacity • Some, but lim- tion and use, but transparency donors gaps, particularly ited M&E training plan in place to • Little/no training in implementa- and capacity deal with these or M&E capacity tion and use building building Low Current M&E Capacity in Country High Source: Author’s illustration. 3 or supporting an appropriate NMES capacity-building part of this study, there is considerable documented and strategy and action plan. public information about the state and development of While each country is unique in how far and how fast South Africa’s NMES from which to determine its rela- it may roll out a national M&E capability (and indeed, how tive placement along the continuum (see, for example, that may be institutionalized), under the new paradigm, Goldman et al. [2012]). countries do share the broad goal of developing an effective An assessment of M&E capacity and identification of and sustainable NMES, centered around national owner- capacity gaps was conducted for each country and examined ship and M&E that is linked to the national vision of the specific factors aligned with each of the four NMES building country, accountability, and good governance. It is impor- blocks. CLEAR (2013) provides a detailed discussion of the tant, however, to recognize that a country with a weak basis assessment’s findings for Benin, Mauritania, and Senegal.3 for an NMES (as determined through an M&E Diagnostic) In examining the NMES building blocks and sub- likely needs a capacity-building strategy somewhat different factors, a pattern of development emerges, both within than a country with a much stronger NMES capacity, that individual countries and across countries. is, a country further along the continuum. The framework It would seem apparent that NMES development must can help in devising a more comprehensive and country- start with the vision of country leadership, which provides specific strategy for NMES development than a generic not only adequate space to allow development of the NMES, strategy that focuses primarily on training country officials, but also demonstrates commitment to fund and develop in the absence of institutional changes and infrastructure a national M&E system. With this commitment, and of- development. ten with the support or encouragement of international development partners, the focus quickly turns to training country officials to build capacity to supply M&E informa- NMES Development in Selected tion. The capacity-building process may slow down or take African Countries considerably longer than anticipated by senior officials The framework of figure 1 and 2 was applied to five Af- though, largely because of the need to clarify and establish rican countries: Benin, Botswana, Ethiopia, Mauritania, the institutional arrangements, roles, and responsibilities and Senegal.2 of the various actors within the system. Institutionalizing Figure 3 shows where each country is situated on M&E may indeed require some form of public sector the NMES development continuum, based on assess- reform, including national planning and budget decision ment of the development of each of the four NMES making. In all level 3 countries examined, establishing the building blocks for each country. While the assignment appropriate institutional arrangements so that generation of countries to a particular level is not an exact science, of M&E information is a normal course of doing business the relative placement of the five countries in relation to has proven to be a lengthy exercise—and one that likely does each other is likely close to an accurate depiction. A sixth not get enough attention. African country, South Africa, is included in figure 3 Another important area of NMES development that for comparative purposes. While South Africa was not a clearly gets too little attention is developing the capacity to Figure 3. Selected African Countries on the NMES Development Continuum (1) (2) (3) (4) (5) No/low Institutional M&E system Some elements Institutional elements of elements for of M&E elements M&E M&E Effective, Some NMES Implementation Essentially Limited sustainable, but efforts Challenges: not static donor driven implementation M&E use, data, and others Mauritania Senegal Benin Ethiopia Botswana South Africa Source: Author’s illustration. 4 use the M&E information within the particular country. lenges across all four building blocks. But what is perhaps The experience of the five selected countries suggests that surprising is that, even for countries well along the NMES this may be due to several factors: continuum (countries at level 3 in this analysis), there are • Less attention is focused on information use, including fundamental elements of the NMES development process how to build the necessary incentives into the system to that still represent challenges and capacity gaps. For the three encourage/force use of M&E information in planning, level 3 performers in the sample though, the nature of the management, and decision making in government. M&E issues and challenges is more subtle. In other words, • The vision for using M&E information is often narrow it should not be assumed that some of the fundamental in scope and may derive largely from the historical need capacity building—for example, raising awareness and un- to satisfy accountability and reporting requirements derstanding of M&E among senior officials—is less impor- of international agencies (for purposes of PRSP, MDG tant for these countries. Despite being at different levels of progress, and so forth), a valuable use but insufficient NMES development, there are still common M&E capacity to support the broad needs of good governance and challenges that each country in this sample faces. Where results-based management (RBM). they likely differ is in the intensity of the challenge and ap- • In the five selected countries, there are problems with propriate intervention—given that, in some countries, there the quality of information currently generated by the is a virtual absence or limited acceptance of M&E, while in M&E systems. These problems derive in part from other countries, M&E capacity development (in some form fundamental problems with data in general and the or other) has been ongoing for some period of time. need for national data development plans. Listed below are key M&E issues/challenges observed • There may be potential roadblocks arising from histori- to be common to all five countries examined: cal perceptions of M&E in general and evaluation in i. Awareness and understanding of M&E roles and uses particular as being a threatening control-type function • Need to raise awareness/understanding of the that serves largely to criticize. In other words, officials various roles and uses of an NMES to improve are not appreciating the knowledge-generating and governance and management decision making learning aspects associated with evaluation as a tool • Clarifying what is meant by a national M&E sys- of the NMES. tem to increase understanding and buy-in at both The other development aspect shown in figure 1 relates the political and institutional level to various institutional elements that help enable NMES • Weak culture and understanding of evaluation success and sustainability in a country—such factors as the and its importance to an NMES existence of a national statistics agency (NSA), a systematic planning function in government, oversight bodies such as ii. Institutional arrangements for M&E parliament and a national audit office (NAO), and others. • Where M&E units exist, key capacity challenges Their development may be on a separate track from NMES also exist—such as inadequate resources and not development, due to, for example, the various initiatives to enough trained staff develop and improve the NSA in each of the level 3 coun- • Lack of clarity regarding roles and responsibilities tries examined. NSA (and data) development needs to be and limited harmonization/coordination of vari- linked to development of the NMES. It must quickly be ous M&E efforts emphasized though that they are not the same thing,4 but • Little/no institutionalization of evaluation—no there is a correlation between the two—if a high priority is systematic evaluation of government programs being given to NMES development, priority also needs to and policies be given to data development, along with associated sta- iii. Human resource expertise tistical and analytical expertise. Similarly, the relationship • Not enough officials trained in M&E; high turn- between NMES development and other enablers needs to over; and lack of evaluation expertise be recognized and fostered. • Little or no local training in M&E • Too little practical M&E training and common M&E Capacity Gaps/Challenges: understanding of M&E tools and methods Countries at Different Levels iv. Data issues of NMES Development • Need to improve reliability, quality, completeness, In examining NMES development in the five selected and accessibility of data countries, it should not be surprising that in level 1 and • Sector data not always consistent with central 2 countries there are fundamental needs and M&E chal- data sources 5 • Not enough and high turnover of data experts iii. supporting evaluation capacity building; and and analysts iv. supporting the development of key enablers of the v. Use of M&E within government NMES. • Potential for increasing the use of M&E in minis- This section examines each category briefly, and, draw- tries and centrally, but many challenges ing on the sample of five countries, illustrates similarities • No evaluation of programs or policies and no link and differences across the three levels of NMES develop- to policy/program development (exceptions may ment in terms of the nature/focus of the capacity-building be internationally funded projects) initiative identified as most appropriate for the specific vi. Performance measurement and reporting country.5 • Not enough focus on results (outputs and (i) Training and human resource (HR) development outcomes) For all countries, regardless of level of NMES development, • Common problems—no data to populate indica- • It is important to recognize the various audiences for tors; inappropriate indicators; poor quality data; M&E training/orientation: political and senior officials; too little analysis; timely release of reports officials needing more technical knowledge on generat- vii. Functions/institutions that help enable NMES ing M&E information; managers whose programs will development be monitored; and the potential users of M&E informa- • NSAs—challenges include not enough trained tion, both centrally and within ministries. staff; data reliability; inadequate analysis; and • The nature of training/orientation clearly differs across processing, storing, accessing, and harmonization different audiences, with nontechnical M&E orienta- of data collection tion designed for political and senior officials. • Planning functions—M&E not always well linked • Awareness/appreciation of the importance of evalua- to or integrated into planning tion (the “E”) as a critical tool in the NMES needs to • Oversight agencies (for example, the NAOs)— be increased. For example, officials need to understand key focus of oversight bodies often on audit, that RBM needs to be supported by both “M” and “E”— control, and fight against corruption, which can “M” on its own is not enough. negatively influence the perception of M&E and performance management For the level 1 and level 2 countries examined, • Civil society organizations (CSOs)—not always • Limited formal exposure to M&E would indicate that clear who speaks for civil society or what influence the priority is to raise awareness and understanding they may have. (advocacy) at the political and senior levels regarding the role and various potential ways that a nation- It should be noted that this set of common M&E issues ally owned, results-based M&E system can be used to and capacity gaps observed across the five selected countries improve public sector governance and management cover all four NMES building blocks. Where they tend to decision making. There needs to be a better under- differ from one country to another is in intensity level, and standing of RBM/performance management and its/ therefore the appropriate responses will differ by country. In their important link to M&E to help enable necessary general terms, observations from the five countries suggest public sector reforms, including national planning and that a distinction needs to be made between level 1/level 2 budget decision making. countries and level 3 countries. This is particularly impor- • Clarify with public sector leaders, as well as M&E tant when examining appropriate strategies and potential champions, what is meant by an NMES, to increase initiatives to support NMES capacity building. understanding and buy-in at both the political and institutional level. Implications for Devising an • M&E training areas need to be identified and a multi- Appropriate NMES Capacity- year training strategy developed, including provision Building Strategy for introductory M&E training. Potential initiatives to be built into the strategy to support For the level 3 countries examined, NMES capacity building in countries at each of the three • The concepts of M&E and RBM are not new, and levels examined are discussed below according to four broad various efforts, some more successful than others, categories, each with a variety of specific activities: have been underway for some time to try to improve i. training and human resource development; national M&E capacity. In this environment, expecta- ii. advising/facilitating the institutionalization of NMES; tions—particularly among senior officials—are likely 6 considerably higher than for level 1 and 2 performers. agency in the country. These could also serve as a Not only is there a need to address existing M&E ca- follow-on to the M&E training/orientation of politi- pacity gaps and challenges, but also a need to manage cal and senior country officials, which are intended to expectations—officials need to understand that NMES explore the concept of institutionalizing M&E within capacity building is long term and iterative. There is the public sector. What this training might include: no quick fix. how best to introduce organizational and operational • In the three countries examined, NMES efforts to changes that may be needed, possible resource implica- date have been fairly narrowly focused (for example, tions, pace of implementation, and more. progress reporting on the national plan), suggesting • Efforts should be made to help ensure all stakeholders a need to broaden officials’ understanding of the are brought to the table—civil society, private sector, various ways that an NMES can support good gover- M&E partners, training institutes, M&E champions, nance, accountability, and improved management and others. practices—in other words, the various roles and uses • NMES institutionalization could be linked to ongoing of M&E in general, and evaluation in particular, that or planned public sector reforms, as appropriate, as a could be built into the government’s management means of clarifying the initiative and gaining broader framework. stakeholder support. • As NMES becomes institutionalized, the HR capacity • With the support of senior country officials, plans gap grows and training needs become more immediate. could be launched to create a central M&E unit to drive Short-term technical needs likely require the intro- future NMES development efforts, and, eventually, duction of more practical and applied M&E training the development of an appropriate multiyear NMES incorporated into targeted NMES development. With action plan. too few in-country M&E specialists, international M&E For the level 3 countries examined, experts are still required, though a mentoring compo- • Analysis of all level 3 countries confirms that their nent should be built into all contracts that include respective NMESs are “still under construction”—the international M&E experts. appropriate institutional arrangements for M&E are • An intermediate and longer-term training strategy often not well established or resourced and there is a and multiyear training plan should be developed that need to clarify roles, relationships, and accountabili- move beyond traditional in-class M&E training to ties of the various actors implicated in the NMES. As also incorporate a plan to develop in-country training noted above, establishing the appropriate institutional capacity via “train the trainer” and eventual develop- arrangements so that the generation of M&E informa- ment of M&E curricula in local and regional training tion is a normal course of doing business has proven to institutions. be a lengthy exercise6—and one that likely has received too little attention in the past. (ii) Advising and facilitating the institutionalization • This implies a need to work with senior officials to of NMES articulate how the current M&E system may need to be For all countries, regardless of level of NMES development, modified, both structurally and in its implementation, • Recognize the need for a plan for overall NMES devel- so as to improve its effectiveness in supporting results- opment and a strong central player/agency within the based public sector management. This likely includes country to serve as the M&E champion to help develop, addressing issues such as: increasing the use of M&E operationalize, and drive the NMES. information across government; expanding the role of • Adopt a phased approach to NMES development, evaluation; linking M&E results to the policy, planning, piloting the introduction of new elements so as to and budgetary cycle; improving the measurement, learn, review, and make the necessary adjustments— monitoring, analysis, and reporting of results; as well as opposed to an immediate, whole-of-government as potential structural and resource implications for rollout all at once. Capacity gaps typically rule out central agencies or ministries. the latter. • Many of the capacity-building efforts need to be aimed For the level 1 and level 2 countries examined, at implementation/operationalization challenges for • With no or limited institutionalization of an NMES, the NMES, that is, moving from theory to reality and high-level discussions would be needed, as well as the gaining full value from the NMES. International good establishment of a steering and/or consultative com- practices are useful to assist this exercise and to arrive mittee. These could be initiated with the support of at an identification of possible modifications to the development partners or the efforts of a lead central current NMES. 7 • Key agencies need to be implicated in the develop- be integrated into ministry-level programming, where ment and roll out of any NMES enhancements and key programs of government would, over time, be institutional changes, either through participation evaluated for their effectiveness. in a high-level coordinating committee or via regular • Introduce, on a pilot basis, an evaluation of a pri- communications and updates from senior country ority area/topic for government. This should be officials. conducted to serve as both a demonstration of the • Once a high-level consensus is achieved, an appropri- benefits that evaluation brings to the NMES as well ate NMES action plan is needed for the way ahead, as a learning experience, where country officials are with specific goals and milestones to be monitored. actively engaged/mentored by international M&E The central M&E unit needs to have the author- experts. ity to proactively lead and champion this NMES • Identify a senior-level committee that would serve as development. a key forum for identifying priority topics for evalua- tion; tabling evaluation findings; and ensuring follow- (iii) Supporting evaluation capacity building through of evaluation recommendations. For all countries, regardless of level of NMES development, • Formal acknowledgement of “evaluation” in the (iv) Supporting the development of key enablers NMES strategy as a tool to measure and analyze of the NMES results-oriented performance of government pro- NSAs—data development grams and policies critical to RBM—in effect, moving • Data challenges are common across all countries and the thinking of “E” beyond a control- or audit-type they need to recognize the importance of investing function to a tool that assesses effectiveness as well as in data development. But, in the case of NMESs, accountability and, in doing so, generates knowledge the need to resolve data issues is more immediate for decision makers. and pressing for level 3 countries, given the positive • Use of log frame (logic model/results chain) approach correlation between NMES development and data to developing results-oriented performance indicators development. and cost-effective measurement strategies. • In the context of NMES development, data develop- • Incorporating evaluation as a key component of the ment should not simply be left to the NSA—there should be coordination among the key central agencies. curriculum for the long-term M&E training and de- Ministry roles also need to be clarified and integrated velopment strategy. with the broader (national) data development. For the level 1 and level 2 countries examined, Evaluation associations—national, regional, • Incorporate into M&E training an awareness of profes- international sional standards of practice and methods commonly • For all countries, development of a national evaluation employed in various types of evaluation. association can help promote an M&E network and • Introduce key elements associated with developing community of practice, as well as support professional terms of reference and managing an evaluation. development. For the level 3 countries examined, • Linking to Web sites and networks of regional and • While having more experience and understanding international evaluation associations is a cost-effective of the NMES, level 3 countries still had considerable method of information sharing regarding M&E prac- difficulty in measuring results—for a variety of reasons, tices, as well as for identifying potential developmental one of which was the paucity of evaluations conducted opportunities. (aside from internationally funded projects). Oversight bodies—monitoring the implementation • Incorporate into NMES strategy specific actions in- of the NMES tended to institutionalize evaluation as a tool of the • Some form of oversight over NMES development NMES. This would include development of an evalu- and implementation is critical for all countries. ation policy, standards, and guidelines that outline ex- Regularly reviewing progress against the plan enables pectations, roles, and responsibilities for the systematic continuous learning and strategy adjustment. It is use of evaluation in the NMES. therefore important to ensure that there are clear • The policy and guidelines would also include a clear accountabilities for various components of NMES expectation that evaluation is a part of the mandate of development. the central M&E function of government (conducting • For level 1 and 2 countries, oversight could be assisted high-level and priority evaluations), and that it should by development partners working with a lead central 8 agency. For level 3 countries, the central M&E unit (in presented extensively to the international M&E community a strong central agency), a high-level committee, or the and is a member of Canada’s Evalution Credentialing Board. NAO could play this role. For most NAOs, this would Lahey has been recognized by the Canadian Evaluation be a new role, thus requiring appropriate training or Society for his contribution to the theory and practice of orientation. evaluation in Canada. Conclusion Notes Regardless of where a country may lie on the NMES 1. The NMES framework is further elaborated in Lahey continuum, it needs a strategy and action plan for NMES (2013) and Lahey (2014). development if improvements are to be made—one that 2. Robert Lahey assisted the World Bank’s CLEAR M&E clearly articulates and assigns roles, responsibilities, and Center serving Francophone Africa in Senegal in the study accountabilities along with milestones that establish ex- of Benin, Senegal and Mauritania, conducted over 2012–13 pectations in terms of the pace and level of improvement. (CLEAR 2013). He led a formal M&E Readiness Assess- Monitoring its development and implementation in effect ment for Botswana’s National Strategy Office and the World amounts to applying the concepts of M&E —both the learn- Bank in 2013. In 2014, Lahey led an M&E Diagnostic for ing and accountability aspects—to help grow and improve Ethiopia’s National Planning Commission and the African the NMES. Indeed, this is good practice at all levels of NMES Development Bank. development. 3. The M&E Diagnostic reports conducted for Botswana As an NMES matures, this level of oversight—ad- and Ethiopia are not currently available. In developing this dressing capacity gaps and identifying an appropriate report, however, analysis has shown strong similarities in strategy for building, enhancing, and modifying the terms of NMES development and capacity gaps between NMES—implies being able to objectively assess the state, each of these two countries and Benin, deemed to be a level acceptance, and utilization of the current version of the 3 performer, as discussed in CLEAR (2013). NMES. This in itself could be threatening to officials 4. While data serve as a critical element/input to enable aligned with NMES efforts. Without this knowledge an NMES, it needs to be emphasized that the NMES is though, of what is working and what is not (and why not simply a mechanism to generate data. The NMES not), an appropriate capacity-building strategy will not links information sets together and within a particular be well informed or address any issues critical to NMES context to provide analysis, knowledge, and advice that effectiveness and sustainability. facilitate critical thinking and evidence-based decision Finally, it should be noted that the NMES capacity- making. building initiatives presented in this note are not intended 5. A more complete discussion of potential initiatives to to be comprehensive—only illustrative of the differences include in capacity-building strategies for countries at each (and similarities) observed across countries at different of three levels is provided in CLEAR (2013). levels of NMES development. These observations rein- 6. In all countries, the adjustments needed to institutional- force the importance of conducting an M&E Diagnostic ize M&E (on both the demand and supply side) may be cul- as a prerequisite to determining an appropriate M&E tural as well as organizational, and international experience capacity-building strategy (Shepherd 2011). But even has shown that this requires considerable time to evolve. more, this note underscores the importance of addressing the fundamental issues of NMES development, even for References countries that may appear to be further along the NMES CLEAR. 2013. An Assessment of Demand for Monitoring, Evalu- continuum. ation and Performance Management Information and Services. Washington, DC: World Bank. (http://www.theclearinitiative. About the Author org/clear_pubs.html) Goldman, Ian, Ronette Engala, Ismail Akhalwaya, Nolwazi Gasa, Robert Lahey was the founding head of the TBS Centre of Bernadette Leon, Hassen Mohamed, and Sean Phillips. 2012. Excellence for Evaluation, the Canadian government’s evalu- “Establishing a National M&E System in South Africa.” Special ation policy center. He has headed evaluation units in a Series on the Nuts & Bolts of M&E Systems Number 21, World number of departments and agencies over a 32-year career Bank, Washington, DC. in the Canadian public service. Since establishing his own Lahey, Robert. 2013. “National Performance Reporting as a Driver consulting firm in 2004, he has advised many countries and for National M&E Development.” In Development Evaluation organizations around the world on building M&E capacity in Times of Turbulence, ed. Ray C. Rist et al. Washington, DC: appropriate to their circumstances. He has written and World Bank. 9 _____. 2014. “National Evaluation (or M&E) Capacity Develop- Picciotto, Robert. 2007. “The New Environment for Development ment: A Framework for Assessing M&E Needs.” In Moving Evaluation.” American Journal of Evaluation 28 (4): 509–21. beyond the Conversation about Poverty:  Shifting the Focus to Segone, Marco. 2010. “Moving from Policies to Results by Devel- Inequality, ed. Ray C. Rist et al. Washington, DC: World Bank. oping National Capacities for Country-Led Monitoring and Mackay, Keith. 2007. How to Build M&E Systems to Support Better Evaluation Systems.” In From Policies to Results: Developing Government. Washington, DC: World Bank. Capacities for Country Monitoring and Evaluation Systems, ed. Menon, Saraswathi. 2010. “A Perspective from the United Nations Marco Segone. Geneva: UNICEF. on National Ownership and Capacity in Evaluation.” In From Shepherd, Geoffrey. 2011. “Conducting Diagnoses of M&E Sys- Policies to Results: Developing Capacities for Country Monitoring tems and Capacities.” Special Series on the Nuts & Bolts of M&E and Evaluation Systems, ed. Marco Segone. Geneva: UNICEF. Systems Number 16, World Bank, Washington, DC. This note is sponsored by the Poverty Global Practice of the World Bank Group. The views expressed in this note are those of the author and do not necessarily reflect the position of the World Bank Group. Additional publications and information can be found at www.worldbank.org/poverty. 10