World Bank Resilience M&E (ReM&E) Good Practice Case Studies World Bank Report | August 2017 CLIMATE CHANGE | SUSTAINABLE DEVELOPMENT ii WORLD BANK RESILIENCE M&E World Bank Report August, 2017 © 2017 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Cover photo: © iStock/LoveSilhouette. Used with the permission of iStock/LoveSilhouette. Further permission required for reuse. Report and Cover design: Lauren Kaley Johnson, GSDPM, The World Bank Group GOOD PRACTICE CASE STUDIES Contents Acknowledgements.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Background.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Kenya Climate-Smart Agriculture Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Country Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 The Kenya Climate-Smart Agriculture (CSA) project.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 CSA Project Resilience M&E. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Mozambique PROIRRI Sustainable Irrigation Development Project–Water Global Practice. . . 11 Country Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Mozambique Sustainable Irrigation Development Project (PROIRRI). . . . . . . . . . . . . . . . . . . . . . 12 PROIRRI Project Resilience M&E. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Status of Implementation and Preliminary Findings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Mekong Delta Integrated Climate Resilience and Sustainable Livelihoods Project. . . . . . . . . . . 17 Country Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Mekong Delta Integrated Climate Resilience and Sustainable Livelihoods Project. . . . . . . . . 18 Mekong Delta Project Resilience M&E.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Annex 1. Mekong Delta Project Results Framework and Indicators. . . . . . . . . . . . . . . . . . . . . . . . . . 22 iv WORLD BANK RESILIENCE M&E GOOD PRACTICE CASE STUDIES 1 Acknowledgements T his publication is a product of the World Bank ‘Results Monitoring and Evaluation for Resilience Building Operations’ (ReM&E) project, which is part of a broader Programmatic Approach on ‘Enhancing Climate and Disaster Resilience of World Bank Sustainable Development Operations.’ This project is managed by the Climate Change Strategy and Operations Team and the Sustainable Development Chief Economist Office, under the Sustainable Development Vice Presidency and the Global Themes Vice Presidency. The ReM&E team is led by Nathan Engle and Ulf Narloch. Core team members include Sundus Siddiqi, and Karima Ben Bih. Additional support to the team has been provided by Anna Williams, Calli VanderWilde, Christina Irene, and Silvia Marquina-Leon. The World Bank’s Independent Evaluation Group has played a key role in facilitating the learning components of this project. Management oversight is provided by Marianne Fay, Stephen Hammer, and Neeraj Prasad. The team also expresses gratitude to the following individuals for their support in serving on the project’s Advisory Group: Rosina Bierbaum, Nancy MacPherson, Heather McGray, Christine Roehrer, Christopher Nelson, Niels Holm-Nielsen, Holger Kray, and Luis Andres. This publication was prepared by Anna Williams and Sundus Siddiqi, under the direction of Nathan Engle. The authors acknowledge, with gratitude, the teams that provided research and analytical contributions to this work. Staff providing useful input and review include Aidan Coville, Anders Jensen, Anjali Acharya, Aniceto Bila, Astrid Zwager, Christophe Crepin, Christine Heumesser, Dinesh Aryal, Florence Kondylis, Iain Shuker, Ladisy Chengula, Maria Iskandarani, and Paul Christian. Finally, the team is deeply appreciative of funding from the Global Facility for Disaster Reduction and Recovery, which has supported the ReM&E project and the drafting of this product. 2 WORLD BANK RESILIENCE M&E GOOD PRACTICE CASE STUDIES 3 Background T hese case studies were developed as part of the World Bank’s Results Monitoring and Evaluation for Resilience Building Operations (ReM&E) project, which aims to develop and increase the application of systematic, robust, and useful approaches to monitoring and evaluation (M&E) for resilience-building projects/programs within the World Bank. The case studies propose to foster a grounded understanding of good ReM&E practices through real-world examples. The intended audiences are Task Team Leaders (TTLs) and operational staff of the World Bank who design and/or oversee the implementation of M&E for resilience-building operations, as well as their counterparts at other development organizations. The case studies focus on three World Bank projects, each highlighting different ReM&E good practices. These projects are the following: • Kenya Climate-Smart Agriculture Project—Agriculture Global Practice; • Mozambique PROIRRI—Sustainable Irrigation Development Project—Water Global Practice; and • Mekong Delta Integrated Climate Resilience and Sustainable Livelihoods Project—Environment & Natural Resources Global Practice. GOOD PRACTICES The following good practices arise from one or more case studies: • Strengthening project design with the help of resilience M&E experts • Engaging relevant stakeholders in the project’s M&E design • Embedding strong resilience framing in project design • Building multiple M&E approaches into project design • Clearly defining resilience-relevant indicators and providing guidance on measurement approaches • Balancing indicator ambition with practicality • Securing resources needed for robust M&E • Making a clear case and choosing clear objectives for impact evaluation • Undertaking evidence-based learning throughout the course of the project to improve implementation and enhance results, in addition to accountability 4 WORLD BANK RESILIENCE M&E Resilience M&E is a relatively new focus area for M&E, and as such these projects reflect a set of emerging good practices. A common theme throughout these cases is “learning by doing,” which includes tailoring M&E design to the unique learning needs and opportunities for a given project. These good practices are just as much process-, capacity-, and resource-focused as they are subject-, sector-, and/or method-focused. Many resilience-focused projects, particularly those that have embedded strong M&E design, are in the early stages of planning or implementation. Their evolution will offer additional lessons over the next several years. The case studies should be viewed in this light—as good emerging practices to continue to observe and learn from—just as our understanding of resilience will continue to evolve over time. GOOD PRACTICE CASE STUDIES Kenya Climate-Smart Agriculture Project COUNTRY CONTEXT A griculture is the dominant source of employment for roughly half of the population of Kenya, contributing to almost 27 percent of the national Gross Domestic Product (GDP) in 2013. Agriculture generates most of Kenya’s food requirements, 65 percent of merchandise exports, and about 60 percent of foreign exchange earnings. About 83 percent of Kenya’s land area is in acutely drought-vulnerable, arid, and semi- arid lands (ASALs) devoted to pastoral practices. Livestock production provides as much as 90 percent of employment and family income in the ASALs, making drought conditions—responsible for more than US$1.08 billion in livestock losses within the last decade alone—especially concerning. Drought response costs, as well as ancillary losses related to production assets and future income, are several times higher in ASALs than in the medium-to-high rainfall areas. The burden on communities only grows as increasing incidence of droughts across the ASALs diminishes the amount of time available to recover, rebuild assets, and strengthen resilience. Only 17 percent of the country, home to 80 percent of the population, is suitable for crop production. An Agriculture Sector Risk Assessment for Kenya conducted by the World Bank in 2015 highlights the significant impact of drought events on Kenya’s agricultural growth: Drought poses the paramount risk to production in the ASALs. Threatened not only by drought, but also by major dislocations, extreme pressure to provide livelihoods for young people, and more frequent and severe food crises provoked by poverty and broader climate change implications, many farms face a potentially untenable future. Furthermore, resource disparities disproportionately affect women, young people, and other vulnerable groups in agriculture, thereby aggravating socio-economic marginalization. 5 6 WORLD BANK RESILIENCE M&E THE KENYA CLIMATE-SMART AGRICULTURE (CSA) PROJECT In this country context, the World Bank Board approved the Kenya CSA project in early 2017, to increase agricultural productivity and build EXHIBIT 1 resilience to climate shocks in arid and semi- arid regions. Co-developed by the Government Project Development Objectives of Kenya and the World Bank, project implementation is scheduled between 2017 and (PDO) and Components 2022. The project development objective (PDO) PDO: To increase agricultural productivity and is to increase productivity and build resilience build resilience to climate change risks in the to climate change risks in targeted smallholder targeted smallholder farming and pastoral farming and pastoral communities in selected communities in Kenya, and in the event of counties in Kenya (see Exhibit 1 for the complete an Eligible Crisis or Emergency, to provide immediate and effective response. articulation of the PDO). Components Four components, including three related to • Upscaling Climate-Smart Agricultural upscaling, strengthening, and supporting CSA, Practices will support the achievement of the PDO (see • Strengthening Climate-Smart Agriculture Exhibit 1). As described in Exhibit 2, the concept Research and Seed Systems of CSA focuses on transforming agricultural • Supporting Climate, Agro-weather, and systems to support development and to provide Market Information and Advisory Services more security in a changing climate. • Project Coordination and Management The Kenya CSA project focuses primarily on: • Improving water and soil management, especially in marginal rainfall zones; • Promoting sustainable, community-driven rangeland management and improving access to quality livestock services in ASALs; • Supporting the generation and dissemination of improved agricultural technologies, innovations, and management practices, including sustainable seed systems; and • Enhancing access to quality agro-weather, climate, advisory, and market information services among farmers/herders for improved decision making. The project beneficiaries include approximately 522,000 households of smallholder farmers, agro-pastoralists, and pastoralists. Of these, approximately 18,000 households are part of vulnerable and marginalized groups. GOOD PRACTICE CASE STUDIES 7 CSA PROJECT RESILIENCE M&E The Kenya CSA project is at the beginning stages of implementation; therefore, this case focuses on aspects of up-front M&E design that reflect good EXHIBIT 2 practices for resilience M&E. The good practices include a carefully designed resilience-focused What is climate-smart agriculture results framework and indicators, engagement of stakeholders in project and resilience M&E (CSA)? design, designing multiple qualitative and The UN Food and Agriculture Organization quantitative implementation approaches for M&E defines CSA as “an approach that helps that will inform mid-course project to guide actions needed to transform and implementation at different scales, and ensuring reorient agricultural systems to effectively that there are sufficient resources for robust M&E support development and ensure food security in a changing climate. CSA aims throughout the project. to achieve three outcomes (‘triple-wins’): (i) sustainably increasing agricultural productivity GOOD PRACTICES | Bringing in resilience and incomes; (ii) adapting and building M&E experts to work with the design team; resilience to climate change; and (iii) reducing strengthening design with M&E strategy to and/or removing greenhouse gas emissions, inform the PDO, project components, and where possible.” See: http://www.fao.org/climate-smart-agriculture/en/ activities; and incorporating M&E with project activities—not simply “tagging on” M&E after the project was designed To understand the current resilience M&E landscape and good practices, the project design team worked with a consulting firm, UNIQUE forestry and land use GmbH (UNIQUE). UNIQUE worked with stakeholders to develop a capacity “lens” to resilience building that they then applied, along with applicable resilience definitions and concepts, to the project’s PDO and theory of change—to the project as a whole as well as to each project component. The UNIQUE consultants helped the project design team refine how project activities would tie to a resilience-focused theory of change; to consider what progress tracking activities might be missing (e.g., feedback loops between farmers and developers of the ICT1 system); and to 1 Information and Communication Technology. 8 WORLD BANK RESILIENCE M&E identify appropriate SMART2 indicators that which inherently involves an iterative learning consider aspects such as aggregation and process with and for stakeholders. The Kenya relevance at different scales. CSA project’s Subcomponent 4.2: Monitoring & Evaluation and Impact Evaluation, includes a budget of US$4.5 million to finance a web- GOOD PRACTICE | Engaging stakeholders in based M&E system that will collect and process the design of M&E project components, theory information at the national, county, and of change (results framework), and indicators community levels, and verify the inputs, outputs, effects, and eventually the impacts of project activities over time. Aside from supporting routine The project task team, UNIQUE, and the Kenyan M&E functions (data collection, analysis, and Government jointly developed project reporting), and the baseline, mid-point, and end components and indicators. Stakeholder of project surveys and assessments, the project discussions generated a holistic view of the will finance a rigorous project impact evaluation;3 project—how each component, subcomponent, conduct thematic studies (quantitative, output and/or outcome statement links with qualitative, and quality of implementation another and with appropriate indicators, which processes) on demand; and support development could inform progress at the different scales. and operation of an ICT-based Agricultural Information Platform. GOOD PRACTICE | Building multiple M&E approaches into project design to inform GOOD PRACTICE | Securing resources needed evidence-based learning during implementation to deploy a range of demand-driven data collection and analysis approaches to utilize at various scales—and thus recognizing the The Kenya CSA project design builds in multidimensionality, interconnectedness, and advanced planning and resource investment scaling considerations particularly relevant to to support ongoing evidence-based learning, resilience interventions as well as rigorous impact evaluation, going beyond standard minimal M&E requirements. This is particularly important for resilience 2 Specific, Measurable, Achievable, Relevant, and Time-bound. 3 This impact evaluation will be undertaken in collaboration with other counterparts, including DIME. GOOD PRACTICE CASE STUDIES 9 SUMMARY The Kenya CSA project has an intentional and • Building multiple M&E approaches into carefully constructed resilience M&E design, which project design to inform evidence-based reflects multiple good practices: learning during implementation; and • Bringing in resilience M&E experts to work • Securing resources needed to deploy a range with the design team; of demand-driven data collection and analysis • Strengthening project design by using a approaches to utilize at various scales. strategic and proactive M&E strategy to Many important learning opportunities, both for inform the project PDO, components, and the Kenya CSA project and for resilience M&E activities; more broadly, will undoubtedly be offered over the • Engaging stakeholders in the design of the next several years as the project is implemented. project components, theory of change (results framework), and indicators; REFERENCES Heumesser, Christine. 2017. Personal communication. 17 May. Iskandarani, Maria and Wilkes, Andreas. 2016. UNIQUE forestry and land use GmbH. Working Paper: A conceptual approach for monitoring and measuring triple wins in climate-smart agriculture projects—the example of Kenya. Prepared for the World Bank. World Bank. 2017. Kenya Climate-Smart Agriculture Project. Washington, D.C.: World Bank Group. 10 WORLD BANK RESILIENCE M&E GOOD PRACTICE CASE STUDIES Mozambique PROIRRI Sustainable Irrigation Development −Water Global Practice Project­ COUNTRY CONTEXT A lthough Mozambique has experienced significant growth over the previous two decades—with an average annual real GDP growth rate of over seven percent—over half of the population (21.5 million) still lives below the poverty line. The prevalence of poverty is generally attributable to a failure to increase yields for smallholder farmers (farmers utilizing less than 10 hectares of land), who dominate 95 percent of the agricultural sector. Over three-quarters of the people in Mozambique depend on agriculture for their primary livelihood, yet only 10 percent of the arable land is currently being cultivated, and agricultural productivity is extremely low. Maize and rice yields are many times below regional and international levels. The majority of agricultural production is rain-fed, and as such rainfall patterns across agricultural seasons critically influence crop yields and affect production variability. Among African countries, Mozambique is the third most exposed to weather-related risks including periodic floods, cyclones, and droughts. Climate change is only expected to increase exposure to extreme weather. In terms of climatic stresses, drought conditions impact the largest number of people by far—more for lack of adequate irrigation infrastructure than lack of water availability. In fact, Mozambique Production of high value crops such as fruits and has abundant water resources, and vegetables in Mozambique requires greater control thus enormous potential to address over water. Source: Development Impact Evaluation drought stress by increasing its capacity Group (DIME) 11 12 WORLD BANK RESILIENCE M&E for irrigated agriculture. At present, only eight agricultural growth and rural development, percent of farmers have access to irrigation. the Government of Mozambique has made Recognizing that promoting sustainable irrigation the establishment of irrigation systems with and drainage is essential to strengthening participatory management through water user smallholder farmers’ resilience to intensifying groups a priority. climate variations, which should in turn stimulate MOZAMBIQUE SUSTAINABLE IRRIGATION DEVELOPMENT PROJECT (PROIRRI) PROIRRI, approved in 2011, seeks to support smallholders: EXHIBIT 3 • to use water for agriculture more efficiently, whereby minimizing producer dependency on PDO and Project Components rainfall patterns; PDO: To increase agricultural production • to improve and diversify their farming systems marketed and raise farm productivity in new or to mitigate their production risks; and improved irrigation schemes in the provinces of Sofala, Manica, and Zambezia. • to increase their yields and either produce a surplus that can be marketed to generate Project Components income (e.g. rice), or take market-led • Institutional Capacity Development and production planning decisions and dedicate Participatory Irrigation Management part of their production for a secured market • Investments in Irrigation Systems and outlet (e.g. outgrowers scheme). Support Infrastructure • Financial Support to Production and Value Exhibit 3 shows the PDO and project components. Chain Development • Project Management and Implementation Support GOOD PRACTICE CASE STUDIES 13 PROIRRI PROJECT RESILIENCE M&E The project design includes a results-based M&E seeks to fill a significant knowledge gap in this system aimed at measuring progress towards the respect. With multiple irrigation schemes offering PDO to allow for timely course corrections and the opportunity to explore diverse sites for evidence-based decision-making. experimentation, the project is well-suited for impact evaluation. Furthermore, willingness on Based on extensive consultations with M&E the part of project stakeholders to invest in a experts during the project design phase, emphasis learning agenda around measurement of water was placed on a rigorous impact evaluation of use and scheme management meant that alternative advisory messages and extension practices would be well-implemented and lessons mechanisms within the management of selected immediately adopted. Additional funding from schemes, with technical assistance provided under donors, including the UK Department for the World Bank’s Development Impact Evaluation/ International Development and the U.S. Agency Agricultural Adaptations and Natural Resource for International Development, also allows for Management (DIME-AADAPT) initiative. continuous innovation and experimentation during the impact evaluation process. DIME’s approach to impact evaluation is to apply randomized control trials to establish a counterfactual to an intervention. The GOOD PRACTICES | Use of impact evaluation counterfactual measures what would have designs to better understand the mechanisms happened to participants had the intervention not that drive resilience (impact); Securing resources taken place, and it is estimated using a for the evaluation comparison group that does not receive the intervention. DIME impact evaluations evaluate both the effectiveness of packages of The impact evaluation, with a budget of interventions (the “what”), and experiment with approximately US$1.35 million, is designed to mechanisms (the “why/how”), to better answer central operational questions, such as understand what drives impact. The latter is the when to scale-up extension service delivery focus of this case study, because the why and the mechanisms, how to measure farmers’ knowledge how provide clues about the adaptive capacity of and adoption of improved technology, and how beneficiaries as indicative of resilience. to assess the relative impacts of simultaneously occurring interventions. PROIRRI’s impact evaluation uses randomized control trials to GOOD PRACTICE | Building multiple M&E establish carefully identified control and treatment approaches into project design groups to generate statistically rigorous information on the impact of the program. It is not feasible, or appropriate, to conduct impact evaluations for all projects and GOOD PRACTICE | Choosing clear objectives interventions. In this case, however, there is a lack for the impact evaluation, including establishing of existing evidence on smallscale irrigation an evidence base where evidence is lacking development projects due to the difficulty of finding suitable comparison groups for irrigation interventions. PROIRRI’s impact evaluation thus 14 WORLD BANK RESILIENCE M&E The impact evaluation specifically intends to: • Gather evidence on which smallholder • Determine the extent to which measuring and irrigation interventions have the potential to reporting water use can improve equity and improve productivity; and efficiency of water allocation; • Build capacity within INIR, the Instituto • Inform protocols for low-cost, community- Nacional de Irrigação (National Institute for based water measurements that can be Irrigation), to monitor other schemes and implemented in schemes throughout larger scales. Mozambique; STATUS OF IMPLEMENTATION AND PRELIMINARY FINDINGS The impact evaluation has thus far employed three irrigation schemes to pilot methods of measuring water flow. Pilot testing and development of water measurement methods began in June 2015 and are ongoing as of June 2017. To date, a community monitoring system, with a set monitoring protocol and designated responsible community personnel, has been established. Source: DIME Pattern of availability does not seem to track requirements by EXHIBIT 4 growth stage (one of several impact evaluation preliminary results) 6 5 Av. water mm of water per day availability 4 Maize 3 Baby Corn Cabbage 2 1 1 2 3 4 Growth Stage Source: DIME GOOD PRACTICE CASE STUDIES 15 Preliminary results indicate that the pattern extension for alleviating resource of water availability does not seem to track overexploitation. By analyzing farmers’ responses requirements by growth stage (see Exhibit 4), to either general reminders about best practices, suggesting that people have plenty of water in or to feedback on their own water use, the impact early growth stages. However, those with the most evaluation will provide input onto the most water in the early growth stage seem to get lower effective design of extension programs that can yields. Preliminary results also show that conflict enhance resilience. over water happens in months when farmers feel they do not have enough water. GOOD PRACTICE | Utilizing the impact The second piloting stage began in November evaluation designs to inform and help adapt 2016. Aimed at adjusting program ongoing implementation of the program implementation based on evidence-based learning, it involves a feedback experiment in which half of the farmers receive reminders about Informed by the work of the impact evaluation water requirements for their main crops, and half completed thus far, PROIRRI is planning to of the farmers receive feedback on how much scale-up its measurement to additional schemes, water is measured in their field. The experiment and to build further capacity within INIR, including proposes to uncover the instrumental modes of offering financial literacy and matching grants. information provision and components of SUMMARY Building rigorous impact evaluation into program/ uncommon practice in the nascent field of project design requires both significant up-front resilience M&E, can be suitable and useful as an planning and recognizing the conditions that option for resilience-building operations. make the approach suitable and feasible. Early on in project design, the Mozambique PROIRRI Incorporating rigorous impact evaluation is not project made the case for pursuing an impact feasible or appropriate for all interventions. The evaluation, and secured the resources to do ReM&E project has published an Evaluation so. The impact evaluation has been carefully Guidance document that seeks to provide designed and is being carefully implemented. conceptual and operational guidance support It is informing and refining PROIRRI as it moves to improve the design of evaluations for forward, and is building a broader evidence resiliencebuilding projects/programs. The ReM&E base for design of irrigation schemes across project has also published an M&E guidance Mozambique and beyond. document to provide practical guidance for M&E of resilience results of World Bank operations, The final outcomes of this impact evaluation with the aim to improve the understanding of will not only inform agricultural resilience and design options for resilience-relevant operations. production in Mozambique, but will also provide Teams are encouraged to refer to these insights into how impact evaluation, a relatively complementary efforts of the ReM&E project. 16 WORLD BANK RESILIENCE M&E REFERENCES DIME. 2015. Impact Evaluation Concept Note: Group Interventions for Agricultural Transformation in Mozambique. DIME. 2016. Presentation: Impact Evaluation to improve impact: Sustainable irrigation in Mozambique. i2i DIME. 2015. What is Impact Evaluation? Available at: http://pubdocs.worldbank.org/en/630921467313243167/What-is-IE- 5-6-15-edited.pdf . World Bank. 2011. PROIRRI - Sustainable Irrigation Development Project. Washington, D.C.: World Bank Group. World Bank. 2016. Proposed Project Restructuring of the Mozambique Sustainable Irrigation Development Project. Report No: RES21715 World Bank: Washington, DC. GOOD PRACTICE CASE STUDIES Mekong Delta Integrated Climate Resilience and Sustainable Livelihoods Project COUNTRY CONTEXT V ietnam has experienced rapid and inclusive economic growth since the early 1990s, transforming it from one of the poorest countries in the world to a lower-middle- income country. The percentage of people living in extreme poverty dropped from around 50 percent in the 1990s to under three percent by 2015. Development of the agriculture sector, particularly in the Mekong Delta, has contributed significantly to the development of Vietnam. The Mekong Delta alone contributes to 50 percent of Vietnam’s rice (90 percent of which is for export), 70 percent of its aquaculture products, and a third of Vietnam’s GDP. The Mekong Delta is also home to 22 percent of Vietnam’s population, most of whom live in rural coastal areas and are highly dependent upon rice or shrimp farming for their livelihoods. Many of these households are “near poor” and are vulnerable to external shocks that can push them back below the poverty line. The Mekong Delta has been identified as one of world’s deltas most vulnerable to the impacts of climate change. In addition to increased pressures from the unsustainable use of land and water resources, economic growth of the Mekong Delta is challenged by climate change impacts in the form of increased saline intrusion in coastal areas, greater coast erosion, and higher levels of flooding. Already, Vietnam is experiencing wetter wet seasons, drier dry seasons, intensified rainfall, flash flooding, and more frequent tropical cyclones. Over time, poor and marginalized groups will incur the greatest burden from these and other climate change impacts. 17 18 WORLD BANK RESILIENCE M&E The Government of Vietnam recognizes the no tools or frameworks that allow delta planners threats and has started to develop a more to systematically assess the resilience of their holistic and spatially integrated vision to manage investment decisions against the breadth of current and future risks and opportunities in the potential change. Mekong Delta. However, at present, there are MEKONG DELTA INTEGRATED CLIMATE RESILIENCE AND SUSTAINABLE LIVELIHOODS PROJECT In response to these challenges, the World Bank EXHIBIT 5 Board approved the Mekong Delta Integrated Climate Resilience and Sustainable Livelihoods Project in 2016. The project aims to enhance tools for climate-smart planning and improve climate PDO and Project Components resilience of land and water management practices PDO: To enhance tools for climate-smart in selected provinces of the Mekong Delta. planning and improve climate resilience of land and water management practices in The PDO (see Exhibit 5) will be achieved through selected provinces of the Mekong Delta in the provision of capital investments (especially Vietnam. in water management infrastructures), technical assistance (related to agricultural and aquaculture Project Components livelihoods), and capacity building for farmers • Enhancing Monitoring, Analytics, and (in selected Mekong Delta provinces) and Information Systems government institutions (at national and sub- • Managing Floods in the Upper Delta national levels). • Adapting to Salinity Transitions in the Delta Estuary Project activities are estimated to directly benefit • Protecting Coastal Areas in the Delta over 1.2 million people living in nine provinces. Peninsula These include ethnic minorities and farmers • Project Management and Implementation (especially of rice) in the upper Delta provinces, Support and aquaculture farm and fisher folk households along the coastal provinces. The livelihoods of Upper Delta these groups are affected by climate change, Managing Floods salinity intrusion, coastal erosion, and flooding. The project will span a period of six years, with the financing of US$387 million.4 The project Peninsula has five Components, shown in Exhibit 5. Coastal Protection Component 1, “Enhancing Monitoring, Analytics, Estuary and Information Systems,” is specifically Adapting to Salinity designed to provide monitoring equipment and accompanying analyses and decision-support needs to better manage and contribute to 4 US$310 million from international development assistance (IDA) and US$77 million from the Government of Vietnam. The project is seeking an addi- tional US$6 million from the Global Environment Facility to finance research and innovation activities relating to climate-resilient solutions for the Delta. GOOD PRACTICE CASE STUDIES 19 climate-resilient investments for the Delta. In aquaculture models) designed to be climate parallel, Components 2, 3, and 4 work to scale resilient, and demonstrate how multi-sectoral up smaller, successful pilots (e.g., cropping, solutions can be implemented. MEKONG DELTA PROJECT RESILIENCE M&E The World Bank has previously deployed several considerations, assumptions, data needs, in order projects in the area, yet all have been based in to achieve both clarity and rigor (see Exhibit 6). specific sectors, and have operated in parallel “silos.” The Mekong Delta project is different: It squarely focuses on a complex hydro-ecological system with upstream and downstream linkages EXHIBIT 6 that require integrated solutions to build resilience to climate change and development Reflections from the TTL on the impacts. Resilience is not a secondary consideration or co-benefit for this project; it is tough, yet fruitful discussions that the core objective adopted by the World Bank’s informed the results framework project team through a system’s lens with careful “Our practice manager strongly encouraged planning and analysis. us to get to work early and concretely on designing an appropriate M&E foundation— one that reflected the multi-sectoral complexity GOOD PRACTICE | Creating a strong M&E of the project, and yet was measurable and foundation that was more than “checking the box” relevant. The World Bank Senior M&E Specialist we worked with was a tough but fair guy. There The Mekong Delta project’s M&E system consists was a lot of back and forth! The discussions of standard elements5 established through were extremely helpful in the end, and they extensive analysis and discussion that moved helped us deconstruct our project design beyond simply meeting minimum project and devise appropriate ways to measure and requirements. Project development also involved monitor project impacts and outputs. making the case for additional resources to The team spent a lot of time pouring over maps commission an M&E firm to provide technical and deliberated on the individual subprojects assistance throughout implementation. Although to discuss the context. These discussions also the project is currently in the preliminary stages helped us fine-tune our project design and of implementation, foundational M&E good improved the specificity of our subproject practices have already been undertaken. descriptions. A very useful feature of our final framework was providing definitions of terms The M&E design stemmed from discussions like ‘river bank’ and ‘coastline protection’—this between project TTLs, a World Bank Senior M&E enabled us to have a consistent interpretation of the precise indicators,” – TTL of Mekong Delta specialist, and other sector experts. This design Integrated Climate Resilience and Sustainable team held numerous discussions, some of which Livelihoods Project. were challenging, to sort through different 5 This includes the PDO, results framework with PDO indicators and interim outcome indicators, required monitoring and reporting systems, mid- term review, and final evaluation. 20 WORLD BANK RESILIENCE M&E The team also closely collaborated with the transformation by influencing long-term, climate- Government of Vietnam and reached out to other resilient, integrated master planning and piloting stakeholders and donors working on Mekong investments that demonstrate a balanced Delta initiatives. M&E discussions involved approach. identifying the PDO, project components/ subcomponents, and indicators (which collectively comprise the project results framework), as well as GOOD PRACTICE | Balancing indicator ambition with practicality emphasizing the need for additional M&E technical assistance to support monitoring throughout project implementation. The team also aimed to design an M&E framework that captures both accountability and GOOD PRACTICE | Embedding strong evidence-based learning, which will provide data resilience framing in project design for evidence-informed policy—and decision- making in the Mekong Delta. The project budget thus earmarked resources for a specialized M&E Unlike other relatively simplistic results consulting firm to monitor project progress and frameworks that rely on common, often generic, update monitoring indicators from participating sector-based measures (e.g., km of dykes built), the multi-sectoral team developed the results framework to reflect the spatial context of the planned sub-projects. These sub-projects comprised of integrated “packages” of water- and agriculture-related infrastructures as well as climate-resilient livelihood practices, articulated within a resilience framework. Component 1 focuses on enhancing water and salinity monitoring systems, providing decision-support, and developing an Integrated Master Plan for the Mekong Delta. Components 2-4 are spatially based to reflect the resilience framing, in contrast with traditionally sector-based components. GOOD PRACTICE | Engaging stakeholders and securing donor buy-in and collaboration upfront So as to not overpromise delivery, the team carefully considered what objectives the project could realistically achieve in the country context over a six-year period, keeping resource constraints in mind. The conversations crystalized a need to define simple and practical project objectives while still clearly aiming for transformational turning points. The TTL described how the approach sought to support GOOD PRACTICE CASE STUDIES 21 provinces. The project design also took into GOOD PRACTICES | Designing for learning; consideration global experience from delta Allocating resources to support evidence-based initiatives in other countries, building in learning learning, enhancing results and accountability from the beginning. Finally, the design team dedicated time and effort toward carefully documenting each indicator’s GOOD PRACTICE | Clearly defining indicators definition and measurement approach in the and providing guidance on measurement project appraisal document (see World Bank approaches 2016). The process leads to more transparent and robust indicators, and creates institutional memory that will be useful as project staff turnover occurs during the course of the six-year project. SUMMARY The Mekong Delta Integrated Climate Resilience • Balancing ambition with practicality; and Sustainable Livelihoods Project is a strong • Designing for learning; example of several good resilience M&E • Allocating resources to support evidence- practices, including: based learning, enhancing results and • Creating a strong M&E foundation that was accountability; and more than “checking the box;” • Clearly defining indicators and providing • Engaging stakeholders and securing donor guidance on measurement approaches. buy-in and collaboration upfront; • Embedding strong resilience framing in This case holds promise, and it will undoubtedly project design; offer numerous insights and lessons on resilience M&E over the next several years. REFERENCES Acharya, Anjali. 2017. Personal communication. 15 May. Jensen, Anders. 2017. Personal communication. 4 May. World Bank. 2016. Vietnam - Mekong Delta Integrated Climate Resilience and Sustainable Livelihoods Project. Washington, D.C.: World Bank Group. 22 WORLD BANK RESILIENCE M&E Annex 1. Mekong Delta Project Results Framework and Indicators Project Development Objectives (PDO): To enhance tools for climate-smart planning, and improve climate resilience of land and water management practices in selected provinces of the Mekong Delta in Vietnam. PDO Level Results Indicators • Adoption of Mekong Climate Resilience Assessment by MONRE (score) • Area with climate resilient land and water-management practices supported by the project (hectares) • Project supported farm households who have adopted climate resilient land and water management practices (percentage) • Direct project beneficiaries, percent of which female (number, percentage) • Citizens in selected provinces who participated in consultations on formulation of district land use plans (number) Intermediate Results Indicators Component 1: Enhancing Monitoring, Analytics, and Information Systems 1.1: Mekong Delta Center established and operational (yes/no) 1.2: Specialized Studies to facilitate climate resilient decision making supported by the project (number) 1.3: Monitoring stations established or upgraded through project support, and fully operational (number) Component 2: Managing Floods in the Upper Delta 2.1: Project supported farm households in selected provinces transitioned to third rice crop alternatives (percentage) 2.2: Flood retention areas with water management infrastructure supported by the project (hectares) 2.3: August dikes rehabilitated and operational supported by the project (kilometers) GOOD PRACTICE CASE STUDIES 23 2.4: August dike sluice gates constructed and operational supported by the project (number) Component 3: Adapting to Salinity Transitions in the Delta Estuary 3.1: Farm households in selected provinces who have transitioned to climate resilient alternatives livelihoods supported by the project disaggregated by: (a) Estuary Provinces; (b) Peninsula Provinces (percentage) 3.2: Brackish water aquaculture area with sustainable and climate resilient infrastructure supported by the project disaggregated by (a) Estuary Provinces; (b) Peninsula Provinces (hectares) 3.3: River bank and coast line protection supported by the project: Coastal dike; River bank; Embankment (kilometers) 3.4: River Bank and coastal sluicegates constructed and operational through project support (number) Component 4: Protecting Coastal Areas in the Delta Peninsula 4.1: Coastline protection through project support (kilometers) 4.2: Coastal sluice gates constructed and operational through project support (number) 4.3: River bank and coast line protection supported by the project: (a) Coastal dike; (b) River bank; (c) Embankment (kilometers) 24 WORLD BANK RESILIENCE M&E CLIMATE CHANGE | SUSTAINABLE DEVELOPMENT