Approach Paper Ethiopia Country Program Evaluation November 30, 2023 1. Background and Context 1.1 The objective of this Country Program Evaluation (CPE) is to assess how well the World Bank Group supported Ethiopia in addressing key challenges that constrained its development and how that support adapted over time to respond to changing circumstances, an evolving relationship, and lessons from experience. The evaluation will cover fiscal years (FY)13–23. The time period is selected to include the last two Bank Group strategies to support Ethiopia and coincides with the period of the previous two political administrations. The evaluation aims to inform the next Bank Group–supported Country Partnership Framework (CPF) for Ethiopia, expected in FY25. 2. Political, Economic, and Social Context 2.1 With a population of over 115 million people as of 2020, Ethiopia is the second most populous nation in Africa and one of the most diverse. It hosts about 86 ethnic groups and 90 spoken languages and is religiously and geographically diverse. More than 80 percent of the population lives in rural areas. A landlocked country at the center of the Horn of Africa, it borders on six mostly fragile countries: Djibouti, Eritrea, Kenya, Somalia, South Sudan, and Sudan. 2.2 Following the overthrow of the military regime in 1991, the Ethiopian People’s Revolutionary Democratic Front, a coalition of ethnic-based parties (dominated by the Tigray People’s Liberation Front) introduced a federal structure primarily on ethnic lines. The system devolved powers and mandates to regional states and to the woreda (district) and kebele (village) levels. The Economist Intelligence Unit Democracy Index has classified Ethiopia as an authoritarian regime throughout the evaluation period. Elections results in 2005 and 2015 were contested. Between 2014 and 2018, protests multiplied throughout the country, which led to the resignation of Prime Minister Hailemariam Desalegn in 2018. Abiy Ahmed Ali subsequently emerged as the prime minister, marking a shift in power within the Ethiopian People’s Revolutionary Democratic Front toward the Oromo ethnic group. Since 2018, the administration implemented a wave of domestically and internationally popular reforms liberalizing political, civil, and press rights. Abiy Ahmed Ali also negotiated a peace accord with Eritrea after a decades-long border conflict, earning him the Nobel Peace Prize in 2019. 2.3 Ethiopia faces multiple drivers of fragility that resurfaced in recent years as violent conflict episodes have increased since 2017 (World Bank 2022b). These include 1 disagreement over the most appropriate federal arrangement; competition for land, territory, and resources; fragmentation in the security sector that has led to the emergence of ethnic-based militias; and inequality, perceptions of inequitable development, and exclusion from political power. Although some of these have long histories (for example, as a result of control over local resources), tensions have been exacerbated during the political transition because of the intensification of political competition and rivalry between elites at the local, national, and regional levels. Underlying tensions within the political transition exacerbated by the postponement of elections because of the COVID-19 pandemic led to the outbreak of the conflict in the north in November 2020, which began in Tigray and spread to Amhara and Afar regions and became a severe humanitarian crisis until a peace accord was signed in November 2022. However, the political situation remains fragile, with frequent violent unrest. The country has been on the Bank Group’s list of countries affected by fragility, conflict, and violence since 2022. Ethiopia has historically hosted one of the largest refugee populations in the world, and internal displacement has only increased with the conflict in the north. 2.4 Ethiopia has been one of the fastest growing economies in the world since 2004. Growth has averaged 8.5 percent since 2012, and per capita output has doubled from US$459 in 2012 to US$925 in 2021. The country has pursued a development state model that promoted high public investment in infrastructure under two Growth and Transformation Plans (GTPs): GTP I from 2009–10 to 2014–15 and GTP II from 2015–16 to 2019–20. This was combined with heterodox policies, including financial repression that kept interest rates low and directed the bulk of credit to public infrastructure, an overvalued exchange rate, and monetary financing of the deficit that has contributed to high inflation. These policies distorted the allocation of factors of production and the behavior of economic actors, which contributed to growing macroeconomic imbalances. 2.5 Yet Ethiopia’s high growth rates have been declining, and structural transformation has stalled. Growth averaged 11.4 percent from 2003 to 2010, 10.2 percent from 2011 to 2015, and 8.5 percent from 2016 to 2019 before falling to 5.6 percent for 2020 and 2021 during the COVID-19 crisis. Over the past decade, services and construction have driven growth; the contribution of agriculture to GDP growth has declined since 2015; and the contribution of manufacturing to GDP growth has remained modest, particularly since 2017, whereas productivity has declined. Scarce foreign currency, combined with financial repression that limits credit for private sector investment, has inhibited private sector and export growth. Historically, the state and ruling party have played an active role in the economy and state-owned enterprises. Additional constraints to competitiveness include trade and customs regulations, burdensome regulations and administrative procedures (entry barriers to starting a business), access 2 to reliable energy and land, trade logistics, and skills shortages. The road network quadrupled from 1997 to 2015, but road density remains the lowest in Africa. 2.6 With low domestic savings rates and weak domestic resource mobilization, high public investment has been largely financed by external borrowing by the government and state-owned enterprises. In recent years, the COVID-19 pandemic and the conflict in the north have exacerbated fiscal challenges. Thus, although Ethiopia had benefited greatly from debt relief through the Heavily Indebted Poor Countries Initiative in the 2000s, the country’s risk of debt distress has gradually increased from low to moderate risk in 2015 and reached high risk in 2018. In 2021, it requested debt reprofiling under the Common Framework for Debt Treatments. 2.7 Following the political transition in 2018, and recognizing the shortcomings of the state-led development model amid slowing growth and widening economic imbalances, the new administration launched the Homegrown Economic Reform agenda in 2019. The Homegrown Economic Reform included reforms across three pillars, which aimed at addressing the core macro-financial instability, introduced structural reforms to rebalance the roles of the public and private sectors, and promoted reforms to unlock growth potential across five sectors. 2.8 The country is highly vulnerable to climate change and has experienced severe weather events (including droughts) over the past years that have contributed to adverse impacts on livelihoods. Ethiopia is currently experiencing severe and prolonged drought after almost six consecutive years of failed rainy seasons. Rain-fed agriculture remains the dominant source of income, increasing vulnerability to climate change and resulting in frequent droughts. Moreover, a potential simultaneous increase in floods poses a serious water pollution threat, affecting the health of wetland and forest ecosystems. Pastoral communities in the fragile, semiarid lowlands are also prone to climatic shocks. Volatility constrains investment in profitable crops and technologies. Weak natural resource management, insufficient availability of irrigation, and the need for safety nets drive vulnerability to climate shocks. The depletion of forests has been contributing to increased water stress, and investments in sustainable land management in recent years have helped reverse these trends. 2.9 Ethiopia has made remarkable progress in poverty reduction in the last two decades, but it has stalled in recent years because of conflict, drought, and higher inflation. The national poverty head count ratio declined from 38.7 percent in 2004 to 29.6 percent in 2011 and to 23.5 percent in 2015 (World Bank 2023). Although urban poverty nearly halved from 2011 to 2016 (from 25.7 percent to 14.8 percent), the decline was less dramatic in rural areas, where it fell from 30.4 percent to 25.6 percent (World Bank 2022c). 3 2.10 Ethiopia has also made considerable progress in human capital outcomes, though from a low base. Achievements include meeting the Millennium Development Goal on child mortality (which halved), primary school enrollment (which quadrupled), and access to clean water (which doubled). Primary school enrollment has increased from 66 percent in 2000 to 91 percent in 2019. From 2005 to 2019, under-five child mortality decreased from 123 to 55 deaths per 1,000 births, whereas the school population more than tripled. Ethiopia’s Global Gender Gap Index score was 0.66 in 2016, indicating that females in the country were 34 percent less likely to have the same opportunities as males, and the score improved slightly to 0.71 in 2022. 2.11 Even though Ethiopia has had one of the strongest improvements globally in Human Development Index scores in the 2000s, the country’s overall score reflected shortcomings in primary health care, educational attainment, gender gap, and poverty prevalence. The Human Development Index score improved from 0.435 in 2013 to 0.498 by 2021, but it was still in the low category (UNDP 2022). Ethiopia’s Human Capital Index score is 0.38, ranking at 156 out of 173 countries. This indicates that a child born in the country today will achieve only 38 percent of their development potential. The Human Capital Index is affected primarily by childhood stunting (which has reduced considerably but remains high) and high illiteracy, despite improvements in primary school enrollment rates. 3. Key Development Constraints 3.1 Despite considerable progress in the last two decades, Ethiopia faced several major development constraints at the start of and during much of the evaluation period. A review of World Bank and external diagnostic and analytical work highlights the following: • An increasingly unsustainable development model and increased macroeconomic vulnerabilities: The government’s development model allocated the bulk of domestic bank credit to public investment while rapidly building up external debt. Limited credit for private investment affected domestic firm growth especially because of lack of access to foreign sources of credit. An overvalued exchange rate worsened prospects for exporters, with increasingly severe foreign exchange shortages restraining access to imports needed to expand manufacturing and exports. The government is now heavily constrained in further borrowing, whereas the private sector has seen only limited transformation. • Weak business environment and competitiveness: A challenging business environment has constrained the development of the private sector and 4 structural transformation. In addition to the overvalued exchange rate, foreign exchange shortages, and limited private sector credit, other business environment challenges hampered the ability to develop the domestic private sector and attract foreign investment. Moreover, poorly developed rural road infrastructure was a major constraint on market access for farmers, raising the cost of market access for purchasing inputs and selling produce, thereby reducing incentives for agricultural investment and opportunities for rural nonfarm income growth. • Lack of resilience to drought and climate shocks: Seasonal rainfall is volatile in much of the country, and vulnerability to drought is high, which is only expected to increase with climate change. Irrigation has the potential to reduce rainfall vulnerability; however, less than 1 percent of smallholder-cultivated land was irrigated at the start of the evaluation period. Ethiopia’s energy needs are dependent on renewable resources, such as hydropower, which is vulnerable to drought. Water scarcity has led to internal displacement of people and may escalate disputes between Ethiopia and its neighboring countries. Climate shocks continue to affect poverty and vulnerability, and their impacts will only increase over time. • Conflict and fragility: Conflict drivers include both old and new social and political grievances, local and national politics, citizen neglect and frustration, climate change and land degradation, and rural-urban migration, with a history of conflicts being framed in ethnic terms. Conflict affected incomes, economic growth, fiscal policy, access to services, development outcomes, and internal migration. • Low human development: Ethiopia has improved human capital outcomes because of high levels of public investment in education and health, but it started from a low base, and outcomes remain low. Inequities in availability of services, lack of quality services, and constraints to household investments contribute to low education and health outcomes. Unsafe water use in rural areas impedes good health and nutrition and is an impediment to improving agricultural productivity among women. 4. Evolution of World Bank Group Strategy 4.1 Two key documents represented the backbone of the Bank Group’s strategic approach in Ethiopia during the FY13–23 period: the Bank Group Country Partnership Strategy (CPS) for FY13–16 and the CPF for FY18–22. The FY13–16 CPS supported the government of Ethiopia’s GTP I (from 2009–10 to 2014–15), which aimed to extricate the 5 country from poverty to reach middle-income status by 2023 through strong economic growth within a stable macroeconomic framework, driven by agriculture and industry, quality infrastructure and social services (particularly health and education), and good governance. 4.2 The Bank Group FY13–16 CPS acknowledged shortcomings in the government’s developmental state model. Although the model had shown significant results and high growth over the previous decade through high levels of public investment, the CPS warned that this may not be sufficient to attain the goals of GTP I (World Bank 2012). Instead, the government would need to complement its own financing with additional domestic and foreign investment and increase the formal private sector’s share in the economy. Overreliance on public sector investment could increase external vulnerabilities, whereas high fiscal deficits could crowd out private investment further or contribute to increasing inflation if they are monetized. 4.3 The CPS supported GTP I through two pillars: (i) fostering competitiveness and employment, and (ii) enhanced resilience and reduced vulnerabilities, with a foundation of good governance and state building (figure 4.1). Objectives of the first pillar included (i) a stable macroeconomic environment; (ii) increased competitiveness and productivity, particularly in agriculture, manufacturing, and services, and improved access to finance; (iii) increased and improved delivery of infrastructure; and (iv) enhanced regional integration. As part of the first pillar, the International Finance Corporation (IFC) strategy focused on de-risking the business environment to attract private investment by supporting three areas: key strategic sectors (for example, agribusiness, infrastructure, financial services, tourism, and manufacturing); investment climate; and entrepreneurial capacity. The second pillar’s objectives were to support Ethiopia in improving the delivery of social services and developing a comprehensive approach to social protection and risk management. In addition, gender and climate change were included as cross-cutting issues. 4.4 GTP II (from 2015–16 to 2019–20) built on the vision of GTP I and updated the overall goal of attaining lower-middle-income status to 2025. Although agriculture was still expected to be the main source of growth, GTP II put more emphasis on the private sector and manufacturing specifically as an engine of growth. The promotion of rapid industrialization was to play a key role in structural transformation to create jobs and improve income, promote exports, improve competitiveness, and enhance technological capability and skills development. 4.5 The FY18–22 CPF was formulated to support GTP II and was informed by the 2016 Systematic Country Diagnostic. The Systematic Country Diagnostic detailed how the country’s unorthodox strategy faced considerable challenges to sustainable financing, 6 and many aspects of it undermined the domestic private sector, export growth, financial intermediation, and external debt sustainability. It highlighted how “a change in [growth] strategy will be needed before the net benefits of the current strategy turn negative” (World Bank 2016, x). It also underscored how the government needed stronger feedback mechanisms, particularly to transition toward a more private sector– oriented growth strategy. Beyond these two key challenges, most of the binding constraints addressed in the CPF were already being supported in the earlier strategy. Although supporting a stable macroeconomic environment was a key objective in the first strategy, the second strategy broke this down into various subobjectives, including improving exchange rate policy, supporting more sustainable financing, and improved revenue mobilization. 4.6 During the latter half of the first CPS period, social unrest highlighted spatial disparities in resource allocation and economic opportunities. This unrest subsequently informed the spatial lens of the FY18–22 CPF, which was intended to support more inclusive growth and greater social inclusion. It aimed at achieving spatial development outcomes through interventions in three areas: fostering productivity in the agricultural sector, expanding infrastructure services to increase access to markets and services, and strengthening spatially neutral institutions that provide services of equal quality to populations in all areas of the country. 4.7 Following the onset of the conflict in the north in November 2020, the CPF was adjusted in FY22 toward what was termed a more people-centric approach. The approach focused on delivery of basic services, food security, and addressing the needs of vulnerable populations (especially internally displaced people and women, including survivors of gender-based violence) while retaining a long-term development focus. This included a halt to budget support operations and a change in implementation agreements, providing for third-party implementation and third-party monitoring in conflict areas for new operations approved from FY22. 7 Figure 4.1. Progression of World Bank Group’s Strategic Priorities and Systematic Country Diagnostic Constraints, Fiscal Years 2013–22 Sources: World Bank 2012, 2016, 2017a. Note: MSE = micro and small enterprise; MSME = micro, small, and medium enterprise. 8 5. World Bank Group Portfolio in Ethiopia 5.1 Bank Group financing to Ethiopia increased significantly in the second part of the evaluation period, supported by the approval of several large Programs-for-Results and development policy operations (figure 5.1). Bank Group financing totaled US$23.6 billion over the evaluation period.1 The International Development Association portfolio included 31 active projects (US$4.6 billion) at the beginning of the evaluation period, and 57 projects (US$16.6 billion) were approved during the evaluation period. The majority of lending approved was in the form of investment project financing (US$10 billion), followed by Programs-for-Results (US$4 billion), which expanded in the second strategy period. Budget support operations were approved only in 2019 and 2020 for a total value of US$2 billion (for three projects). Ethiopia benefited from 25 IFC investments (US$409 million) and eight Multilateral Investment Guarantee Agency guarantees (US$125 million). The total value of World Bank advisory services and analytics reached US$730 million, with a significant increase in the second part of the evaluation period. Figure 5.1. World Bank Financial Commitments to Ethiopia, Fiscal Years 2013–22 a. First strategy period b. Second strategy period 3,500 Financial commitments (US$, 3,500 3,000 Financial commitments (US$, 3,000 2,500 2,500 2,000 millions) 2,000 millions) 1,500 1,500 1,000 1,000 500 500 0 0 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 Year Year IPF Program-for-Results DPF IPF Program-for-Results DPF Source: Independent Evaluation Group. Note: DPF = development policy financing; IPF = investment project financing. 5.2 The largest sectors receiving World Bank financing were social protection, public administration, and agriculture, amounting to almost half of total investment project financing during the evaluation period (figure 5.2). New commitments increased in the second strategy period—particularly for public administration and industry and trade— as a result of budget support operations. Agriculture commitments also increased significantly with additional Program-for-Results financing. New transportation commitments fell to zero after being the second-largest sector in the first strategy period, 9 and water and sanitation commitments also fell considerably. Advisory services and analytics covered social protection, public administration, and energy. Figure 5.2. World Bank Commitments to Ethiopia by Sector and Strategy Period a. First strategy period (fiscal years 2013–17) Water, sanitation, and waste Transportation Social protection Public administration Information and communication Industry, trade, and finance Health Energy and extractives Education Agriculture 0 200 400 600 800 1,000 1,200 1,400 1,600 1,800 Financial commitments (US$, millions) IPF Program-for-Results DPF b. Second strategy period (fiscal years 2018–22) Water, sanitation, and waste Transportation Social protection Public administration Information and communication Industry, trade, and finance Health Energy and extractives Education Agriculture 0 200 400 600 800 1,000 1,200 1,400 1,600 1,800 Financial commitments (US$, millions) IPF Program-for-Results DPF Source: Independent Evaluation Group. Note: DPF = development policy financing; IPF = investment project financing. 10 5.3 IFC investments totaling US$285 million were approved mostly in the first strategy period and supported the financial and agriculture sectors (figure 5.3). IFC advisory main areas were manufacturing, regional advisory, and public-private partnerships. The Multilateral Investment Guarantee Agency entered the evaluation period with US$16.8 million of gross exposure and added US$108 million during the evaluation period, offset by US$6.8 million in cancellations (because of change in client corporate strategies and other reasons) and U$4.6 million in contract expirations. The gross exposure over the period included US$67.5 million in infrastructure, US$39.7 million in manufacturing, and US$6.5 million in agriculture. Figure 5.3. International Finance Corporation Investments in Ethiopia by Industry and Strategy Period, Fiscal Years 2013–22 250 200 150 Investments (US$, millions) 100 50 0 First evaluation period Second evaluation period Source: Independent Evaluation Group. 5.4 Budget support was approved only after the political transition in 2018, when a new administration opened a window for increasing reform momentum. In 2018, the World Bank approved the first budget support operation to Ethiopia in 10 years: the Growth and Competitiveness development policy financing, a US$1.2 billion programmatic series of four budget support operations. The operation was structured in 11 relation to three strategic pillars to maximize finance for development, improve the investment climate and financial sector, and enhance transparency and accountability. Prior actions promoted public-private partnerships; improved competition and efficiency in the power, logistics, and telecommunications sectors; streamlined business regulations and reduced constraints to accessing credit; and improved state-owned enterprise management (figure 5.4). In 2019, a subsequent development policy financing for US$500 million was approved with an additional US$250 million in supplemental financing because of the unanticipated impact of COVID-19. Figure 5.4. Development Policy Operations Prior Actions by Theme, Fiscal Years 2019– 20 6% Private sector development 11% Public sector management 33% Economic policy 11% Environment and natural resource management Finance 17% Social development and 22% protection Source: Independent Evaluation Group. 5.5 The overall outcome of most investment projects approved during the evaluation period was rated moderately or fully satisfactory. One of the 11 projects validated by the Independent Evaluation Group (IEG) received a moderately unsatisfactory rating, 4 projects were rated moderately satisfactory, and 6 projects were rated satisfactory. On Bank performance, 1 project received a rating of unsatisfactory, 6 projects were rated moderately satisfactory, and 4 projects were rated satisfactory (figure 5.5). In 2017, IEG conducted the Completion and Learning Review Validation of the FY13–16 CPS and rated the development outcome as moderately satisfactory and Bank performance as good.2 The FY08–12 CPS received the same ratings. 12 Figure 5.5. Independent Evaluation Group Outcome and Bank Performance Ratings for Projects Approved in Fiscal Years 2013–22 100% Share of projects approved (%) 80% Satisfactory 60% Moderately satisfactory 40% Moderately unsatisfactory Unsatisfactory 20% 0% Outcome rating Bank performance Source: Independent Evaluation Group. 6. Purpose and Evaluation Questions 6.1 This evaluation will focus on three evaluation questions, with the first covering the full portfolio and the second and third assessing the effectiveness of support for different special themes. The first evaluation question is standard to all CPEs and entails an assessment of (i) the relevance and coherence of the Bank Group–supported strategy to conditions and major development constraints at the start of the evaluation period, (ii) the adaptation of the strategy over the evaluation period in response to changing circumstances and priorities and learning from experience, and (iii) implementation of the strategy and portfolio. Subsequent evaluation questions address cross-cutting themes relevant to Ethiopia. The first pertains to the effectiveness of Bank Group support in helping Ethiopia transition from a state-led development model to one with a greater role for the private sector. The second seeks to determine the extent to which the Bank Group helped Ethiopia build resilience to climate change, particularly with respect to its support to agriculture and land and water management. We identified these themes through a review of World Bank and external analyses and experience, preliminary interviews with senior Bank Group staff familiar with Ethiopia, and discussion with a range of stakeholders in Ethiopia during a scoping mission to Addis Ababa in June 2023. Based on this, the CPE will address three evaluation questions: 1. How relevant was Bank Group support to addressing the main development needs of Ethiopia, and how well did it adapt over the evaluation period to respond to changing priorities, evolving country context, and learning from experience? 13 2. How relevant and effective was Bank Group support in helping Ethiopia transition away from a state-led development model to one with a greater role for the private sector? 3. To what extent did the Bank Group help Ethiopia build resilience to climate change, particularly with respect to its support to agriculture and land and water management? 6.2 Question 1 will assess how well the Bank Group adapted its strategy and operations to support Ethiopia over the evaluation period, including after the 2018 change in administration, during the COVID-19 pandemic, and over the conflict. In doing so, it will consider the adaptive relevance and coherence of how well the Bank Group identified and addressed development constraints through its strategies of support, how the portfolio evolved over time in relation to CPFs and strategies, and how it reflected geospatial considerations as possible. It will also assess how effectively World Bank instruments were used and how well Bank Group engagement adapted to the dynamic context of increasing fragility (including because of the onset of the COVID-19 pandemic and the conflict in the north). 6.3 Question 2 will assess the relevance and effectiveness of the Bank Group support in helping Ethiopia transition away from a state-led development model to one characterized by a greater role for the private sector. The limitations of a development model characterized by state-led growth became apparent over the evaluation period. Acknowledging that Ethiopia needed to transition to a more sustainable development model that created more jobs, the Bank Group stepped up its support to Ethiopia’s productive sectors (for example, agriculture, manufacturing, and services) with the aim of increasing competitiveness. Both Bank Group–supported strategies in place over the evaluation period sought to increase productivity in agriculture and employment and output in manufacturing and services through support to improve access to finance, markets, and critical infrastructure (that is, transportation).3 The World Bank made commitments of approximately US$4 billion during the evaluation period to support activities related to the productive sector. Within agriculture, the World Bank focused on land tenure, skills development, access to input and output markets, and green infrastructure. In the manufacturing sector, interventions supported by the World Bank were related to improving access to finance, industrial infrastructure, access to markets, and regulatory environment (including establishing an exchange rate market). Question 2 will seek to assess the adaptive relevance, coherence, and effectiveness of this support, including in the context of policy dialogue, analytical work, and project and operational interventions to support private sector growth. In doing so, it will seek to assess three subquestions: 14 • How relevant was the Bank Group’s approach to support structural transformation through agricultural productivity, manufacturing exports, and services, given Ethiopia’s binding constraints identified through analytical and diagnostic work, including political changes and shocks (such as foreign exchange)? • How effective has the Bank Group’s engagement been in increasing agricultural productivity by focusing on capacity development, land management, and access to input and output markets (such as transportation and green infrastructure)? • How effective has the Bank Group’s engagement been in supporting the manufacturing and services sectors through access to finance, improvements in the regulatory environment, reforms to state-owned enterprises, and the establishment of industrial zones? 6.4 Question 3 will assess to what extent the Bank Group helped Ethiopia build resilience to climate change, particularly with respect to its support to agriculture and land and water management. Ethiopia is vulnerable to climate variability and climate shocks related to hydrometeorological events, such as droughts and floods, which exacerbate poverty and stoke conflict. With climate shocks only expected to increase, Ethiopia will need to strengthen the transition toward increased climate resilience by developing social, economic, and environmental systems that can more successfully cope with and manage impacts of climate change trends and shocks while preventing such impacts from becoming worse, and by maintaining the capacity for adaptation, learning, and transformation. Climate change challenges will affect several key sectors of the economy negatively, particularly agriculture, which is estimated to lose more than 6 percent of its annual agricultural output if the current decline in average annual rainfall for primary agricultural zones continues until midcentury (World Bank 2022c). Furthermore, impacts are expected on the availability of drinking water and management of natural resources, such as forests and energy (including hydropower, which provides about 90 percent of Ethiopia’s power supply). The country’s shifting rainfall patterns will lead to greater incidence of floods and landslides, which would increase soil erosion and reduce arable land. This will increase the vulnerability of poor rural households that depend mainly on agriculture. 6.5 The Bank Group has been an active partner supporting Ethiopia’s efforts to adapt to climate change. During the two strategy periods, the World Bank helped the country in building the resilience of vulnerable households to food insecurity, adopting disaster risk management systems, and increasing agricultural and land productivity and practices that are important in building resilience to climate change. The productive safety net projects provided cash transfers to chronically food-insecure households, built 15 community assets, and enhanced access to complementary livelihoods services for crop and livestock production, off-farm income-generating activities, and employment links. The pastoral community development projects strengthened participatory planning at the district and community levels and invested in infrastructure, such as feeder roads, water supply, and early-warning systems through disaster preparedness and early- response grants. Furthermore, the agriculture and water and irrigation projects invested in agricultural research and extension and irrigation and water management, including flood control, to enhance productivity and increase market access through value chain development, capacity building of farmers, market infrastructure, and so on. In addition, a series of sustainable land management projects supported the country’s watershed and landscape development through land and water conservation, afforestation and reforestation, rehabilitation of degraded areas, and protection of ecologically critical ecosystems—all critical areas in building resilience to climate change. 6.6 Question 3 will evaluate the adaptive relevance, coherence, and effectiveness of the Bank Group’s support in building resilience through climate actions in key areas— agriculture, land, and water—where climate change impacts are particularly high. Building resilience requires the ability of the country to effectively adapt, cope, and recover from climate-related shocks. Among dimensions to be considered, the CPE will consider the appropriateness of the balance of Bank Group interventions between short- term crisis response and long-term resilience building. It will also consider how well the Bank Group supported programs in building climate resilience and if key lessons drawn from earlier engagements were integrated into subsequent phases of the projects. Under question 3, the CPE will seek to assess the following subquestions: • How effective was the Bank Group in reducing land degradation and building resilience through sustainable land and water management? • How relevant and effective was the Bank Group in building resilience to climate risks (for example, droughts and floods) through adoption of climate-smart agricultural practices, including in improving access to irrigation and drainage services? • How relevant and effective was the Bank Group in building safety nets to reduce vulnerabilities and enhance resilience in rural areas? 7. Evaluation Design 7.1 The evaluation will implement a theory-driven approach using a mixed methods design that will support the triangulation of findings from multiple sources of evidence. We added a member of the IEG methods team to the evaluation team to help further develop the most appropriate and rigorous methods and to ensure that they are 16 continually applied and adapted. Two criteria will apply to all three evaluation questions: (i) adaptive relevance of the World Bank’s engagement considering long-term challenges, shocks, and changing circumstances; and (ii) coherence of the World Bank’s activities, including coordination with development partners. For evaluation questions 2 and 3, the CPE will also assess effectiveness by measuring the results achieved in light of the magnitude of challenges and tracing contributions to country outcomes along main plausible direct or indirect pathways. The team will assess the credibility of the theory of change on which the World Bank strategy and its interventions are based. 7.2 The report will map Bank Group strategies and support to the relevant results chain to assess the credibility of what the Bank Group set out to do relative to the underlying theory of change. For the first evaluation question, assessing relevance will involve mapping the development needs and constraints identified in analytical and diagnostic work on Ethiopia to the FY13–16 CPS and FY18–22 CPF and to how the strategies were implemented and adapted to changing circumstances. The team will devote attention to assessing the extent to which a fragility lens was applied (based on analysis available at the time) in adapting Bank Group engagement in the lead-up to and during the conflict in the north. To establish the relevance of strategy and operational design and objectives, we will perform content analysis of advisory services and analytics; project, supervision, and (self-)evaluation documents; and other relevant evaluative evidence. Semistructured interviews with key stakeholders will help triangulate findings. The team will select interview partners based on a mapping of relevant stakeholders, with a focus on the inclusion of relevant interest groups. The following summarizes the methodological approaches to be followed for the three evaluation questions: • Review of existing evaluative and analytical evidence. The CPE will collect and review internal and external existing evaluative evidence, including IEG reports such as Implementation Completion and Results Report Reviews, Project Performance Assessment Reports, and case studies from thematic evaluations (for example, Toward Productive, Inclusive, and Sustainable Farms and Agribusiness Firms: An Evaluation of the World Bank Group’s Support for the Development of Agrifood Economies (2010–20); The Natural Resource Degradation and Vulnerability Nexus: An Evaluation of the World Bank’s Support for Sustainable and Inclusive Natural Resource Management (2009–19); and World Bank Group Support in Situations Involving Conflict-Induced Displacement), to inform its assessment of the Bank Group’s support to Ethiopia. In addition, the team will review available analytical work from the Bank Group and external stakeholders, including from academia, other development partners, and nongovernmental organizations to establish the relevance of Bank Group support, the extent to which relevant 17 literature highlights the potential risks associated with the Bank Group support, and insights on the content and context of the Bank Group support to Ethiopia. • Review of internal and external indicators and data sets. The evaluation will collect relevant indicators of outcome and Bank Group performance from project documents and external databases (such as agriculture maize productivity, manufacturing output, and manufacturing exports) to understand the underlying country context, its evolution over time, and the contribution of Bank Group support to that evolution. Examples of such databases include Food and Agriculture Organization Statistics, Enterprise Surveys, the Worldwide Governance Indicators, the World Development Indicators, and the Central Statistical Agency of Ethiopia. • Portfolio review and analysis. The evaluation will conduct a review of the lending and nonlending portfolio in Ethiopia active between FY13 and FY23, using a custom-designed protocol that will include variables of interest (for example, interventions, instruments, activities, and implementation problems) and analyze the information gathered to answer evaluation questions. The analysis will identify the key characteristics of Bank Group support, such as the type of activities supported and patterns and trends across time and location. It will subsequently identify project-level results and drivers of success and failure, with the aim of understanding patterns within the portfolio. Documents we intend to review include Project Appraisal Documents and program documents, meeting minutes, aide-mémoire, Expanded Project Supervision Reports, and Project Completion Reports. • Semistructured interviews. We will conduct semistructured interviews with Bank Group management and staff, government officials, development partners, relevant academics, civil society, and beneficiaries to collect qualitative information and identify lessons from experience, applying a template with questions or topics as appropriate. Issues we will cover include the rationale, nature, and extent of Bank Group support; the roles of the different Bank Group institutions; the adequacy of the Bank Group’s assessment of Ethiopia’s development challenges; complementarity with other activities; coordination with development partners and counterpart agencies; and views on the efficacy of Bank Group support. • Fragility analysis. Although the World Bank formally added Ethiopia to the list of countries affected by fragility, conflict, and violence only in 2022, drivers of fragility have been long-standing. Particularly for evaluation question 1, the CPE will include a fragility lens to assess how the World Bank’s understanding of the 18 underlying sources of conflict and fragility evolved over the evaluation period and how the World Bank adjusted its program in response. This will be informed by the 2022 Risk and Resilience Assessment and earlier analyses to understand when various issues were known and understood, including the evolving relationship between the central state and regions and how this affected programming. • Geospatial analysis. Depending on data availability, we will conduct a geospatial analysis, in close consultation with the methods team, to inform the assessment of the Bank Group’s contributions to support intraregional and interdistrict (woreda) disparities in access to services and their relevance to the drivers of fragility, largely as part of addressing evaluation question 1 and part of question 3 to assess outcomes on land restorations and land productivity. This analysis will seek to geolocate investment project implementation sites at the woreda level. For projects without geospatial data, the evaluation will attempt to infer coordinates from appraisal-stage project documents. We will seek this information from the relevant national statistical agency, along with supplementary information available from remotely sensed data and other global information sources. • Field and project site visits. Based on the mapping of project implementation sites, for evaluation question 3, we will use a purposive sampling to select key regions and woredas for the evaluation team to visit. At the field level, we will conduct semistructured group interviews with selected communities, community members, and local-level institutions (woreda offices, farmer groups, and committees). Additionally, we will make site visits to assess the investments the World Bank made in local-level physical infrastructure related to agriculture, land, and water management. • Resilience framework. The World Bank prepared a guidance note, Operational Guidance for Monitoring and Evaluation (M&E) in Climate and Disaster Resilience- Building Operations, based on an existing and well-accepted framework of resilience (World Bank 2017b). We will also use this framework to map the Bank Group portfolio for evaluation question 3 to assess interventions that have been prioritized or may require greater attention in building climate resilience. 8. Limitations of the Methodology 8.1 The evaluation will face several limitations. Ethiopia has a history of minimal private sector and civil society discussion on government policy directions, which may hinder the openness with which some stakeholders may be willing to discuss current 19 and past government policies and performance, including those of the Bank Group. Data limitations will exacerbate this—some basic economic data are missing through the evaluation period (including on poverty, for which the most recent data are from 2015) and some data during the conflict period. Given the conflict situation that has restricted World Bank staff travel to various parts of the country, the evaluation team may not be able to access all relevant areas of Bank Group activity outside of Addis Ababa to assess impact. Furthermore, the turnover of key government officials, Bank Group management and staff, and development partners, especially during the early part of the review period, will create challenges in reaching all relevant stakeholders. The team will seek to triangulate sources of evidence and will engage local consultants and include a member of the IEG methods team as part of the CPE team to address these issues. A relatively small number of closed projects has been evaluated and validated, and the difficulty in attributing results to Bank Group interventions will add to the evaluation’s challenges. In addition, given the presence of many development partners and programs that have been supported by several donors, attributing broader changes to any one intervention or actor may be difficult. 9. Quality Assurance Process 9.1 To ensure quality, we will vet the evaluation through internal and external peer review, IEG management oversight, and staff feedback. The Approach Paper and final report will be peer reviewed by Kathie Krumm (Ethiopia country director, International Growth Centre); Dr. Girma Tesfahun Kassie (principal agricultural market economist, International Center for Agricultural Research in the Dry Areas); Turhan Saleh (Ethiopia resident representative for the United Nations Development Programme); and Paul Walters (development director in Ethiopia for the United Kingdom’s Foreign, Commonwealth, and Development Office). The analysis will be conducted under the guidance of Jeffrey Allen Chelsky (manager, Economic Management and Country Programs Unit) and Theo Thomas (director, Human Development and Economic Management Department) and the overall direction of Sabine Bernabe (director-general, Evaluation). 10. Expected Outputs, Audience, and Outreach 10.1 The main output of this evaluation will be a report that presents relevant findings and lessons to inform the next CPF for Ethiopia. The CPE is planned for submission to the Board of Executive Directors’ Committee on Development Effectiveness in the first quarter of FY25. The primary audience for this CPE is the Board, and Bank Group management and staff working on Ethiopia. Its findings may also be of interest to the government of Ethiopia, development partners, and Ethiopian civil society and the private sector. Once completed, we will develop a dissemination 20 strategy, including presentations, blogs, and videos as appropriate to communicate the report’s key findings. Outreach efforts will target key stakeholders, including staff at headquarters and in country offices, other multilateral and bilateral donors, government authorities, and members of civil society. 11. Resources 11.1 Patrick Hettinger (senior economist) will lead the evaluation team, which will include Corky de Asis (evaluation assistant), Gabriela Chamartin Escobar (consultant), Giuseppe Iarossi (senior economist), Najibullah Nor Isak (voice secondee), Chikako Miwa (evaluation analyst), Stephen Porter (senior monitoring and evaluation specialist), Samjhana Thapa (evaluation officer), a member of the methods team, and other staff and consultants as appropriate. The estimated budget is US$575,000. 1This portfolio analysis covers fiscal years 2013–22, but the full Country Program Evaluation will include fiscal years 2013–23. 2The Completion and Learning Review Validation was called Completion and Learning Review Review before May 1, 2023. No change was made to the methodology. 3The 2015 Enterprise Survey found that access to finance was the most important constraint to manufacturing firms in Ethiopia, followed by electricity and trade regulations as important obstacles. The 2016 Systematic Country Diagnostic added the overvalued exchange rate and barriers to entry to those constraints. More recently, a World Bank Country Economic Memorandum (World Bank 2022c) discussed the policies adopted to support manufacturing (for example, industrial zones, access to credit and foreign exchange, and facilitating investments) but recognized that productivity had not grown since 2015 because of several factors, such as civil unrest, unavailability of foreign exchange (which restricted access to inputs), and bottlenecks to market access. 21 Bibliography Hausmann, Ricardo, Tim O’Brien, Tim Cheston, Ibrahim Worku Hassen, Can Soylu, Kishan Shah, Nikita Taniparti, Pankhuri Prasad, and Pablo Andrés Neumeyer. 2022. “Development in a Complex World: The Case of Ethiopia —A Compendium of Project Research on Advancing Economic Diversification in Ethiopia.” CID Faculty Working Paper 423, Center for International Development at Harvard University, Cambridge, MA. IMF (International Monetary Fund). 2014. “The Federal Democratic Republic of Ethiopia: 2014 Article IV Consultation.” Country Report 14/303, IMF, Washington, DC. IMF (International Monetary Fund). 2015. “The Federal Democratic Republic of Ethiopia: 2015 Article IV Consultation.” Country Report 15/300, IMF, Washington, DC. IMF (International Monetary Fund). 2016. “The Federal Democratic Republic of Ethiopia: 2016 Article IV Consultation.” Country Report 16/322, IMF, Washington, DC. Millennium Challenge Corporation. 2020. Ethiopia Constraints Analysis Report. Washington, DC: Millennium Challenge Corporation. Rahmato, Dessalegn, and Meheret Ayenew, eds. 2018. Challenges and Opportunities for Inclusive Development in Ethiopia: Proceedings of Conferences Held in 2017. Addis Ababa: Forum for Social Studies. UNDP (United Nations Development Programme). 2022. Human Development Report 2021/2022: Uncertain Times, Unsettled Lives: Shaping Our Future in a Transforming World. New York: UNDP. World Bank. 2012. Ethiopia—Country Partnership Strategy for the Period FY13–16. Washington, DC: World Bank. World Bank. 2015. 4th Ethiopia Economic Update: Overcoming Constraints in the Manufacturing Sector. Washington, DC: World Bank. World Bank. 2016. Ethiopia—Priorities for Ending Extreme Poverty and Promoting Shared Prosperity. Systematic Country Diagnostic. Washington, DC: World Bank. World Bank. 2017a. Ethiopia—Country Partnership Framework for the Period FY2018–FY2022. Washington, DC: World Bank. World Bank. 2017b. “Operational Guidance for Monitoring and Evaluation (M&E) in Climate and Disaster Resilience-Building Operations.” Guidance Note 122226, World Bank, Washington, DC. 22 World Bank. 2020. “Ethiopia—Sustainable Land Management Project I and II.” Project Performance Assessment Report 153559, Independent Evaluation Group, World Bank, Washington, DC. https://documents1.worldbank.org/curated/en/126731603826296434/pdf/Ethiopia- Sustainable-Land-Management-Project-I-and-II.pdf. World Bank. 2022a. “Ethiopia—Response-Recovery-Resilience for Conflict-Affected Communities in Ethiopia Project.” Project Appraisal Document PAD4735, World Bank, Washington, DC. World Bank. 2022b. Ethiopia—Risk and Resilience Assessment. Washington, DC: World Bank. World Bank. 2022c. Ethiopia’s Great Transition: The Next Mile. Country Economic Memorandum. Washington, DC: World Bank. 23 Appendix A. Evaluation Design Matrix Table A.1. Evaluation Design Matrix Evaluation Questions Methods Strengths and Limitations Question 1: How relevant was • Document review. Review and analysis of Bank The team will triangulate World Bank Group support to Group documents related to program description evidence from interviews, addressing the main and implementation to reconstruct the types and review of Bank Group development needs of sequence of activities supported and to identify portfolio documents, and Ethiopia, and how well did it intended contributions; review and analysis of available data. adapt over the evaluation (self-)evaluation documents (and validation) to Missing or superficial period to respond to identify relevance of design and objectives of information contained in changing priorities, evolving operations supported by the Bank Group and to project documents related country context, and learning identify evidence of contributions to outcomes. to key dimensions of from experience? • Data analysis. Analysis of external data (for interest. Question 2: How relevant and example, from the Food and Agriculture Key counterparts and effective was Bank Group Organization, World Development Indicators, and informants for the early support in helping Ethiopia Ethiopia’s Central Statistical Agency) to identify years of the evaluation transition away from a state- the evolution of key country or sector outcomes period might not be led development model to indicators (such as maize productivity, reachable, and there could one with a greater role for the manufacturing output, and manufacturing be recall bias. private sector? exports) that the Bank Group sought to influence. Unavailable or unreliable Question 3: To what extent • Analytical work review. Review of Bank Group data at the required did the Bank Group help analytics and external diagnostics to understand longitudinal and sectoral Ethiopia build resilience to the factors driving the evolution of World Bank levels. climate change, particularly support and outcome data. Georeferencing of Bank with respect to its support to Group operations might • Portfolio review. Analysis of World Bank portfolio agriculture and land and not be sufficiently to understand the coherence with World Bank water management? disaggregated. strategies, classify interventions, collect outcome indicators, and identify factors that contributed (or did not contribute) to the achievement of project outcomes. • Semistructured interviews with Bank Group staff to bridge information gaps, and with clients and partners to collect opinions about the relevance and adaptation of Bank Group operations over the evaluation period, additional evidence about the significance of the Bank Group contribution, and additional sources of evidence. • Field visits to project sites based on purposive sampling method. The objective of the visits is to talk with project beneficiaries on project expected and unexpected outcomes. • Geospatial analysis of operations to gauge the relevance of Bank Group targeting (as possible). • Fragility analysis of program and project design. • Existing resilience framework and principles to assess types and categories of interventions (for question 3). Source: Independent Evaluation Group. 24 Appendix B. World Bank Group Partnerships The World Bank Group was the largest source of international concessional financing to the country during the evaluation period, but development partners meaningfully matched its financial contributions. Table B.1 provides the commitment amounts for the 10 largest donors, along with the total for all donors, for the calendar years 2011–21.1 These top 10 financiers accounted for 80 percent of total commitments during the period. Total commitments amounted to US$53 billion from 97 organizations. This donor financing amounted to 43 percent of general government total expenditure and 7 percent of annual GDP during the same period (estimated based on OECD 2022 and IMF 2022).2, 3 Table B.1. Financing Commitments by Top 10 Donors in Ethiopia during Calendar Years 2011–21 No. Donor Commitments (US$, billions) Share of Total Commitments (%) 1 World Bank Group 18.5 35 2 United States 10.2 19 3 United Kingdom 2.4 5 4 European Union institutions 2.4 4 5 Germany 2.2 4 6 African Development Fund 1.8 3 7 Korea, Rep. 1.3 2 8 Global Fund 1.3 2 9 Canada 1.1 2 10 Netherlands 1.1 2 97 Total 53.0 100 Sources: OECD 2022; World Development Indicators Database (World Bank, 2023); World Bank partnerships. 1 The full Country Program Evaluation will include donor analysis for fiscal years 2013–23. 2Organisation for Economic Co-operation and Development (OECD) Creditor Reporting System (CRS) Aid Activity database (data extracted on February 22, 2023, 18:12 UTC); Independent Evaluation Group. 3 International Monetary Fund, World Economic Outlook Database, October 2022 25