Report No. 178502-AF ‘Through the Looking Glass’: Lessons from the World Bank Afghanistan Portfolio for FCV Engagement Tony Verheijen, Atiqullah Ahmadzai, Richard Hogg © 2022 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Contents 1. Introduction ..........................................................................................................................................................1 2. Context: Continuous efforts in balancing World Bank and ARTF support with off-budget assistance.................5 3. Three sets of lessons: Context–design–implementation ......................................................................................7 3.1 Contextual issues: The importance of adapting to changes in country context ................................................. 8 3.1.1 Responding to intensifying conflict: Scaling down and up, as needed ........................................................8 3.1.2 Importance of transparent targeting criteria ...............................................................................................9 3.1.3 Ensuring greater continuity between analytical work and operational work and mainstreaming political economy .............................................................................................................................................................10 3.2 Design and readiness.........................................................................................................................................12 3.2.1 Maintain the independence of the World Bank in decisions on project design ........................................12 3.2.2 Make sure that projects are ready to be implemented, unbundle projects where needed to achieve readiness, and use emergency processing proactively .......................................................................................13 3.2.3 Technocratic solutions have significant limitations ...................................................................................15 3.3 Implementation and portfolio management .................................................................................................... 16 3.3.1 Carefully consider limited government capacity both to manage additional projects and implement reforms, especially in conflict contexts...............................................................................................................16 3.3.2 Ensure a comprehensive risk screening of the portfolio at least annually and in circumstances like Afghanistan, probably biannually ....................................................................................................................... 18 3.3.3 Avoid locking large sums in project designated accounts in conflict situations ........................................19 3.3.4 TPM is a critical tool in FCV contexts but not a panacea for a lack of trust ...............................................19 4. Connecting investment and development policy financing engagements .........................................................21 5. Conclusions and recapping lessons.....................................................................................................................23 5.1 Linking diagnostic and risk management tools to automatic review triggers and advice to management ......23 5.2 Ensuring a more coherent and sustainable approach to institution building in a conflict-affected setting .....24 5.3 Adjusting the World Bank’s culture and business model to working in conflict-affected countries like Afghanistan .............................................................................................................................................................25 6. Looking ahead: Incorporating lessons in a new environment ............................................................................26 Annex 1: Portfolio on August 15, 2021 ........................................................................................................................28 Annex 2: Project Risk Ratings at Appraisal ..................................................................................................................30 Annex 3: Data sheet on ICR rating ...............................................................................................................................31 References ...................................................................................................................................................................33 List of Abbreviations Abbreviation Meaning ACReMAP Anti-Corruption and Results Monitoring Action Program ADB Asian Development Bank AGASP Afghanistan Gas Project AITF Afghanistan Infrastructure Trust Fund ARTF Afghanistan Reconstruction Trust Fund BMZ Federal Ministry of Economic Cooperation and Development - Germany CASA Central Asia South Asia CCAP Citizens Charter Project CDC Community Development Council CMU Country Management Unit CPF Country Partnership Framework DIME Development Impact Evaluation DRC Democratic Republic of Congo E&S Environment and Social EATS Emergency Agriculture and Food Supply Project ENETAWF Early Warning, Early Finance and Early Action Project ERA Economic Recovery for Afghanistan Study ESF Environmental and Social Framework EU European Union EZ-KAR Eshteghal Zaiee – Karmondena FCS Fragile and Conflict-Affected Situations FCV Fragility, Conflict, and Violence FSP Fiscal Performance Improvement Support Project GAC Governance and Anti-Corruption GDP Gross Domestic Product GIRoA Government of Islamic Republic of Afghanistan HEIS Hands-On Expanded Implementation Support HRM Human Resource Management ICAI Independent Commission for Aid Impact ICR Implementation Completion and Results IDA International Development Association IDLG Independent Directorate of Local Governance IEG Independent Evaluation Group IP-DPG 2021 Afghanistan Incentive Program Development Policy Grant IRAGO Incentivizing Reforms in the Attorney General's Office ISAF International Security Assistance Force ITA Interim Taliban Administration M&E Monitoring and Evaluation MRRD Ministry of Rural Rehabilitation and Development NCO Non-Completion Note ii NGO Nongovernmental Organization INGO International Nongovernmental Organization NRVA National Risk and Vulnerability Assessment NSP National Solidarity Program NTA National Technical Assistance NUG National Unity Government PAD Project Appraisal Document PAISA Payments Automation and Integration of Salaries in Afghanistan PCCs Provincial Capital Cities PDO Project Development Objective PFM Public Financial Management PIU Project Implementation Unit PPP Public-Private Partnership RCW Recurrent Cost Window REACH Relief Effort for Afghan Communities and Households RRA Risk and Resilience Assessment SAFI Afghanistan Financial Sector Intermediation SCD Systematic Country Diagnostic SIGAR Special Inspector General for Afghanistan Reconstruction SORT Systematic Operations Risk-Rating Tool UN SRSG UN Special Representative of the Secretary-General TPM Third-Party Monitoring TTL Task Team Leader UN United Nations UNAMA United Nations Assistance Mission in Afghanistan UNDP United Nations Development Programme WBG World Bank Group iii 1. Introduction This paper aims to provide a gateway to the lessons learned from 7 critical years of program implementation (out of the 20 years of World Bank engagement since 2001) leading up to the political events of August 15, 2021. This will both support ongoing efforts to safeguard some gains of the long- term engagement by development partners and inform potential future interventions when the enabling environment would allow for a more comprehensive program. At the end of this report, we outline five initial takeaways (scope, adjustability, impact of analytics on design, sustainability, early thinking about transitioning between off and on-budget) that the World Bank and partners are testing in Afghanistan in the initial 12–14 months’ engagement post August 15, 2021. Following the political events of August 15, 2021, in Afghanistan, the World Bank paused all disbursements in its portfolio of 29 projects and over the next nine months repurposed this funding in Afghanistan. Reviewing the lessons from the paused portfolio was critical to the decision to make funds available again. By end-August 2022, the World Bank had completed 23 full Implementation Completion and Results Reports (ICRs) for its projects, in addition to five Non-Completion Notes (NCOs) for projects that had not become effective or disbursed. Currently, one regional project in the pre-August 2021 Afghanistan portfolio remains active.1 Pausing the disbursements under these projects meant that all activities halted as of August 15, 2021. There have been several earlier evaluations on programmatic engagement in Afghanistan, with the Meta-Review of Development Assistance to Afghanistan carried out by the Federal Ministry of Economic Cooperation and Development - Germany (BMZ 2018) on a sample of 2008–18 programs one of the most comprehensive ones This review highlighted important lessons, especially on political economy constraints to achieving results in Afghanistan. The unique feature of our analysis is that it is based on ICRs of (mainly) investment projects that were at various stages of implementation and were paused at the same point in time. This gives a ‘benefit of hindsight’ and makes this review unique. This implies that some of the issues raised in the BMZ paper still resonate (such as political economy constraints, corruption, capacity) but others less so (such as conclusions on relevance and ‘fit’), as will be outlined later in this paper. The recently published country portfolio review of the UK Independent Commission for Aid Impact (ICAI 2022) covers the same period (2014–21) as in this lessons learned document and draws three main conclusions: (a) comprehensive support should only be provided in the context of a viable and inclusive political settlement (and this was not the case in Afghanistan), (b) support should not finance paramilitary operations by police and security agencies (which the UK support did), and (c) spending levels should have been adjusted based on more thorough scenario planning. We will come back later in this paper to issues (a) and (c), as the same lessons also came up in the ICR: the limitations on the ability of government to deliver major programs under a political settlement that was not inclusive (and effectively meant the country remained in conflict) and the limited adjustability in programs are elements that deserve more 1 While the Central Asia South Asia (CASA) regional power project remains active, activities have been discontinued and disbursements stopped due to Afghanistan’s arrears status with respect to IDA repayments. 1 reflection when engaging in similar situations and contexts in the future and should have a bearing in particular on the breadth and depth of support. Key lessons from political economy work that were discussed internally in the World Bank add some further insights, including (a) the risk of fragmented (off-budget) aid support driving political fragmentation and conflict, especially when aid flows are instrumentalized to support the achievement of security objectives; (b) the potential for contradictions and tensions to arise when aid flows are being used both to incentivize reform and provide resource transfers to a fledgling state experiencing conflict— conditionality is not effective when the threat of withholding aid is not credible (a point that will be further discussed in section 4 of this paper); and (c) the unresolvable tension between institution building and rapid program delivery objectives (further discussion in section 3.3 of this paper). While the second point is not relevant in the current political context, the other two issues have a bearing on the way future engagement is defined. On this first point, we would argue that aid coordination through shared financing mechanisms, including continued support to the local institutions that play a critical role in program delivery (Community Development Councils, farmers’ associations, health providers delivery network, and so on), is a proven model for effective service delivery while mitigating the political risks associated with aid fragmentation. On the third point, while direct support to state institutions is not possible in the current context and programs are implemented through third parties, we believe that there remains an argument for limited and well-defined technical dialogue to help sustain some key systems, such as the public financial management (PFM) system and the Central Bank role in supporting the financial system. Finally, the political environment in which the World Bank operated in Afghanistan, including the deep engagement of the international community in the country, particularly on security issues, heavily influenced the extent, timing, and nature of World Bank engagement. Such realities need to be considered in any evaluation. Coming back to the main element of this paper, the abrupt pause of 29 projects, with a total commitment of US$4.6 billion (International Development Association [IDA] US$2.2 billion, Afghanistan Reconstruction Trust Fund [ARTF] US$2.2 billion, and other trust funds US$174 million), had a critical impact on the country. ARTF- and IDA-financed projects, as the main sources of on-budget financing in Afghanistan (see the discussion in section 2 below), constituted the foundation of critical service delivery in health, education, community development, agriculture, and other core services. For example, until August 2021, the Sehatmandi health sector support project covered 75 percent of the basic health care budget. The portfolio also included strategic infrastructure engagements in transport, energy, digital services, and telecommunications as well as budget support programs and investment projects in the financial and private sectors. Further financing of large infrastructure programs was funded through the Afghanistan Infrastructure Trust Fund (AITF), administered by the Asian Development Bank (ADB). AITF- funded activities were also paused on August 15, 2021. Using the same mechanism, institutional development support was provided through projects that supported public sector management. Targeted institutional building support was also embedded in sector operations, particularly in bottom-up efforts to build community-level institutions. The dedicated public sector management projects were part of a long-standing effort to build an institutional foundation for the Afghan state. In addition, efforts in various sectors focused on building sustainable institutions for service delivery. Critical examples include the institutional system for basic and primary health care delivery, under the Sehatmandi program, which created a robust system of planning, monitoring, and 2 delivering of health services through third parties; support for the development of community development councils (CDCs), community-level institutions established to support livelihoods and local level infrastructure development, which ‘took root’ in Afghanistan; and farmers collectives, critical local institutions involved in deciding on as well as building and maintaining irrigation systems and other critical enablers for agricultural production. Hence, institution building was approached from the macro perspective and from a sector and community perspective. It is important to note that several of these critical institutions and systems still exist and are active today, including the PFM system (as well as revenue management institutions), CDCs, and farmers collectives, while the Sehatmandi approach is also still being used.2 The full portfolio as of August 15, 2021, is presented in Annex 1: Portfolio on August 15, 2021. Although there are other examples of the World Bank having a broad engagement in fragility, conflict, and violence (FCV) contexts,3 it is unusual for portfolios in these settings to be as all-encompassing as the one in Afghanistan and have as significant a component on infrastructure. The international political context, the pressures generated by it, and the sense of urgency to realize the development potential of Afghanistan, as well as the availability of a large resource envelope through the ARTF, are the main reasons for this and make it additionally important to carefully draw lessons. Within the complex context described above, the purpose of this paper is to inform the discussion among development practitioners on the lessons to draw from the World Bank’s and ARTF’s operational engagement in Afghanistan during 2014–21: what worked, what did not, and what could have been done better. It is based on the 23 ICRs completed in August 2022 and the discussions held while finalizing them.4 While the ICRs were completed during a period when World Bank teams were mostly unable to travel to Afghanistan, the ARTF Third-Party Monitoring (TPM) agent employed by the World Bank remained on the ground and was able to conduct verification that allowed teams to finalize the analyses. The reports provide a unique perspective on the Afghanistan program; conducting such in-depth project reviews simultaneously allows the identification of cross-cutting as well as sector-specific lessons. The proposed ratings of the projects reviewed in the 23 ICRs are presented in Annex 3, along with other data. While the focus of our analysis is the operational engagement during 2014–21, it is important to understand the broader achievements of the Afghanistan engagement between 2001 and 2021. For example, • The proportion of secondary-age children enrolled in secondary school increased from 13 percent to 54 percent between 2003 and 2017, • The number of connections to the electricity grid increased from 230,000 to nearly 1.5 million, • 142 more children survive to the age of five for every 1,000 births, 2 There have been recent challenges by the Interim Taliban Administration (ITA) over the selection of nongovernmental organizations (NGOs) to deliver health services, which is proving to be problematic, though the principle of running health care services through nongovernment providers remains in place. 3 As an example, the Democratic Republic of Congo (DRC) program in the mid to late 2000s, immediately after the end of major conflict, was similarly broad. 4 ICRs will be made public after their submission to the Board and are subject to review by the Independent Evaluation Group (IEG). The publication date is linked to the original closing date of each project. 3 • The youth literacy rate increased from 47 percent in 2011 to 67 percent in 2018, • The fertility rate declined from 7.3 births per woman in 2002 to 4.6 births per woman in 2017, and • Real per capita income increased 75 percent between 2001 and 2020. The ARTF-IDA financed portfolio contributed significantly to these gains as the main on-budget facility supporting social service delivery, private sector development, and institutional strengthening. To safeguard the above achievements and provide basic needs support to the people of Afghanistan, following authorization by the Board on March 1, 2022, the World Bank support to health, education, livelihoods, and agriculture has been reconstituted based on repurposed ARTF funds.5 The ADB (https://www.adb.org/countries/afghanistan/overview) and other partners (for example, the European Union [EU], European bilateral partners, and the United States) have made additional funds available to the same areas. This has reduced immediate financing gaps6 and maintained basic service delivery. These projects are now implemented off budget through United Nations (UN) agencies and international nongovernmental organizations (INGOs), based on a request of the international community (represented by the UN Special Representative of the Secretary-General [UN SRSG] for Afghanistan). Current support is limited to basic service delivery and livelihoods but may be extended in 2023 to include private and financial sector support. The ARTF was and continues to be a unique feature of the Afghanistan portfolio. It is the largest single country multidonor trust fund in the World Bank and provided US$12.3 billion (ARTF 2020) of almost exclusively on-budget support to Afghanistan between 2002 and 2020. In comparison, IDA funding in the same period amounted to US$5.3 billion. Throughout this period, the ARTF was an equal and, in some instances, the ‘senior partner’ in the financing of the World Bank-managed program (though IDA funding gradually increased to annual levels like those provided under the ARTF), which ensured that World Bank- financed projects had a large and well-coordinated support base. The role of the ARTF in coordination with the donor and the Government remains a unique and overall positive example of pooled on-budget support in a fragile country environment. ARTF financing decisions are made collaboratively with development partners , which means that when IDA and ARTF support were increasingly blended over time, development partners had influence over decisions on overall program priorities, individual project design, and approval of IDA-funded projects but also provided support to implementation discussions (see Box 1). An additional unique feature of the ARTF is the deployment of a TPM agent, which applies across the portfolio, to mitigate risks arising from limited access to project sites due to security and other constraints. No other program in the World Bank has a similar portfolio-wide TPM system in place. Apart from ensuring the program had eyes and ears on the 5 Projects under implementation include the Health Emergency Response Project, the Community Resilience and Livelihoods Project, the Emergency Food Security Project, the Education Emergency Response for Afghanistan project and the NGO capacity building project, totaling close to US$900 million in support through UN agencies. 6 The Economic Recovery for Afghanistan Study (ERA) paper on Afghanistan economic recovery estimated that to sustain basic services, up to US$1.3 billion in financing is required, a significant part of which will be provided by external (off-budget) financing. However, even counting all main funding streams, a significant shortfall in funding remains. 4 ground when World Bank staff could not travel within the country, the monitoring system also generated a wealth of data that helped validate and confirm development results achieved. Finally, the paper’s title, ‘through the looking glass’, is a deliberate choice. When reading the ICR and looking at the wealth of previous analytical work on Afghanistan, one cannot escape the conclusion that to be effective one has to go through the looking glass and understand the world on the other side—a world that might not resemble the confusing but benevolent reality in Alice in Wonderland but that needs an adjustment of perspective, approach, and working methods. This is a test on which the World Bank and its partners had some, but limited, success, regardless of intense, committed, and valiant efforts, and where the question of how we can do better in similar future contexts is an important one for the World Bank as well as other development institutions. Box 1: Blending ARTF and IDA resources to broaden support and ownership Combining trust fund and IDA resources in World Bank/ARTF projects and annual development policy grant operations helped focus scarce government capacity and align the government, ARTF donors, and the World Bank behind a common development policy framework. The administration of the ARTF allowed the World Bank to leverage the much larger funds attached to the ARTF to maximum advantage. After the end of the ARTF Incentive (budget support) Program in 2018, the World Bank and ARTF donors took an innovative approach to recurrent cost support by (a) combining IDA and ARTF resources in a new operation, (b) front-loading IDA funds against an agreed set of prior conditions, and (c) following up with release of ARTF resources in a series of tranche releases against completion of timebound reforms; any delays triggered a decline in the size of the tranche release by an agreed and set proportion. The ICR for the 2018 Incentive Program pointed to four key lessons: (a) the innovative approach helped manage the risks of transitioning from the former multiyear ARTF Recurrent Cost Window (RCW) program to one that was fully consistent with World Bank Development Policy Financing guidelines and made all disbursements conditional to policy reforms, (b) government ownership was critical to reform progress and was facilitated through a clear assignment of responsibility and regular monitoring, (c) multidonor engagement and oversight facilitated effective policy dialogue, and (d) the Development Policy Operation served a valuable purpose in focusing scarce government capacity. 2. Context: Continuous efforts in balancing World Bank and ARTF support with off- budget assistance Afghanistan was, pre-August 2021, noteworthy for its extreme levels of aid dependency, the significant imbalance between on- and off-budget support, and the predominance of security support in the overall assistance program. Despite a decline from its peak in 2010/11 when aid represented nearly 100 percent of gross domestic product (GDP), it still accounted for around 43 percent of GDP in 2020, financing 50 percent of the budget and 75 percent of total public expenditure, including 90 percent of security expenditures. At the Geneva conference held in November 2020, donors renewed their commitment to support Afghanistan for the next four years, but pledges were lower by 20 percent compared to commitments made at the Brussels conference in 2016. Several major donors provided only single-year pledges, with future support made conditional on the government achieving accelerated progress in efforts to combat corruption, reduce poverty, and advance ongoing peace talks. 5 Aid was mostly off budget and security focused. The data on aid flows in 2011 is a good example to illustrate the structure and distribution of aid. The international community committed US$16 billion in development and security assistance to Afghanistan during 2011. Of the total committed amount, only US$13 billion was disbursed; 68 percent of the assistance, or US$8.8 billion, was allocated to the security sector and 32 percent, or US$4.2 billion, was allocated to the civilian sectors (OECD 2012). Of the US$13 billion disbursed, only US$2.3 billion was channeled through the national budget and the remainder was implemented off budget with minimal coordination with the recipient government. About 49 percent of the on-budget assistance was in the form of bilateral contributions and the remaining 51 percent was routed through pooled resources (OECD 2012). A similar pattern is evident in the aid data (Figure 1) between 2003 and 2020. Figure 1: Evolution of on- and off-budget spending 2003–20 (US$, billions) 18 16 14 12 10 8 6 4 2 0 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 Security (on budget) Security (off budget) Civilian (on budget) Civilian (off budget) Source: Bank Staff Estimates based on the Publicly Available Data IDA and the ARTF collectively remained Figure 2: IDA and ARTF financing 2014–20 (US$, the largest contributors to on-budget millions) assistance. During the years on which IDA ARTF this analysis focuses, 2014–21, IDA and 1600 the ARTF disbursed on average US$1.2 1400 billion a year through the nationa 1200 l budget. However, this was only 12,5 and 1000 800 10 percent of the overall external 600 assistance to Afghanistan. As highlighted 400 above, IDA and the ARTF had 29 active 200 projects in August 2021, ranging from 0 support to public-private partnership 2014 2015 2016 2017 2018 2019 2020 (PPP) initiatives to financing basic and essential health services. Source: ARTF and IDA Financial Statements 6 Most of the financing went to health, education, community development, and basic infrastructure projects. It is important to note that a large portion of the ARTF financing was used to cover government recurrent cost, skewing the ARTF to IDA ratio. The objective of the government at the time was to create a financially sustainable state. While the long-term objective was for the government to reduce its dependence on financial assistance, this was not likely to be achieved, even in the medium term, or at least not as long as conflict-related security cost would have remained at the same very high level. While the nature of on-budget recurrent cost support changed over time, with an increasing emphasis on conditions and incentive-based transfers to drive reforms in government, a large gap between expenditures and revenues remained. Domestic revenues increased dramatically from 2003 to 2011 but dropped as a percentage of GDP during the Transition in 2012–147 and then started to increase again from 2015 to 2019, reaching 14 percent. But much of the growth driving the 2019 figure was achieved by one-off sources, including State Owned Enterprise profits and a transfer from Da Afghanistan Bank, while the underlying trend was flat. In 2020, revenues collapsed. The bulk of public expenditures were related to security costs. Government revenues have exceeded recurrent civilian expenditures since 2009 but the outsize scale of security costs meant that the government relied on the international community (mainly the US) to pay for 90 percent of these costs, primarily off budget.8 In 2019, fiscal sustainability, defined as domestic revenue as a share of operating expenditures of government, stood at an estimated 74 percent (World Bank 2020a). Fiscal sustainability was never achieved nor was likely to be achieved in the medium term given the high security expenditures and volatility of growth. The abrupt pause of aid to Afghanistan following the political events of August 2021 revealed the country’s stark reliance on external assistance. The sudden pause in aid triggered an economic crisis and disruption to basic services that had previously depended on international assistance, including basic health and education. This, combined with an ongoing drought, led to a sharp increase in food insecurity with an estimated 70 percent of Afghan households having insufficient income to meet basic food and non-food needs. The situation led many bilateral donors and multilateral institutions to continue their engagement, in support of the Afghan people, primarily through UN agencies, with off-budget financing outside of the control of ITA. 3. Three sets of lessons: Context–design–implementation The lessons drawn from the 23 ICRs completed in August 2022 are important for the design of future engagements in Afghanistan and in FCV settings in general. The ICRs cover the full spectrum of World Bank operational engagements, including investment operations in all key sectors as well as policy-based lending programs. While this paper focuses on the investment operations, reference will be made to the policy-based lending programs where they played a critical role in securing the enabling conditions for effective investment lending support. This came in the form of support for broader institutional reforms (PFM, procurement, civil service capacity) and in taking forward sector policy reforms in key sectors, 7 The Transition referred to here is the gradual handover of security responsibilities to Afghan forces. 8 The government contributed up to 5 percent of GDP from its own revenue, while a similar percentage was provided on budget for fuel and other armed forces expenditures. 7 including financial and private sector reform and regulatory management. Following the logic of the investment project cycle, the lessons learned are presented in three main blocks: context, design, and implementation. 3.1 Contextual issues: The importance of adapting to changes in country context Considering context in design is a critical success factor for development projects, and the link to and reflection of context (country and sector specific) is reviewed in depth in the ICRs. In the Afghanistan program, the political, security, and economic context was constantly evolving, and an important challenge was to anchor projects in an up-to-date analysis of context despite this. Analytical work, particularly just-in-time political economy analysis, can provide this anchor and can be used to contextualize design while in-project adjustments and proactive risk management can do the same during implementation. The three lessons discussed here demonstrate how the World Bank and development partners can improve the ways they adjust project design and implementation for context. 3.1.1 Responding to intensifying conflict: Scaling down and up, as needed The starting point for a discussion on ‘fit for context’ is the nature of the context, which, in FCV environments, includes an analysis of dynamics around a postconflict settlement (if there is one) or resurgence of conflict, where this occurs. In the case of Afghanistan, the political settlement after the Bonn Conference in 2001 did not include all parties to the conflict and hence, in subsequent years, the conflict between those who signed on to the political settlement and those who were excluded from it increasingly emerged as a fault line in the delivery of development assistance. While the World Bank FCV strategy9 emphasizes the need to remain engaged in countries in active conflict, it also sets out guidance on the nature of such engagement, proposing a narrower focus in fragile and conflict-affected situations (FCS) on ensuring the delivery of basic services. Following this approach, and in a context that combined an only partially inclusive postconflict settlement with a resurgence of conflict, one response would have been for the World Bank to focus its engagement as the conflict intensified. In Afghanistan the opposite happened, in part due to the emphasis of partners on the World Bank playing a role in supporting the Republic. Given this, contextualizing projects took on added significance. The discussions in the ICRs indicate a strong bias toward continuity in design rather than adjustability: the World Bank essentially kept doing the same things. Development organizations face difficult dilemmas when it comes to decisions on expanding or reducing aid in conflict contexts. In Afghanistan, as in other conflict-affected states, such as Iraq (World Bank 2017a), as the conflict worsened there was more pressure to increase the flow of funds. The World Bank continued approving large budget support disbursements to the government, as late as end-June 2021, as a response to and a concern over an impending fiscal crisis. While these disbursements were based on government taking policy or reform actions, there were growing questions over the extent to which government had the control and ability to deliver on policy and reform commitments. In addition, there was pressure to respond to the COVID-19 crisis. Ultimately, despite the ongoing and intensifying 9 World Bank Group Strategy for Fragility, Conflict, and Violence 2020–2025 (English). Washington, DC: World Bank Group. https://documents.worldbank.org/curated/en/844591582815510521/World-Bank-Group-Strategy-for- Fragility-Conflict-and-Violence-2020-2025. 8 crisis, the World Bank did not adjust its approach until June–July 2021. Other than to release money for mitigating the health and social impact of COVID-19, there was little effort to reduce the size or scope of operations by cutting, downscaling, or re-prioritizing projects in response to the ongoing conflict until just before the collapse of the government, when the Ministry of Finance asked the World Bank to re-prioritize the portfolio. This is regardless of analyses that indicated the need to respond to a changing context starting with the Afghanistan Transition report (Hogg et al. 2014). It is important to note that the World Bank was not alone in maintaining its portfolio and disbursements; other development partners did the same. These decisions may have been made in good faith, but a more realistic adjustment to the 2014– 21 operating context would have been justifiable in hindsight. Table 3 shows the allocations from 2014 to 2021, a period marked by rapidly increasing conflict. In this respect, a first critical lesson is the need to build in adjustability in programs in conflict settings, based on a set of identified triggers and checkpoints. Risk and Resilience Assessments (RRAs) and other analytical studies could be used to identify these triggers and checkpoints. Table 3: IDA and ARTF Annual Allocations (US$, millions) Fiscal Year 2014 2015 2016 2017 2018 2019 2020 2021 IDA Annual approvals 106.7 21.5 250.0 448.0 366.0 460.0 312.9 784.5 ARTF Annual approvals 785.2 986.1 946.3 647.9 1,151.3 743.5 405.0 1,196.0 3.1.2 Importance of transparent targeting criteria The World Bank’s mission is to end extreme poverty and promote shared prosperity. While conflict is a critical driver of poverty, it is not the only one. Efforts were made in several projects to use poverty indicators as criteria to define target areas as well as to adjust the implementation approach to include high conflict risk areas, such as use by the Citizens Charter Project (CCAP) of its ‘High Risk Area Implementation Strategy’ (as discussed in the ICR on the CCAP), which allowed for continued delivery by NGOs in areas controlled by the Taliban. At the same time, there is evidence of different criteria being applied at different times: pre-2014 research, looking at all sources of development financing (including the funds provided by the Provincial Reconstruction Teams), indicated the following: “Provincial data show that higher levels of conflict were positively correlated with both a larger presence of troops (international and Afghan) and larger aid flows. Household data show that the negative impact of conflict on consumption was more than offset by the positive impact of aid and troops.” (Vincent, López-Acevedo, and Rama 2012). There was another shift in criteria after most international troops were withdrawn in 2014 as there was less pressure for conflict-related geographical targeting and as rural engagement became more insecure without the security provided by international troops. Instead, there was a move toward greater engagement in urban areas, as seen in the increasing number of projects focusing on cities (the Kabul Municipal Development Project, Cities Investment Program, and the urban component of the Eshteghal Zaiee - Karmondena [EZ-KAR] Project) or to more peaceful and accessible districts where programs could be delivered more easily, with access and staff security increasingly dominating programming decisions. 9 The choice to do more urban-focused projects at the time was strongly driven by the government of Afghanistan partly as a response to the challenges faced by the growing numbers of internally displaced and returnees from other countries who were settling in urban areas but also by more political considerations to provide a bulwark against growing Taliban influence and improve the legitimacy of the state in vulnerable urban areas. For example, at the time of appraisal of the EZ-Kar Project, it was estimated that 1.7 million Afghans were internally displaced (often having fled a rural area for security reasons), while up to 1.5 million returnees had arrived in the previous two years. About half of these two groups were settling in urban areas, putting enormous pressure on service delivery systems. In a similar vein, the Cities Investment Program was designed to help provincial capital cities (PCCs) handle urban growth and increasing poverty. The 34 PCCs were home to one-third of the population; the project aimed to assist 9 PCCs to improve service delivery and infrastructure—and ultimately living conditions—while setting an example for other cities as well. Sometimes, it has been practical to design projects for a specific location to pilot the approach before rolling it out more broadly, especially when there have been concerns about the capacity of the counterpart agency. Herat Electrification aimed to provide electricity in population centers and rural villages in a single province, for these reasons. Successful procurement and good implementation progress before suspension demonstrated the value of this choice of limited geographic reach that did not overburden the implementing agency. Over time we have seen changing priorities for targeting districts, both at the portfolio level (shifts in and out of conflict areas, as well as changes in the balance between rural and urban engagement) and within individual projects. While Afghanistan had quite granular poverty data (see the various National Risk and Vulnerability Assessments [NRVAs] from 2007/8, 2011/12, and 2016/17), poverty indicators were not always consistently used to drive programming. In a conflict situation there were other important considerations, including—above all—the safety of staff and consultants. In reading the ICRs the reader gets a sense of limited consistency in the way selection criteria were used, especially when comparing different project approaches to targeting. In this regard, it is important that clear guidance is given to teams on targeting and the application of agreed criteria, partly to avoid perceptions of the instrumentalization of aid but also to provide clarity to task team leaders (TTLs) on the range of legitimate criteria to use, including by discussing this up front in the Country Partnership Framework (CPF) and other strategic documents. Based on the above, a second lesson is to consistently review the use of poverty indicators as the main criterion in defining programming and in particular geographic targeting. Deviations from the use of poverty indicators, while admittedly difficult to avoid in FCV contexts, should be made explicit and explained, while principles of engagement should be laid out clearly in strategic planning frameworks like the CPF. 3.1.3 Ensuring greater continuity between analytical work and operational work and mainstreaming political economy The ICRs point to a disconnect between project design and critical lessons from World Bank analytical work. The World Bank in Afghanistan has a record of producing excellent analytical studies, but the 10 findings were often not picked up and mainstreamed or were considered too politically sensitive to be shared widely. As an example, parts of the Systematic Country Diagnostic (SCD) and CPF in 2016 that discussed elite politics in the country were toned down. This point was noted by IEG in its 2021 evaluation of World Bank programs in conflict countries: “However, the client-facing nature and the potentially broad distribution of conflict analyses in the World Bank have sometimes prevented frank assessments of fragility and conflict drivers, limiting the transmission of conflict considerations into portfolio and operational decisions. The FCV strategy indicates that the Bank Group’s approach and interventions should be tailored to FCV drivers, but the analysis and redress of these drivers are more difficult when the root causes of conflict are more political in nature (that is, geopolitics, elite capture, corruption, and pervasive governance challenges” (2021b, 16). As a further example, a comprehensive and actionable country RRA was finalized in early 2021 but came too late to influence the program trajectory in a decisive way.10 It is interesting to note that the RRA predicted a worse-case scenario which would lead to off-budget engagement through UN agencies and INGOs, which reflects what happened post August 2021. Even where project implementation proceeded in parallel with related analytical and impact assessment work, for instance, the work done by Bedoya et al. (2019) on the effectiveness of support mechanisms to ultra-poor households (supported by the Development Impact Evaluation [DIME] program under the Development Economics Department of the World Bank), this is not considered or referred to in the relevant project ICRs. This even though the work done under the DIME-sponsored engagement was relevant to the operational work being done under the Financial Sector, CCAP, Relief Effort for Afghan Communities and Households (REACH), and Women Economic Empowerment projects. ICR authors did not consider the parallel impact studies in their assessment, illustrating the limited links between operational engagement and relevant analytical work. Finally, the absence of documented political economy analysis and discussion around specific projects is notable. Political economy discussions are limited and often nonexistent in the ICRs. This reflects a broader issue that exists across the World Bank: regardless of earlier efforts to mainstream such work in project design in the context of the Governance and Anti-Corruption (GAC) Strategy (see the IEG 2013 review of country-level engagement on governance and anti-corruption), project preparation and implementation processes still do not routinely include analytically based attention to political economy issues. A third critical lesson is to devote more systematic attention in project pre-appraisal and appraisal discussions on how well critical analytical lessons, particularly political economy analysis, are reflected, as well as to monitoring shifts in the political economy context throughout implementation, which could improve just-in-time adjustment of projects to reflect greater realism. This may require revisiting the 10 In IDA countries on the World Bank Group (WBG) FCS list, RRAs are required as part of the country engagement process to feed into CPFs. In the case of Afghanistan, a fragility assessment was conducted in 2015, feeding into the last CPF for Afghanistan which was from FY17. 11 gradual marginalization of political economy work since it was first mainstreamed under the GAC Strategy and addressing this with priority for FCV country engagement. 3.2 Design and readiness Design and readiness are the second block in lessons learned. Good project preparation and ensuring that project activities are ready to be launched as soon as approval is obtained are a second critical success factor. Given the added complexity of effective design and of ensuring readiness in challenging FCV contexts, particular attention in the ICRs was devoted to the quality of preparation and readiness. There is an inherent tension between readiness and good design on the one hand and agility and responding to emergency situations on the other. The ICRs indicate that in many instances projects were prepared in a de facto emergency mode (even though they followed regular preparation procedures), pushing important aspects of design into the implementation phase. The CASA community development project, the Trans-Hindukush Road Connectivity project, and the Herat electrification project are all examples of this. While the tension between preparation quality/readiness and responsiveness is critical, the way this was addressed in the Afghanistan portfolio was generally to apply standard design processes and to ‘argue away’ readiness issues. This, in turn, created unrealistic expectations on delivery and put task teams in difficult positions, especially when it came to making adjustments in a difficult political economy context: once a project was within the sphere of influence of a particular group, the ability to adjust was significantly reduced due to the power dynamics resulting from the political settlement. In addition, the portfolio—as it stood in August 2021—included several formal emergency projects (as per Bank Operational Procedure 8.00) that were prepared as part of the COVID-19 response. In the section on implementation, we will come back to the way in which ‘rushed’ design can be compensated for by adaptability and proactive risk management in implementation. 3.2.1 Maintain the independence of the World Bank in decisions on project design The design of World Bank-financed projects is a joint effort by the World Bank and the client. Client efforts to ensure projects meet their political objectives always pose a risk. There was constant pressure on the World Bank from the government to act or not act, to approve certain projects, or free up funds for recurrent cost support (instead of investment project) because revenues are declining, or cancel/restructure projects at short notice, often for political reasons. The relationship between the World Bank and the country leadership in Afghanistan was particularly complex, exacerbating the pressures to arrive at often quite fundamental portfolio decisions. Walking the thin line between being responsive to the client and being responsible to protect the institution and the quality of the portfolio from undue political pressure from country leadership posed a constant challenge to the World Bank in Afghanistan. An exacerbating factor in Afghanistan was the perceived links between development and military operations in a context of active conflict. Historically, there were efforts from the International Security Assistance Force (ISAF) to use the National Solidarity Program (NSP), the largest community development project in the country, as part of its counterinsurgency operation in Kandahar and Helmand by offering to 12 contribute funds.11 While this was met with pushback from the World Bank, it illustrates some of the dilemmas faced in project design in an active conflict context where political-/conflict-related priorities and developmental ones can be hard to distinguish. The critical lesson is that mechanisms should be built into design and implementation processes so that attempts at political instrumentalization of development projects (in an FCV and active conflict context) can be safely flagged and escalated by task teams and managers. 3.2.2 Make sure that projects are ready to be implemented, unbundle projects where needed to achieve readiness, and use emergency processing proactively While this lesson applies to all projects globally, it was particularly relevant in Afghanistan where the government was under continuous pressure to deliver quick results to maintain legitimacy in the face of an intensifying conflict. Because of the need for the government to show quick results, project preparation times were often reduced. As a result, many of the projects in the portfolio were not ready at approval. TTLs and project management spent months or sometimes years of a 5- or 6-year project implementation period on de facto preparation work like recruiting staff and obtaining approval for procurement packages. These delays affected disbursement schedules to the point that most projects in the portfolio at the time of the pause had more than 50 percent undisbursed funds against commitment, after several years of implementation. Scott Guggenheim, adviser to several Afghan governments, noted, “Aside from some poorly conceived outlier projects, all donor projects in Afghanistan fail for one or more of the same three reasons: nobody in government “owns” the project; nobody qualified occupies the positions expected to implement it; or its budget bears no resemblance to reality. Focusing on assuring ownership, demanding qualified staffing in key positions, and verifying budgets would benefit everybody, Afghans most of all” (Guggenheim 2021). In this respect, spending more time on preparation would be more than justifiable. One example is the CASA community development project , which was designed to support communities affected by the construction of the high voltage power line between the Kyrgyz Republic and Tajikistan and Pakistan (through Afghanistan) under the CASA-1000 regional power project. While the CASA community development project was approved in 2014, the project could not start implementation until 2017, as the exact trajectory of the power line funded under the regional power project and communities affected were only defined at that time, leading to implementation delays in the community development facility. Additionally, one of the lessons drawn from the ICR of the Trans-Hindukush Road Connectivity Project was that bundling design and implementation of a large-scale infrastructure project into one operation 11 This point is based on two sources. First, according to Beath, Fotini and Enikolopov (2017, 9), “In the aftermath of the fall of the Taliban regime [2001], NSP was the primary vehicle for projecting the authority of the new administration to the countryside and, as such, served an implicit state-building function in establishing the government as a benevolent provider of public goods and services. As the insurgency gained strength after 2006, military and political advisors and media commentators expressed interest in NSP as a means by which to stem the expansion of neo-Taliban control across rural Afghanistan. American counterinsurgency experts, for instance, urged increased funding for the program, arguing that it would improve security “by building an Afghan state through Afghan means.” The second source is discussions with the TTL of the NSP around later similar efforts. 13 was detrimental to its success. Given the uncertainty over possible construction bottlenecks and the need to calibrate implementation to evolving geopolitical and economic circumstances, the design work needed considerable time. Using a series of projects or multiphase approach would have allowed adequate time for the design and detailed implementation planning, followed by a second period for works. In an FCV context like Afghanistan, attempting a comprehensive approach in one challenging infrastructure project may be overwhelming and ineffective. In contrast, for the Afghanistan Gas Project (AGASP), a phased approach was adopted within the project design, prioritizing construction and fieldwork to deliver gas quickly and then addressing institutional issues, to avoid overburdening implementation counterparts with too many priorities at once. During preparation, the decision to not include a proposed component from a related subsector helped keep the project streamlined. Despite a compressed period for preparation, the project began with some momentum after approval. While changes in key project staff and delays in contracting appropriate consultancies slowed implementation, the overall phased approach suited the Afghanistan context better. An additional risk of a ‘rush to commit’ and the focus on volume is that it eliminates financing buffers that in turn prevent adequate responses to actual emergencies. The fragmented approach to COVID-19 restructuring, where small amounts were taken out of many projects to fund the emergency, illustrates the combined risks of not retaining buffers along with the failure to consider readiness factors in fund reallocation. Project performance and readiness were not considered when trying to free up resources, 12 reportedly over concerns about alienating different factions within the government. As a result, COVID- 19 restructuring became a missed opportunity to close projects that were either nonperforming or no longer relevant in a rapidly deteriorating political and security context. Overall, an effort to design projects that are simpler and more achievable can be based on a more proactive use of emergency project preparation under Operational Policy 8.00 from the start. The Emergency Agriculture and Food Supply Project (EATS) is a positive example of an emergency project, prepared in a few months in response to the COVID-19 pandemic. It aimed to support critical food supply chains and create short-term economic opportunities through simple interventions. In the first year of implementation, the project distributed the intended number of seed packages and made substantial progress on establishing new orchards and providing kitchen garden kits, as well as carrying out the planned activities that would lead to improving irrigation schemes and watershed management in the second year of the project. It was unable to achieve much on job creation, which would have picked up with the works on irrigation. Although this project was suspended and could not reach its final targets, much of its design has been continued in the Emergency Food Security Project approved under Approach 2.0. Finally, we would like to highlight the challenges of the application of the Environmental and Social Framework (ESF) in FCV contexts. As an example, under the AGASP, the application of the World Bank’s ESF required preparation and submission of 16 documents, running to hundreds of pages, due to the risk profile of the sector and the country. This quantity of specialized technical work in a low-capacity FCV context absorbed resources and attention. The World Bank may consider ways to support ESF 12 An alternative, applied in other settings, would have been to close nonperforming projects and reallocate their resources to address the COVID-19 emergency. 14 implementation in FCV countries, such as by trying to build environment and social (E&S) capacity on a systematic basis rather than project by project or establishing an approach like the Hands-On Expanded Implementation Support (HEIS) used for procurement. Alternatively, the World Bank could consider more carefully whether projects presenting such E&S complexities in the face of weak capacity should be undertaken and how much preparation time should be allowed to fulfill ESF requirements. The fifth lesson is one of patience and staying the course on ensuring projects are as implementation ready as possible at approval. Apart from implementation delays, an additional risk in locking up resources in projects that are not ready is that they are then unavailable to the country for urgent needs. In addition, keeping the design simple and avoiding projects that require highly complex ESF documentation (for which capacity is not available locally) need to be considered. 3.2.3 Technocratic solutions have significant limitations While development financing helped achieve some institutional development and public sector objectives, these were insufficient to challenge the entrenched interests in the country. As the World Bank’s RRA states, “Bank/ARTF support to on-budget programs and efforts to build accountable public finance and civil service institutions have proven ultimately insufficient to offset the impact of broader political economy drivers and patronage-based governance systems. Efforts to track the impact of Bank activities on governance, conflict, and fragility through results monitoring have been limited. Institutional incentives toward provision of new or continued support despite major governance deficits has weakened the incentive power of aid to drive improved management in the face of negative political economy incentives. Inadequate in-the-field (as opposed to in-office) presence plus rapid staff turnover has sometimes led to programming and knowledge discontinuities, and a tendency to privilege technocratic solutions over context-specific solutions that are feasible within governance and capacity constraints. Reliance on technocratic solutions and faulty political economy assumptions have weakened outcomes in non fiscal governance reform – especially in relation to civil service and private sector reforms” (World Bank 2021b, 7). The review of the 23 ICRs revealed many instances where projects were designed based on first best technical approaches. For example, several of the projects focusing on the banking and financial sector, such as the Financial Sector Intermediation (SAFI) project and the PPP project, were ill suited to a country which required an engagement on much simpler and basic ‘bread and butter’ reforms. Another example is the Digital CASA Project; this built on the country’s earlier progress in expanding internet access (including two earlier projects supported by the World Bank) and aimed to increase access, crowd-in private investment, and deliver digital government services. A core element was to obtain expanded international internet bandwidth and government network connectivity through procurement of long-term contracts using procurement methods that were new to Afghanistan. Although financially sensible, this approach was ultimately rejected by the counterparts during implementation. Compounding the difficulties faced by the project were the frequent changes of minister and inadequate technical staff at counterpart agencies; some authorities and experts who had agreed to the project’s original design were no longer involved later. Political economy analysis, critical to achieving ‘fit for purpose’ solutions in project design, largely remained as an afterthought. 15 The ICR reviews showed an across-the-board tendency to prefer ‘first best’ over ‘politically achievable’ projects. The sixth lesson is therefore that especially in FCV contexts, contextual analysis and political economy proofing of projects are required to ensure realism and alignment with country capacity and its economic and institutional development. This needs to take precedence over fast-tracking approval processes. In this way, project teams could have partly avoided the significant project performance issues that are highlighted in many of the ICR reviews. 3.3 Implementation and portfolio management Effective project and program implementation is the responsibility of the government. However, the World Bank has a responsibility to ensure that projects are implementable and that the government has the capacity to deliver the program. Given the ARTF’s mandate to support institution building, the broader World Bank engagement in Afghanistan also included a strong emphasis, until August 15, 2021, to support the government with its implementation capacity. 3.3.1 Carefully consider limited government capacity both to manage additional projects and implement reforms, especially in conflict contexts In the last seven years of the Islamic Republic of Afghanistan, the absorptive capacity of ministries to take on additional projects was severely strained. Complex fragmented internal politics, combined with highly centralized decision-making at the top level, undermined the capacity of ministries to implement programs and reforms. As the evaluation of the long-running ARTF recurrent cost operation commented, “The goals of the international community in Afghanistan were expansive, including the establishment of a legitimate and responsive state as a means of preventing further violence and fragility. The program therefore aimed to strengthen the delivery capacity of government systems and institutions. This objective reflected: i) prevailing assumptions that effective delivery of services by the state would enhance state legitimacy and help address conflict and fragility; and ii) the international community’s desire for a program that would support the emergence of stronger state institutions to achieve broader shared state- building goals” (World Bank 2019b, 17). Seeking durable institutional reforms in a country as fragile as Afghanistan in an active conflict context was always going to be challenging. Government capacity was extremely limited and TTLs spent considerable time trying to put together workaround solutions that bypassed weak government institutions. The reforms that worked (such as the earlier discussed institution building efforts at community level, delivery systems in the health sector, and the establishment of PFM systems) neither challenged the established order nor created an effective foundation for ministries or local government institutions to manage the implementation of development programs. As a workaround, most World Bank investment projects worked through dedicated Project Implementation Units (PIUs) or through national technical assistance (NTA), circumventing normal government processes and civil servants. In addition, World Bank projects were protected by stand-alone fiduciary and monitoring arrangements under the ARTF-funded Anti-Corruption and Results Monitoring Action Program (ACReMAP), which, while effective in achieving their objectives of limiting the scope for abuse of funds, operated in parallel to government 16 systems. Hence, workaround arrangements enhanced effectiveness (and reduced corruption risks) but reduced the sustainability of the intervention. The CCAP, for example, effectively had its own contracted staff dedicated to the project in the two institutions it worked with, the Ministry of Rural Rehabilitation and Development (MRRD) and Independent Directorate of Local Governance (IDLG). The MRRD had 2,460 contract staff (and 313 civil servants) working on the project, almost half the entire staff of the ministry. The IDLG, which managed the smaller urban component, had 209 contract staff and only 18 regular civil servants. HABITAT also appointed oversight consultants to CCAP’s urban component for the first two years of its implementation. Establishing dedicated ways to implement and protect projects contributed to setting up islands of excellence removed from the normal government processes that can be used to enhance the effectiveness of any intervention but also risks adding to institutional fragmentation. The use of PIUs in a weak capacity environment like Afghanistan (and many similar low-income countries) was an inevitable short-term solution for delivery of services. The alternative would have been to wait for capacity to be developed without any or minimal service delivery. This was never a viable option in the context of a government and donor community that wanted to see quick results on the ground. In the end, despite the contradictions, both were rolled out together—the World Bank tried to build long-term government capacity through various civil service reform initiatives (for example, Taghir and its predecessor, the Capacity Building for Reform project) while also complementing it by using PIUs and NGOs to deliver services through on-budget financing with the government (reluctantly) contracting NGOs, for example in Sehatmandi or CCAP, to deliver health services and community mobilization. The contradictions between these different strands were an inevitable consequence of the drive to get results but led to a fragmentation of approaches on the ground. This was both a strength and a weakness—it spread the risks of failure by diversifying approaches but also undermined the effort to build longer-term civil service capacity and contributed to institutional fragmentation. Finally, the World Bank had the option to remain fully on budget and yet use mixed implementation modalities. This could have been done through outsourcing implementation of some projects to NGOs with sound governance structures and implementation capacity (as happens in many conflict contexts) and maintaining implementation through government agencies on others. This could have been an alternative solution that would keep some critical projects mainstreamed in line ministries (as per the ARTF and World Bank objectives) and others implemented through third parties (while remaining reflected in the budget). Government consent for this, however, would have been necessary, which might have been difficult to obtain (though this has worked in other contexts like South Sudan), but it was a potential alternative. The seventh lesson is that there is a need to consider government capacity and the impact of World Bank approaches on overall capacity reform efforts, with greater candor when running a large and expanded portfolio in an FCV context. Workarounds, such as standalone PIUs, while useful in the short term, unless carefully calibrated with wider reform efforts and time bound, may ultimately erode longer-term government capacity development. A blended implementation model (critical basic services, to the extent 17 feasible, through government13 and outsourcing of more complex project activities to third parties) might be considered as an alternative in such a context while always ensuring that financing remains on budget. 3.3.2 Ensure a comprehensive risk screening of the portfolio at least annually and in circumstances like Afghanistan, probably biannually The risk tool the World Bank uses, based on its Systematic Operations Risk-Rating Tool (SORT), did show heightened risk to the projects at the time of the Presidential elections in 2019 and mid-year 2021 combined with an overall decline in performance. However, the level of risk showed only a rough 65:35 split between High and Substantial in mid-2019, an improvement by mid-2020, and a further increase by mid-2021, with a 60:40 split between High and Substantial (see Annex 2). Once the United States signed a peace agreement with the Taliban in February 2020, the World Bank should have re-reviewed all its projects and reconsidered all risk ratings as well as put a watching brief on the overall program, given that it was increasingly likely that either the government would negotiate a settlement with the Taliban or the Taliban would take over. Ultimately, there seemed to have been little planning for a Taliban takeover. The strategic assessment that was carried out as part of the RRA in 2020/21 came too late to allow it to be translated into operational actions, as such portfolio adjustment to risk started only in July 2021. In this respect, it would have been important for Afghanistan to have conducted an RRA at a much earlier stage, as was the case for many countries that were less affected by conflict. At an individual project level, few projects seem to have assessed the level of risk appropriately at appraisal. Indeed, at an institutional level, there is an implicit disincentive in the World Bank to rate projects as High risk. In Afghanistan, given the context, it is arguable that most projects should have been rated as High risk. Project appraisal meetings should have imposed a much more rigorous standard in assessing risk in the proposed risk mitigation plans put forward by task teams. The CPF was right to highlight that “The two principal risks to the program are the intensifying conflict and the political instability of the NUG14 coalition. Managing these risks, which are interrelated, will not be easy as they are both outside the direct control of management” (World Bank 2016a, 40) but never came up with a convincing mitigation plan, largely because there was none, except to downscale or close the program. “However, there is only so much the Bank Group can do to insulate its programs from a deteriorating security situation” (World Bank 2016a, 43). Based on the above, the eighth lesson is that the World Bank needs to ensure that regular risk assessments translate into actions to downscale or adjust the level of its exposure, especially in an active conflict setting , and that, at the project level, more agile restructuring and adjustment processes are instituted specifically for FCV countries. 13 An important caveat here on the health sector, where basic service delivery through NGOs proved both more cost-efficient and effective. 14 NUG = National Unity Government. 18 At the portfolio level, a proactive use of diagnostic tools like the RRA should help in informing such decisions. RRAs should be conducted as a priority in active conflict countries, which was not the case in Afghanistan, which was one of the last countries to do a full RRA.15 At the level of individual projects, better and less rigid project adjustment tools need to be provided to task teams so that adjustability can compensate for an inadequate level of readiness. Current restructuring processes are not only cumbersome but can also lead to adjustments being held hostage to internal conflicts and disagreements within the client government. 3.3.3 Avoid locking large sums in project designated accounts in conflict situations Prudential risk management would suggest reducing amounts in project designated accounts as the risks to government increase. Partly as a response to COVID-19 and the need to disburse funds in REACH and CCAP quickly, ceilings on advances to designated accounts in these projects were increased significantly in 2020. This came back to haunt the World Bank in mid-2021. At the time of Taliban takeover, the World Bank had more than US$562 million in the Central Bank in designated accounts as payment in advance against future expenditures. A yet undetermined portion of this is unlikely to be properly documented.16 According to Ministry of Finance data as of August 9, 2021, REACH had over US$95 million and the CCAP with its large number of CDC accounts had nearly US$151 million outstanding in designated accounts. As a ninth lesson, a safer way needs to be found in active conflict contexts to channel resources to communities and contractors including direct payments which do not expose either the World Bank or government to large losses. As noted in lesson seven, this may be through more ‘up-front’ use of INGOs as implementation partners and fiduciary agents and even holding project advances in offshore accounts.17 3.3.4 TPM is a critical tool in FCV contexts but not a panacea for a lack of trust In a conflict situation, TPM is critical to provide reassurance that funds are being used for their intended purpose, especially where government is not in control over the full territory. TPM in Afghanistan was and is unique in its purview to cover almost the entire program. The World Bank increasingly relied on TPM to assess everything from project physical outputs to project fiduciary compliance and to verify statements of account before advances were made to designated accounts. This increasing reliance on TPM provided the World Bank and donors with a reasonable level of assurance that funds were being used properly. However, it also became a convenient way to gloss over a far more fundamental issue— that government was not trusted to perform its own monitoring role adequately, including on activities 15 A more limited conflict assessment was finalized in 2015, but it did not have direct operational implications. In IDA countries on the WBG FCS List, RRAs are required as part of the country engagement process to feed into CPFs. In the case of Afghanistan, the last CPF for Afghanistan is from FY17. 16 The World Bank has a strict definition for ineligible expenditures. While projects had fiduciary staff and followed the government PFM rules and regulations, the World Bank would only consider expenditures from the designated accounts eligible and accounted for if the TPM agent would review the statement of expenditures with the supporting documents, confirming that the expenditures were fully aligned with the relevant grant and financial agreements and agreed procedures. The World Bank therefore reports such a high number as either unaccounted for or not properly documented. 17 Acknowledging that this would require government consent. 19 taking place in areas of the country under its control.18 The TPM mechanism, critical as it was, in the end absolved the government of its responsibility to monitor its own expenditures. As the 2017 Scanteam evaluation report on ARTF states, “The Government aspires to take on more of the results monitoring and reporting responsibilities, but needs to show that it has the capacity, skills and political will . All Ministries have quality assurance units, and all project reporting uses GIRoA19 data and systems for basic information. When it comes to producing critical and in-depth studies, the view among donors is that Government is not yet pushing for independent and evidence-based reporting. While there is strong support for GIRoA-led quality assurance of activities, this role can only evolve as Government shows a will for increased transparency, openness and accountability, and capacity and quality of GIRoA monitoring bodies are put in place” (Scanteam 2017, 48). Full reliance on an effective but expensive TPM agent mechanism is one way of insulating a portfolio from the twin risks of weak government capacity and abuse of resources. However, it also reduces accountability and does not facilitate the institution building objective that was an explicit feature of the Afghanistan portfolio. The trade-off here is between ensuring that projects are delivered and remain on budget while ensuring fiduciary obligations are also respected, in a context where government was seen as insufficiently trustworthy to do so. Regardless of the above concern, some question whether traditional TPM agent verification is enough in a context where corruption is as endemic as in Afghanistan20 and whether there was an opportunity to engage more directly in the fight against corruption, including by making measurable actions against corruption a precondition for engagement and financing or expanding the approach to cordon off IDA and ARTF funds within country systems. This could have included mandating the TPM agent to identify the underlying reasons for ineligibility of public spending when it was conducting eligibility assessment for funding received by the government from the ARTF RCW and or under the Incentive Program. The argument here is that this could have pushed the government to enforce stronger controls on public spending and improved eligibility could be used as a precondition for future financing. While such steps might be worth considering in other contexts, we would argue that in Afghanistan this could have diluted government accountability even further and was not likely to produce results given the low trust environment, of which the discussion in itself is indicative. As a tenth lesson, combining an approach where full TPM agent is applied to most or at least all higher- risk projects while using a less comprehensive TPM agent arrangement as a safety valve (focusing on random controls and reviews) for a smaller number of defined projects in ministries with higher capacity 18 Several projects, including the critical CCAP community development project, provided support to communities in areas not controlled by the government and were tacitly or explicitly allowed to operate there. 19 GIRoA = Government of Islamic Republic of Afghanistan. 20 Afghanistan remained on the list of most corrupt countries during 2002–21, ranked 165 in 2020 and 174 in 2021 among 180 countries on the Transparency International Index. Afghanistan’s Country Policy and Institutional Assessment score for Accountability of the Executive remained at 1.5 out of 6. 20 (based on solid institutional assessments) is an option that could be considered in similar contexts in the future, especially in low trust contexts. 4. Connecting investment and development policy financing engagements The relation and compatibility between investment and development policy financing in Afghanistan was multifaceted and complex. At the best of times, they strongly supported each other, with projects closely aligned to, and supporting the achievement of, development policy indicators, but at other times concerns that projects may have undermined the more holistic approach to government reform, especially civil service reform (given the very high wage bill of extremely large PIU structures and their overall distortive effect on the civil service), were embedded in development policy approaches. While the balance between them and makeup changed, investment financing in projects has run alongside recurrent cost and development policy financing from the start. In 2002, following the Bonn Conference, the government had limited resources to pay civil servants and provide for the basic functions of the state. From 2002 through to 2019, the international community provided unconditional recurrent cost support to pay for these through the ARTF. In 2009, the ‘open check’ approach was gradually replaced by a more ‘conditions-based’ approach to incentivize critical structural and PFM reforms. The unconditional baseline support element of the package gradually petered out. From 2002 until its closure in 2019, the ARTF recurrent cost support operation contributed over US$5 billion to the government. During the last three years of its operation, 2017–19, it contributed some US$1.3 billion, over one-quarter of the total amount. Of this, only US$175 million consisted of unconditional baseline support. In addition, in most years, the World Bank has contributed development financing through its own IDA resources. From 2005 to 2017, around US$425 million was contributed in this way, annual amounts ranging between US$35 and 100 million to largely support PFM and civil service reform. From 2017, the World Bank moved to replace the ARTF operation with a standard development policy grant unifying both IDA and ARTF recurrent cost support under one project. From 2018 to 2021, annual Development Policy Grants accounted for around one-third of total annual World Bank and ARTF support to Afghanistan (around US$1.5 billion). Through both IDA and ARTF recurrent cost support, in the last five years the World Bank disbursed around US$2.5 billion in development policy financing to the government. The innovative blending of ARTF and IDA funds, post 2018, in a new hybrid Development Policy Operation helped focus scarce government capacity and align ARTF donors and the World Bank behind a single development policy instrument. The ICR for the 2018 Incentive Program (prepared in 2020) judged the overall outcome as Moderately Satisfactory. The ICR acknowledged the possibility of policy reversal and drew attention to the security situation and large parts of the country effectively beyond the reach of public servants. That reform progress achieved under the operation could be reversed if the security situation deteriorated further, including through (a) donor withdrawal and reductions in the capacity of international partners to support and finance implementation, (b) distraction of scarce political attention and implementation capacity away from sustaining reform progress and toward managing security risks, and (c) reduction in areas over which reforms could be implemented following further loss of territory to the government. 21 The high risks of the Development Policy Operations are further reflected in the supporting documentation to the final 2021 operation. The Project Appraisal Document (PAD) for the 2021 development grant operation largely sums this up: “The overall risk rating for the operation is high despite mitigation measures. Political and governance risks arise from the planned withdrawal of international security forces, overall weak governance, and uncertainties associated with the ongoing peace process. Macroeconomic risks arise from heavy aid dependence and uncertainties around future international grant support. Overall, institutional capacity for implementation and sustainability is weak, especially in the context of continued frequent changes in senior- level government personnel and institutional arrangements. Security risks are high, given the active-conflict environment” (paragraph 7, page 6). Within its own terms many aspects of the ARTF recurrent cost operation and its successor from 2018 were successful, including gradual improvements to domestic revenue collection over the period and improvements to PFM as well as provision for basic service delivery. The operation worked best when it was harnessed to closely aligned project interventions in support of key benchmarks. This happened critically for PFM investments, which were directly under the control of the Ministry of Finance but worked less well when the benchmarks were anchored in other ministries. High project staff turnover in ministries also undermined the momentum for reform, particularly when affected by wider political economy factors across the administration. At the same time, there were recurrent tensions between the ‘short term-ism’ of some investment projects, particularly the overuse of PIUs and contract staff, over the more medium- and longer-term institution building goals of development financing operations. The 2019 evaluation of the ARTF recurrent cost operation, while acknowledging the achievement of many of the operation’s results indicators, nevertheless concluded that “The complexity and ambition of the operation design and supported reforms were not always well-matched to the security and governance environment. The operation may have been strengthened through i) explicitly articulating assumptions regarding future political and security conditions; ii) reflecting these assumptions in a clear intervention logic; and iii) periodically testing these assumptions against actual developments and revising program design and intervention logic accordingly” (World Bank, 2019b). Because of the factors identified in the evaluation report, critical outcome-level results such as the development of a merit-based civil service remained elusive. Anti-corruption measures, such as asset declarations by senior officials and ministers, were in the end not successful in stemming widespread corruption across government. Reforms did happen, but they were never deep enough or far-reaching enough to change the nature of a state that had been shaped by the 2002 post-Bonn political settlement. Institution building is a long-term endeavor and impatience its principal enemy. As the 2011 World Development Report states “Creating the legitimate institutions that can prevent repeated violence is, in plain language slow. It takes a generation. Even the fastest performing countries have taken between 15 and 30 years…” (World Bank 2011, 10). Even this is optimistic. Afghanistan has had a long history of conflict, which has created deep schisms in society. These were never going to be overcome in a short span of time. The following are critical lessons for the World Bank on this program aspect: (a) budget support operations play an important role in supporting states in conflict but need to be evaluated carefully for the inherent potential political risks they contain in helping maintain an (undesirable) status quo; (b) the selected prior actions/indicators need to be focused and realistic in terms of local political economy constraints and supported by project investments that can work in tandem to achieve the same or linked objectives; (c) these operations provide a powerful way to reduce aid fragmentation and provide a single development policy platform for donors, but investment projects need to be aligned rather than working 22 against the reforms being sought; (d) in a stretched capacity environment like Afghanistan, budget support operations offer a way to focus scarce government capacity; (e) institution building takes time and it is important that the World Bank can stay the course despite setbacks on the way and the often slow progress in achieving results; and (f) it is important that Country Management Units (CMUs) can manage the tensions that periodically arise based on the short- and longer-term perspectives of different project teams. 5. Conclusions and recapping lessons Three critical issues stand out. One is inherently political, which is whether to engage in a broad manner in a context as fragile as Afghanistan, covering the range of World Bank instruments and sectors. This is a question of choice for the institution. As noted above, Afghanistan is not the only example of a broad engagement in a fragile or conflict setting. The other issues are related to how one engages effectively when a decision is taken to remain broadly engaged. Here there are three issues: one is of the triggers to propose scaling up or down, the second is of linking program implementation and institution building objectives, and the third is on the engagement model. 5.1 Linking diagnostic and risk management tools to automatic review triggers and advice to management The World Bank has invested significantly in diagnostic and risk management assessment and management tools. Some of these were unfortunately deployed late in Afghanistan. Others (like SORT) were used but suffered from an overly rosy (and eventually not corrected) assessment of risk and implementation capacity. One of the most important lessons from the ICRs and other reviews is the disconnect between the use of diagnostic and risk tools (and also political economy considerations) and portfolio management decisions. While decisions on maintaining or curtailing a portfolio and individual projects are the prerogative of the senior management, such decisions need to be based on information derived from candid and rigorous risk assessments. Afghanistan should have brought forward the RRA when conflict intensified. The scenario-based approach that emerged from it could have had an impact on decisions had the RRA been conducted in 2017–18 rather than in 2020. The same is true for portfolio risk reviews that insufficiently challenged conclusions on projects that were not performing and not owned. Risk mitigation was overly focused on fiduciary risks and not sufficiently on performance and implementation risks. It is not often that there is an opportunity to have a large body of evidence from a paused and suspended portfolio, and it is important that this opportunity be used to adjust the way risk assessment tools are deployed in high-profile FCV settings, including an early deployment of new tools, a rigorous use of portfolio and project risk tools like SORT, and a renewed effort to focus on political economy reviews as part of project design and implementation. 23 5.2 Ensuring a more coherent and sustainable approach to institution building in a conflict- affected setting This conclusion relates to the trade-off between ring-fencing projects from the normal workings of government to allow for better performance and building government systems. Most World Bank projects did the former, working through dedicated PIUs or through NTA (deploying around 6,000 contract staff), circumventing normal government processes and civil servants. Working around the constraints of weak government institutions became a critical project design principle. Project teams tried to minimize the effects of weak government capacity by working through PIUs and contracting staff as much as possible and even by weakening the fiduciary systems by adding further layers of control. The overall effect of this was to create enclaves of higher capacity and reduce leakages, but the consequence was to undermine the much slower civil service reform processes and displace corruption pressures to other parts of government and off-budget projects. ARTF donors were always surprised that the World Bank reported only minor irregularities with the very large ARTF program it managed in a country with one of the worst reputations for corruption in the world. But the fact was that World Bank and ARTF-financed programs were so tightly managed and scrutinized that there was little opportunity for large-scale corruption. This, however, does not mean that it did not happen, but that corruption did not reach egregious levels. But the costs of delivery in such an environment become very high. TPM contracts do not come cheap and the prospect of handing over the responsibility to the government remained remote. World Bank civil service reform efforts to rein in the burgeoning parallel civil service of contract staff on high salaries never succeeded. World Bank projects alone employed close to 6,000 non-civil-service Afghan contract staff in 2019. The entire government edifice was propped up by around 20,000 contract staff, mostly employed by the government at high wages from its discretionary budget. 21 The Afghanistan program combined contradictory and conflicting elements. On one hand, huge effort was made to build capacity and government systems as part of the state building agenda but on the other hand the program became dominated by workaround and often ad hoc solutions to working with weak government institutions. ACReMAP, TPM, and use of contract staff in PIUs were all ways to shore up weak institutions to provide reassurance to the World Bank and ARTF donors and to address lack of capacity. Despite its best intentions, it was difficult for the World Bank country management to hold the line on use of PIUs by its own staff when many ARTF donors, and the government itself through its discretionary budget, used non-civil-service contract staff. At the same time, the international community wanted sustainable and durable solutions to government fragility. The ARTF Incentive Program and, after 2017, the annual Development Policy Grants tried to introduce medium-term systemic reforms to government to enhance capacity and accountability. These often contradictory strands ran through the program and encapsulated the trade-off between short-term and longer-term development solutions to meet government demand and the often 21 In addition to the nearly 20,000 ‘on budget’ contract staff, a further 8,000–9,000 were employed in ‘off budget’ projects. 24 contradictory internal incentives within the World Bank and ARTF donors to invest in short-term projects, many of which among ARTF donors were ‘off budget’ but also to help build a functioning government. The importance of strengthening core institutions is highlighted in the FCV strategy (at the same time recognizing this can take decades). However, the evidence of the Afghanistan program shows how difficult it is to reconcile this longer-term objective with the short-term priority of implementing a broad and challenging program in a conflict-affected state. While the World Bank cannot be responsible for the actions of other development partners financing off-budget NTA and in this way undermining institution- building efforts, the majority of NTA was provided on budget under the World Bank’s own programs. As discussed previously in lesson seven, an alternative could be a blended approach by providing on-budget support, implemented through government systems, in certain well-defined sectors, and third-party execution of more complex and higher risk (but still on-budget) programs. This could be an effective way forward to make the two objectives more compatible. This should become a key feature of FCV engagements and strategy. 5.3 Adjusting the World Bank’s culture and business model to working in conflict-affected countries like Afghanistan Several of the lessons emerging from the analysis of project implementation in Afghanistan relate to the way the World Bank operates and its business model. Working in Afghanistan, which is one of the most complex and fragile contexts among FCV countries, required a different template from one adjusted to ‘normal’ low-income countries. In this regard, the World Bank model was ultimately limited by its relationship to the state, which decided who it could give money to and the projects it prepared and invested in. It was also under intermittent pressure to cover for domestic revenue shortfalls. As the state unraveled and its reach became more limited, the World Bank’s authorizing environment could have adjusted the program to these changes, including by exercising flexibility to downsize the portfolio, change plans and direction, and spend less money and diversify the channels through which money could be spent. Yet any flexibility that might have existed was constrained by the counterpart administration’s views on the nature of the state-citizen social contract, its ability to make these prevail in high-level dialogue, and the World Bank’s own internal incentive structures. All these factors combined to prevent adjustments that would have been applied under different circumstances. The situation was further complicated by the onset of COVID-19 in 2020, which masked the full extent of Taliban gains in the countryside and the increasing isolation of the government. Staff attention and time was taken up with restructuring projects to release money for COVID-19 rather than preparing a new strategic framework in light of the US peace agreement with the Taliban in February 2020. In the end, as the recent IEG review of World Bank operations in conflict and fragile states observes, there is a need for a more consistent approach and guidance. “There are, however, marked differences in operational responses to otherwise similar security instances. The World Bank does not provide concrete guidance to heads of mission on how to systematically process data on changes to conflict risk levels as they pertain to the country portfolio. Without this, responses to similar security incidents have varied depending on the risk tolerance of the head of office, and there is no process in place to foster an optimum approach” (World Bank 2021c, XIV). One answer to this conclusion is to invest more in senior staffing of FCV country offices, particularly staff with political economy and conflict analysis capacity, so 25 that these could be embedded as ‘public goods’ in critical project teams and provide better support to heads of mission by helping inform critical decisions on how to respond to changing conflict risk levels. Finally, a critical question remains on what an optimum approach might have looked like in Afghanistan: one that was not so beholden to the state (given that Afghanistan never had an all-directive central state but was always characterized by different layers of legitimacy and authority), that was more sensitive to the evolving political and conflict situation, and that was aware of the heightened level of risk to project implementation. An optimal approach also would have been to diversify channels of delivery and recognize the need to sharpen the focus on service delivery and poverty reduction while making the sustainability of its approaches critical, reducing the fragmentation of too many projects, and reducing the scale and ambition of some of the projects that were intended to transform the state. In the end, hindsight can offer important lessons learned but also needs to consider that at the time decisions on the program were taken, the future contours of Afghanistan were difficult to predict. 6. Looking ahead: Incorporating lessons in a new environment The reconstitution of engagement in Afghanistan comes with a fundamental change in approach to delivery; off budget, with a focus on ensuring basic services continue to be delivered and based on principles that are ‘markers’ for funding partners rather than the ITA. The contrast with the program discussed here could not be starker—on-budget, all-encompassing program, with extensive para- governmental project implementation structures and with additional external checks and balances through a portfolio-wide TPM system. However, the challenge is that both extremes (the one analyzed through the ICR and the one operating now) are in the end unsustainable. Hence, in this final section we will raise five critical questions that should help in feeding our reflection going forward. Question 1: Interpreting the FCV strategy on scope. While Afghanistan is not the only outlier among FCS country-level engagements, it was on the extreme side of the scale. The World Bank’s FCV strategy suggests carefully scaling engagement, given capacity constraints, and focusing on basic services first as well as macro stability and basic institutions. In programming the new engagements, this advice has been carefully heeded. It will be important, if in the future an opening arises, to continue a carefully phased approach. This is important especially given the internal dynamics described in the lessons section that will push toward broadening engagement more rapidly than may be good for the context. Question 2: Adjustability. One of the most critical issues identified in the lessons is the inability to ‘right the ship’ when programs were not delivering. Political constraints were part of the equation, as the World Bank operates in a highly political environment, something that needs to be acknowledged and understood. However, the lack of proper use of potential tools to analyze and address risks at the portfolio and project level and adjust projects stands out. The new engagements use tranching to mitigate such risks. Unlike past practice, the current engagements base tranche allocation decisions on a qualitative assessment and project performance rather than funding needs only. This is paired with the use of Entry Criteria for Access, which is a second monitoring tool to ensure all Afghans can benefit from the support provided. The consistent use of these mechanisms paired with more rigorous risk assessment should help drive greater adjustability to context. Of course, this is somewhat easier to do in implementation partnerships with UN agencies when compared to a government-implemented program, with its political constraints, yet the current setup provides an opportunity to enhance adjustability. 26 Question 3: Analytics and design. In the current context, analytical products come at a premium and their use to inform project design is a crucial one. The World Bank’s ‘Afghanistan Futures’ program includes critical survey work (supported by the TPM agent in its new blended role of supervision support and data gathering) as well as program-related analytical studies. In a context where there is (currently) no reliance on government-provided data, ensuring strong alignment between studies and project design is even more relevant that under the previous regime. Maintaining a strong quality control system to ensure these links should be a priority for the World Bank and its partners. Question 4: Sustainability. This is a critical question, as the off-budget implementation modality is not easily compatible with sustainability and even medium-term efforts at system building. The way this issue is currently being approached is by focusing on the lowest-level institutions that are still functioning (CDC, farmers collectives, and so on) and involving these in project implementation, with the hope that from there and in a bottom-up manner some of the previous delivery capacity will be sustained. The shift from an engagement that was predominantly central government oriented to a bottom-up approach might seem radical, but it does also represent continuity, as these same institutions were part and parcel of delivery before. The same is true for the health care delivery system. Eventually, and when the authorizing environment is favourable, building this out to engage with local governments in areas where such engagement is possible could be an important next step. Local governments traditionally had a limited involvement in donor delivery programs; hence, an investment in building such relations would need to be a deliberate and well-thought-through decision. Question 5: Thinking about transitioning, early on. Based on the experience in Yemen and South Sudan, among others, it could take a long time for support to transition back to on-system and on-budget. Yet, it is worth thinking now on what such a transition could eventually look like. Here we would like to come back to lesson seven in section 3 of this paper, which is about blended delivery models. These were not considered as an option previously, even if they are not incompatible with ‘on-budget’ principles. While the bottom-up approach to preserving systems and capacity will be a way of dealing with the concerns over sustainability, thinking through what a transition to using government systems in some areas could look like is equally important. Based on the experience, neither the current nor the previous extreme are the right recipe to deal with a severely capacity-constrained government and a blended model would be one that might hold promise. This would also be meaningful for discussions on how to reduce reliance of para-governmental PIU systems. These five questions and the tentative answers framed here based on the first six months of implementation under new modalities will need to be revisited regularly and linked with adjustments to engagement, in substance and in delivery model. This will ensure we take more of the positives from the experience gained from the previous portfolio and avoid more of the negatives and hence provide the most relevant and impactful support to the Afghan people. 27 Annex 1: Portfolio on August 15, 2021 % PROJECT MILESTONE DATES COMMITMENTS (US$, MILLIONS) BALANCE (US$, MILLIONS) UNDIS- # PROJECT NAME ID BURSED APPROVAL CLOSING TOTAL IDA ARTF OTHER TOTAL IDA ARTF OTHER 1 P160615 Afghanistan Sehatmandi Project 28-Mar-18 30-Jun-22 465.00 140.00 290.00 35.00 67.79 0.05 63.14 4.60 14.6 2 P173775 Afghanistan COVID-19 Emergency Response and Health Systems 02-Apr-20 31-Mar-24 163.40 160.40 0.00 3.00 101.42 98.42 0.00 3.00 Preparedness Project 62.1 3 P174348 Emergency Agriculture and Food Supply 04-Aug-20 31-Aug-22 100.00 55.00 45.00 0.00 86.85 48.61 38.24 0.00 Project 86.9 4 P159378 EQRA22 28-Sep-18 31-Dec-23 298.50 75.50 100.00 123.00 208.88 33.28 57.15 118.45 70.0 5 P132944 Naghlu Hydropower Rehabilitation 14-Dec-15 30-Sep-22 83.00 0.00 83.00 0.00 61.99 0.00 61.99 0.00 Project 74.7 6 P145347 Trans-Hindukush Road Connectivity 20-Oct-15 31-Dec-22 150.00 150.00 0.00 0.00 88.59 88.59 0.00 0.00 Project 59.1 7 P146184 Higher Education Development Project 07-Jul-15 21-Dec-22 55.00 0.00 55.00 0.00 10.77 0.00 10.77 0.00 19.6 8 P149410 CASA-1000 Community Support Program 24-Mar-14 30-Jun-22 40.00 0.00 40.00 0.00 8.40 0.00 8.40 0.00 (P149410) 21.0 9 P145054 CASA 1000 Electricity Transmission and 27-Mar-14 31-Mar-23 284.00 284.00 0.00 0.00 106.39 106.39 0.00 0.00 Trade Project 37.5 10 P156894 Afghanistan Digital CASA Project 28-Mar-18 31-Mar-23 51.00 51.00 0.00 0.00 46.51 46.51 0.00 0.00 91.2 11 P158768 Public-Private Partnerships and Public 27-Jun-18 28-Jun-23 27.81 10.00 17.81 0.00 19.00 7.69 11.31 0.00 Investment Advisory Project 68.3 12 P159655 Fiscal Performance Improvement 19-Dec-17 28-Dec-22 60.00 25.00 35.00 0.00 15.00 0.00 15.00 0.00 Support Project (FSP): 25.0 13 P160567 Citizens' Charter Afghanistan Project 27-Oct-16 31-Dec-22 865.00 262.70 602.30 0.00 320.17 69.16 251.01 0.00 37.0 14 P160619 Cities Investment Program 19-Dec-18 30-Jun-24 35.00 17.50 17.50 0.00 27.44 13.99 13.45 0.00 78.4 15 P162022 Herat Electrification Project 13-Jun-17 31-Jan-23 60.00 60.00 0.00 0.00 40.44 40.44 0.00 0.00 67.4 16 P164443 Women's Economic Empowerment 28-Sep-18 30-Jun-23 100.00 25.00 75.00 0.00 76.21 15.74 60.47 0.00 Rural Development Project 76.2 17 P164762 Afghanistan Land Administration System 25-Apr-19 30-Sep-24 25.00 17.86 7.14 0.00 19.01 13.64 5.37 0.00 Project 76.0 18 P166127 Afghanistan: Eshteghal Zaiee - 19-Dec-18 31-Dec-23 193.75 150.00 43.75 0.00 170.65 132.88 37.77 0.00 Karmondena (EZ-Kar) Project 88.1 19 P166978 Tackling Afghanistan’s Government 19-Dec-18 31-Dec-22 55.00 18.40 36.60 0.00 25.62 8.92 16.70 0.00 HRM23 and Institutional Reforms 46.6 20 P168266 Payments Automation and Integration of 25-Apr-19 31-Oct-24 25.00 5.60 19.40 0.00 20.07 4.58 15.49 0.00 Salaries in Afghanistan (PAISA) 80.3 22 EQRA = Education Quality Reform in Afghanistan. 23 HRM = Human Resource Management. 28 % PROJECT MILESTONE DATES COMMITMENTS (US$, MILLIONS) BALANCE (US$, MILLIONS) UNDIS- # PROJECT NAME ID BURSED APPROVAL CLOSING TOTAL IDA ARTF OTHER TOTAL IDA ARTF OTHER 21 P169970 Afghanistan Water, Sanitation, Hygiene 11-Dec-20 29-Jan-26 200.00 50.00 150.00 0.00 196.35 50.10 146.25 0.00 and Institutional Support Project 98.2 22 P171886 Strengthening Afghanistan’s Financial 28-Sep-20 31-Oct-25 100.00 100.00 0.00 0.00 73.72 73.72 0.00 0.00 Intermediation 73.7 23 P172109 Afghanistan Gas Project (AGASP) 26-Nov-19 31-Oct-24 52.50 52.50 0.00 0.00 50.68 50.68 0.00 0.00 96.5 24 P166405 Sheberghan Gas to Power Project 10-Oct-19 01-Mar-25 12.80 12.80 0.00 0.00 12.80 12.80 0.00 0.00 100.0 25 P157827 Mazar-e-Sharif Gas-to-Power Project 10-Oct-19 10-May-25 12.00 12.00 0.00 0.00 12.00 12.00 0.00 0.00 100.0 26 P174119 COVID-19 Relief Effort for Afghan 04-Aug-20 31-Dec-21 280.00 155.00 125.00 0.00 109.45 73.83 35.62 0.00 Communities and Households (REACH) 39.1 27 P176137 2021 Afghanistan Incentive Program 24-Jun-21 30-Jun-22 432.00 132.00 300.00 0.00 300.00 0.00 300.00 0.00 Development Policy Grant 69.4 Total 4,225.76 2,022.26 2,042.50 161.00 2,276.20 1,002.02 1,148.13 126.05 54.0 Note: P173387 - Early Warning, Early Finance and Early Action Project (ENETAWF); P171657 - Incentivizing Reforms in the Attorney General's Office (IRAGO); and P174119 - Covax ARTF US$50 million not included in the above table. 29 Annex 2: Project Risk Ratings at Appraisal Year of Overall Risk24 at Number of SORT Categories Project Approval Approval Marked High out of 10 Citizens Charter 2016 High 3 Herat Electrification 2017 High 3 Fiscal improvement 2017 High 5 Sehatmandi 2018 High 3 Digital CASA 2018 Substantial 4 Public-Private Partnerships 2018 High 5 EQRA 2018 Substantial 3 Women’s Economic Empowerment 2018 Substantial 1 Eshteghal Zaiee-Karmondena 2018 High 5 Tackling HRM and Institutional reform 2018 High 6 Cities Investment 2018 High 4 Land Administration 2019 High 6 Payments Automation (PAISA) 2019 High 7 Gas Project 2019 High 5 COVID-19 Emergency response 2020 High 4 COVID-19 Reach 2020 High 3 Emergency agriculture & Food Supply 2020 n.a. 3 Financial Intermediation 2020 Substantial 3 2021 Afghanistan Incentive Program 2021 High 5 Development Policy Grant (IP-DPG) 24 This is a residual (non-mitigable) risk. The overall risk rating is not a sum or average of the individual category risk ratings. 30 Annex 3: Data sheet on ICR rating25 Outcome Efficacy S. Pre Post Pre Post Risk to Project PDO Overall Overall M&E No. Project Name August 15, August August 15, August Bank Development Code Relevance Outcome Efficacy Efficiency Quality 2021 15, 2021 2021 15, 2021 Performance Outcome 1 P174119 COVID-19 Relief Effort for Afghan H MS MS MU SU SU M SU H S H Communities and Households (REACH) 2 P160615 Afghanistan Sehatmandi Project H MS SU M M SU M SU SU S H 3 P149410 CASA-1000 Community Support SU MU MU U — M N M SU MS H Program (P149410) 4 P176137 2021 Afghanistan Incentive Program MS HU — — HU — — — — MS H Development Policy Grant 5 P174348 Emergency Agriculture and Food H U MU U — M N M SU MS SU Supply Project 6 P146184 Higher Education Development H S HS U — H N H H HS H Project 7 P159655 Fiscal Performance Improvement H MU MS MU — M N M S S H Support Project (FSP): 8 P145347 Trans-Hindukush Road Connectivity SU HU Phase 1: U HU N Phase 1: M N N M MU H Project Phase 2: HU Phase 2: N 9 P166978 Tackling Afghanistan’s Government SU MU MS U M SU N SU M MS H HRM and Institutional Reforms 10 P160567 Citizens’ Charter Afghanistan H MS S MU SU SU M SU H S H Project 11 P162022 Herat Electrification Project H MU S U — S N Pre: SU SU MS H Post: M Overall: SU 12 P158768 Public-Private Partnerships and SU U U U N M N N M MS H Public Investment Advisory Project 13 P164443 Women's Economic Empowerment H MU S U — SU N SU SU S H Rural Development Project 14 P159378 EQRA H MS S MS SU SU M SU SU MS S 15 P166127 Afghanistan: Eshteghal Zaiee - H U MU U N M N M SU MU H Karmondena (EZ-Kar) Project 16 P173775 Afghanistan COVID-19 Emergency H MU MS U — SU N M SU S H Response and Health Systems Preparedness Project 17 P160619 Cities Investment Program H U MU U — M N M SU MS H 18 P164762 Afghanistan Land Administration H U Phase 1: U U — Phase 1: N N M SU MS H System Project Phase 2: MU Phase 2: M 25 All ICR ratings are pending IEG review subsequent to Board submission and are therefore not final. 31 Outcome Efficacy S. Pre Post Pre Post Risk to Project PDO Overall Overall M&E No. Project Name August 15, August August 15, August Bank Development Code Relevance Outcome Efficacy Efficiency Quality 2021 15, 2021 2021 15, 2021 Performance Outcome 19 P168266 Payments Automation and SU U MU U N M N N N MS H Integration of Salaries in Afghanistan (PAISA) 20 P172109 Afghanistan Gas Project (AGASP) SU U U U N N N M SU MS H 21 P171886 Strengthening Afghanistan’s SU U MS U N M N Pre: M M MS H Financial Intermediation Post: N Overall: N 22 P169970 Afghanistan Water, Sanitation, — — — — — — — — — MU — Hygiene and Institutional Support Project 24 P156894 Afghanistan Digital CASA Project SU U U U N — — N N MU S 23 P132944 Naghlu Hydropower Rehabilitation H MU S — M H — SU SU MS — Project Key Details H High SU Substantial MS Moderately Satisfactory MU Moderately Unsatisfactory HU Highly Unsatisfactory U Unsatisfactory S Satisfactory HS Highly Satisfactory M Modest N Negligible 32 References Afghanistan Islamic Republic. 2014. “Realizing Self-Reliance.” Paper presented to London Conference on Afghanistan, December 4, 2014. Afghanistan Islamic Republic. 2016. Afghanistan National Peace and Development Framework. Aleph Strategies. 2021. Independent Review Third Party Monitoring Agent Performance . World Bank. ARTF (Afghanistan Infrastructure Trust Fund). 2018. Partnership Framework and Financing Program, 2018–2020. World Bank. ARTF. 2020. Scorecard. World Bank. Asia Foundation. 2019. Survey of the Afghan People. Washington, DC: Asia Foundation. Azizi, Makhfi. 2021. Afghan Views on the Doha Peace Process. Atlantic Council, February 25, 2021. Bedoya Guadalupe, Aidan Coville, Johannes Haushofer, Mohammad Isaqzadeh, and Jeremy Shapiro. 2019. “No Household Left Behind Afghanistan Targeting the Ultra Poor Impact Evaluation.” Policy Research Working Paper 8877, World Bank, Washington, DC. Beath, Andrew, Christia Fotini, and Ruben Enikolopov. 2017. “Can Development Programs Counter Insurgencies? Evidence from a Field Experiment in Afghanistan.” MIT Political Science Department Research Paper No. 2011-14. https://ssrn.com/abstract=1809677 or http://dx.doi.org/10.2139/ssrn.1809677. Byrd, William. 2020a. “An Essential for Afghan Peace: Funding the Government.” United States Institute for Peace, Washington, DC. Byrd, William. 2020b. “Dismembering Afghanistan’s Ministry of Finance.” United States Institute of Peace, Washington, DC. Central Statistical Organization. 2018. Afghanistan Living Conditions Survey, 2016-2017. Kabul, Afghanistan. Farahi, K., and S. Guggenheim. 2020. Pathways for Post-Peace Development in Afghanistan. Washington, DC: United States Institute of Peace. Federal Ministry of Economic Cooperation and Development. 2018. Meta-Review of Evaluations of Development Assistance to Afghanistan, 2008–2018. Berlin: BMZ. Feroz, Emran. 2020. “Afghan Warlord’s Promotion Highlights the Bankruptcy of America’s Longest War.” Foreign Policy, July 17, 2020. Hogg, R., C. Nassif, J. Osorio, Byrd, and A. Beath. 2014. Afghanistan in Transition: Looking beyond 2014. World Bank, Washington, DC. Hogg, R., S. O’Meally and J. Di John. 2017. Social Service Delivery in Violent Contexts: Achieving Results against the Odds. World Bank, Washington DC. Independent Commission for Aid Impact. 2022. UK Aid to Afghanistan. ICAI London. Landers, C., and R. Aboneaaj. 2021. “World Bank Budget Support in Time of COVID: Crisis Finance with Strings Attached.” Center for Global Development Policy Note. Ministry of Rural Rehabilitation and Development. 2021. High Risk Area Implementation Strategy, Citizens Charter Afghanistan Project. 33 OECD. 2012. Development Cooperation Report. Paris: OECD Vincent, A. Floreani, Gladys López-Acevedo, and Martín Rama. 2012. “Conflict and Poverty in Afghanistan’s Transition.” Policy Research Working Paper 7864, World Bank, Washington, DC. Scanteam. 2017. ARTF at a Cross-Roads: History and the Future; Taking Charge: Government Ownership in a Complex Context. Oslo: Scanteam Analysts and Advisers. SIGAR (Special Inspector General for Afghanistan Reconstruction). 2021. What We Need to Learn: Lessons from 20 Years of Afghanistan Reconstruction. SIGAR. 2022. Collapse of the Afghan National Defense and Security Forces. Interim Evaluation Report. UNAMA (United Nations Assistance Mission in Afghanistan). 2017. Special Report on the Strategic Review of the United Nations Assistance Mission in Afghanistan. A/72/312–S/2017/696. UNDP (United Nations Development Programme). 2020. Human Development Report: The Next Frontier. World Bank. 2011. World Development Report: Conflict, Security and Development. Washington, DC: World Bank. World Bank. 2013. Afghanistan Country Program Evaluation, 2002-2011. Washington, DC: Independent Evaluation Group, World Bank. World Bank. 2016a. Country Partnership Framework for Afghanistan, FY17-FY20. Washington, DC: World Bank. World Bank. 2016b. Systematic Country Diagnostic. Washington, DC: World Bank. World Bank. 2017a. Iraq - Completion and Learning Review of the Country Partnership Framework: IEG Review. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/965801628 861117233/Iraq-Completion-and-Learning-Review-of-the-Country-Partnership-Framework-IEG- Review. World Bank. 2017b. Social Service Delivery in Violent Contexts: Achieving Results against the Odds. Washington, DC: World Bank. World Bank. 2018a. Pathways for Peace: Inclusive Approaches to Preventing Violent Conflict . Washington, DC: World Bank. World Bank. 2018b. Yemen Portfolio Closure Note. Washington, DC: World Bank. World Bank. 2019a. Performance Learning Review of Afghanistan Country Partnership Framework, FY17- FY20. Washington, DC: World Bank. World Bank. 2019b. Afghanistan Reconstruction Trust Fund. Evaluation of the Recurrent and Capital Cost Operation. Washington, DC: World Bank. World Bank. 2020a. Navigating a Seas of Uncertainty, Afghanistan Development Update . Washington, DC: World Bank. World Bank. 2020b. Fragility and Conflict: On the Front Lines of the Fight against Poverty. Washington, DC: World Bank. World Bank. 2020c. Strategy for Fragility, Conflict and Violence 2020-2025. Washington, DC: World Bank Group. World Bank. 2021a. Afghanistan Country Update, Issue 058. April 2021. 34 World Bank. 2021b. Afghanistan Risk and Resilience Assessment. Washington, DC: World Bank. World Bank. 2021c. World Bank Engagement in Situations of Conflict, An Evaluation of FY10-20 Experience. Washington, DC: Independent Evaluation Group, World Bank Group. 35