Approach Paper An Evaluation of World Bank and International Finance Corporation Engagement for Gender Equality over the Past 10 Years October 20, 2023 This Approach Paper proposes an independent evaluation of the results achieved by the World Bank and the International Finance Corporation (IFC) in supporting countries (understood as governments, private sector, civil society, and citizens at large) to address gender inequalities and the contribution of the gender strategy for fiscal years (FY)16–23. 1. Background and Context 1.1 The FY16–23 World Bank Group gender strategy advanced a well-established corporate gender agenda (World Bank Group 2015). Key developments in the longer- term trajectory of this agenda include the 2002 gender strategy (World Bank 2002a) and Gender and Development Policy Framework Guidance Note (World Bank 2002b), the 2007–10 gender action plan (World Bank 2011), the World Development Report on gender equality and development (World Bank 2012), and the establishment of gender as a special theme in every International Development Association (IDA) Replenishment since the 16th Replenishment. The most recent evolution of this agenda, after the launch of the FY16–23 gender strategy, includes the development of the IFC Gender Strategy Implementation Plans I (for FY17–19) and II (for FY20–23); the gender-based violence (GBV) and sexual exploitation, abuse, and harassment action plan (World Bank 2017); the Multilateral Investment Guarantee Agency (MIGA) gender strategy (MIGA 2021); and special attention to gender inequalities in the strategy for countries affected by fragility, conflict, and violence (FCV; World Bank Group 2020). The latest developments were also accompanied by an increased emphasis on the gender implications of climate change, a growing awareness of and progress on the sexual orientation and gender identity agenda, 1 and the repositioning of the gender group in the World Bank (from the 0F Equitable Growth, Finance, and Institutions Practice Group to the Human Development Practice Group) and (several times) in IFC, which, in FY24, established a Directorate for Gender and Economic Inclusion Solutions housed under the Cross-Cutting Vice Presidency. 1.2 The FY16–23 gender strategy was the first joint World Bank–IFC gender strategy. When it launched, the Bank Group and the whole development world had just embraced the Sustainable Development Goals, which made achieving gender equality integral to each of the 17 goals and the focus of the fifth goal. 2 The Bank Group also 1F 1 adopted the twin goals and committed to supporting countries and companies to promote gender equality as a key pathway to lasting poverty reduction and shared prosperity. Since 2006, IFC has adopted an approach to reduce gender inequalities through its dedicated advisory, investments, and research efforts, but it was with the FY16–23 gender strategy that it was formally included in a corporate (Bank Group) gender strategy for the first time. 1.3 The launch of the FY16–23 gender strategy represents a milestone of an established but still evolving agenda that signals the Bank Group’s commitment to gender equality. The strategy is the most recent segment in a longer evolution of conceptual frameworks, operational approaches, and commitments to gender equality. Box 1.1 summarizes the evolution of the gender agenda in the Bank Group, from the introduction of a Women in Development adviser in 1977 to the preparation of the new gender strategy for 2024–30, which is currently ongoing. The fact that each gender strategy builds on previous activities, achievements, and lessons has important implications for their evaluation. Box 1.1. Evolution of World Bank Group Strategic Approach to Gender Equality In 1977, the World Bank was the first multilateral institution to appoint a Women in Development adviser, two years after the United Nations First World Conference on Women. In 1984, the World Bank issued Operational Manual Statement 2.20, which called for World Bank staff to consider women’s issues as part of the social analysis undertaken during the appraisal of an investment project and to assess and address through project design any disadvantageous effects on women. In 1987, a Women in Development unit was established in the central Population and Human Resources Department, and in 1990, regional Women in Development coordinator positions were created. In 1990, an operational directive on poverty reduction recommended that women’s issues be considered when designing poverty reduction programs. At its 1994 meeting, the World Bank Board of Executive Directors discussed and approved a strategy paper titled “Enhancing Women’s Participation in Economic Development” (World Bank 1994). This strategy paper reflected the significant shift in the global agenda for gender equality that would become apparent a year later at the Fourth World Conference on Women in Beijing, China, and marks the switch from the early women in development programs (which treated women as a special target group of beneficiaries in projects and programs) to the gender and development approach (which highlights women’s contributions to development while challenging the structures that maintain gender roles, relations, and hierarchies). The discussion led to World Bank Operational Policy 4.20: The Gender Dimension of Development (OP 4.20), originally established in 1994, revised several times since then, and still operational today. OP 4.20 required staff to “reduce gender disparity and increa se the participation of women in the economic development of their countries” (World Bank 2005, ix) as part of its mandate to reduce poverty. OP 4.20 acknowledged that to reduce poverty effectively 2 and in a sustainable manner, development assistance needed to be designed to address the differential impacts of development interventions on women and men. OP 4.20 marked the shift from the women in development approach—focusing exclusively on women, their roles, responsibilities, and needs, without regard to the gendered power relationship between men and women within which these were embedded—to a broader and structural integration of gender issues into World Bank assistance, which would ultimately increase women’s empowerment and participation in economic development. OP 4.20 underscored the importance of country ownership and commitment for World Bank support to be effective in helping reduce gender disparities. Two points of entry—one through the country strategy and the other through project appraisal—provided the means to integrate gender considerations into World Bank support for the next few years. In 1995, the World Bank president participated in the Fourth World Conference on Women in Beijing. In 1996, the External Gender Consultative Group was established. The group comprised 14 members, including representatives of national women’s organizations, nongovernmental institutions, and political organizations from around the world. These members were to assist the World Bank in designing and implementing gender policies and help strengthen gender-related dialogue with World Bank partners and interested sectors of civil society. In 2011, this external consultative body was relaunched as the Advisory Council on Gender and Development. The council meets twice a year to consider progress on, and constraints to, gender equality globally and to provide feedback and advice on the World Bank’s work in this area. In 1997, the Gender and Development Unit was created as part of the Poverty Reduction and Economic Management Network. The Gender and Development Board—with the Gender and Development Unit as its secretariat—was made responsible for overseeing knowledge management, monitoring and reporting on the status of policy implementation, and building capacity. In April 2001, the World Bank strengthened the country-level approach through a gender strategy paper—Integrating Gender into the World Bank’s Work: A Strategy for Action —discussed by the World Bank Group Board; the strategy went into effect on January 1, 2002, and its goal was to mainstream “gender-responsive actions into the development assistance work of the World Bank” (World Bank 2002a, ix). This strategy introduced the use of a new diagnostic tool — the Country Gender Assessment—and promised that a Country Gender Assessment for every active borrower would be completed by fiscal year (FY)05. Country Gender Assessments were expected to inform country strategies, which were to include, as relevant, gender-responsive interventions. Sector managers and task teams were to ensure that gender considerations received appropriate treatment in operations in sectors and for themes that the country strategy identified as priorities for gender. In 2007, the World Bank launched a four-year gender action plan—Gender Equality as Smart Economics—which closed in 2011. Recognizing shortfalls in gender integration at the operational level in selected sectors, the action plan encouraged gender mainstreaming by providing staff with incentives to integrate gender into predetermined sectors and activities—in particular, agriculture, infrastructure, the labor market, entrepreneurship, and the private sector. In 2010, gender was chosen as a special theme of the 16th Replenishment of the International Development Association. It has remained a special theme in all International Development Association Replenishments ever since, including in the current 20th Replenishment cycle (2022–25). 3 The 2012 World Development Report topic was gender equality and development, which argued that closing gender gaps is important but can also enhance productivity and improve development outcomes. The International Finance Corporation global gender team started in 2013 as the Gender Secretariat. In 2013, the Umbrella Facility for Gender Equality was established. It is the only multidonor trust fund dedicated to financing data and evidence to support policy dialogue and identify what works to close gender inequalities. In 2014, Gender was created as one of the cross-cutting solutions areas. Originally a separate unit, Gender was later placed under the Equitable Growth, Finance, and Institutions Practice Group and was more recently moved under the Human Development Practice Group. In December 2015, World Bank Group Gender Strategy (FY16–23): Gender Equality, Poverty Reduction, and Inclusive Growth was presented to the Bank Group Board. The new strategy identified priority areas for engagement and set up a new methodology to measure results (World Bank Group 2015). In 2016, the International Finance Corporation developed a three-year Gender Strategy Implementation Plan I for FY17–19 and, in 2019, the Gender Strategy Implementation Plan II for FY20–23 to set the direction for the second phase of the gender strategy implementation. In 2016, the World Bank appointed the first global adviser on sexual orientation and gender identity, with the purpose of integrating sexual orientation and gender identity inclusion and nondiscrimination in all World Bank–financed development projects. At the end of 2017, a gender-based violence and sexual exploitation, abuse, and harassment action plan was introduced, after those issues were brought to the fore by the Uganda Transport Sector Development Project Inspection Panel case in September 2015. This action plan is now considered part of the Environmental and Social Framework implementation. In FY21, the Multilateral Investment Guarantee Agency launched its first Gender Strategy Implementation Plan (2021–23), which identified opportunities for increased gender actions aligned with three strategic pillars: corporate, client engagement, and partnerships. In July 2023, the draft of the new 2024–30 World Bank gender strategy was discussed at the Committee on Development Effectiveness of the World Bank. Starting from FY24, the International Finance Corporation has the Gender and Economic Inclusion Solutions department, led by a director. Sources: Independent Evaluation Group; Cortez, Arzinos, and de la Medina Soto 2021; various corporate documents; World Bank 1994, 2002a, 2005, 2009; World Bank Group 2015. Main Features of the World Bank Group’s Gender Strategy 1.4 The FY16–23 gender strategy introduces three innovative elements in the Bank Group’s conceptual approach to gender equality: a paradigm change regarding gender mainstreaming, a renewed focus on knowledge generation, and a new approach to identifying and monitoring projects designed to address relevant gender gaps. 4 1.5 First, the strategy aims to overcome gender mainstreaming and to focus instead on closing clearly identified gender gaps under four strategic objectives (or pillars): (i) improving human endowments, (ii) removing constraints for more and better jobs, (iii) removing barriers to women’s ownership and control of assets, and (iv) enhancing the voice and agency of women and engaging men and boys. 3 According to the original 2F strategy document, these objectives were to be contextualized in FCV environments and in light of the increasingly pressing threat of climate change. The latest developments called on the Bank Group to also prioritize gender gaps that the COVID-19 pandemic widened again after many countries had experienced improving trends (figure 1.1). 1.6 Second, the strategy increases the Bank Group’s emphasis on the role of knowledge generation and evidence-based approaches to inform operations and underpin policy dialogue. The strategy emphasizes two types of research: (i) the production of evidence to identify gaps, problems, patterns, and trends in each specific context and (ii) the production of impact evaluations (especially based on randomized controlled trial design) to identify and test “gender-smart” solutions. In relation to this latter type of knowledge, the strategy assigns a critical role to the Gender Innovation Labs. 1.7 Third, the strategy refines the approach to monitoring progress. The strategy made provisions for introducing a new gender tag for World Bank operations. 4 The3F gender tag was conceived as a tool to identify project-relevant gender gaps and “how they align with those identified through the country engagement framework” (World Bank Group 2015, 87). The gender tag is assigned by the gender group after the project is approved. To receive the gender tag, an operation is required to have a complete results chain of gender gap analysis, interventions, and indicators. The World Bank introduced upstream support to teams during project preparation to tag projects. Operational teams were also expected to use relevant indicators built into the results framework to monitor gender gaps during supervision (through Implementation Status and Results Reports) and at completion (through Implementation Completion and Results Reports). The IFC gender flag was created in 2013, but its uptake began in FY16 for advisory services (after the introduction of a scorecard target) and in FY19 for investments (after the capital increase commitments). The gender flag indicates whether an investment or advisory project is designed and implemented with explicit focus on creating equal opportunities for women and men and uses the same definition and methodology as the World Bank’s gender tag. An ex post quality review process for gender-flagged projects began in FY19. 5 Figure 1.1. The Four Strategic Objectives of the Gender Strategy for Fiscal Years 2016–23 Source: World Bank Group Gender Strategy Implementation Update, PowerPoint presentations to the Board of Executive Directors, various years. 6 1.8 The Bank Group gender strategy relies on five operationalization pathways (World Bank Group 2015). First, the strategy reiterates that the Bank Group should deepen its country-driven approach to gender equality and ensure that the objectives to promote gender equality are set at the country level. This goal requires strengthening the availability and use of country-level sex-disaggregated data and ensuring alignment between gender equality objectives and the instruments to pursue these objectives (“deepening the country-driven approach”). Second, the institution commits to developing a robust evidence base through (i) analytical work that identifies critical issues and gaps and (ii) impact evaluation that provides evidence of what works for gender equality in different contexts (“developing a better understanding of what works”). Third, gender-smart practices, or approaches that work, need to become the norm, which requires greater awareness, improved staff capacity, and increased alignment and collaboration across sectors (“building on what works”). Fourth, a stronger monitoring system needs to guide project teams to identify gender gaps that projects can address, select specific indicators to measure outcomes or impacts, monitor the evolutions of those indicators, and assess performance at project completion (“establishing a strategic approach to mainstreaming that helps achieve results”). Finally, the strategy recognizes that the Bank Group is part of “a global, regional, and country ecosystem of partnerships” and that it needs to collaborate with other partners that are also working to address gender gaps (“leveraging partnerships for effective outcomes”). 1.9 The strategy largely builds on previously established operational mechanisms. This evaluation recognizes that it is not possible to causally attribute results and achievements to the gender strategy, separate from the overall trajectory of the Bank Group strategic approach to gender equality. The country-driven approach to gender equality, for example, has not been introduced by the gender strategy. The World Bank Operational Policy 4.20: The Gender Dimension of Development underscores the importance of country ownership and identifies the country strategy as one of the two entry points for addressing gender inequalities (the other one being the project). This evaluation will therefore focus on assessing the overall achievements of the Bank Group in addressing gender inequalities from FY12 (when the World Development Report on gender equality was published) to FY23 and, in this framework, the contribution of the gender strategy. This longer time frame can help discern any change that can be associated with the adoption of the gender strategy. 1.10 The strategy applies to both the World Bank and IFC, and hence to both the public and private sectors. IFC’s approach to address gender inequalities includes multiple instruments of engagement with clients (research, investments, advisory services, and peer learning platforms), collaboration with internal and external partners, 7 and coordination of efforts at the global, regional, sectoral, and country levels. The strategy explicitly highlights IFC’s work to support women’s entrepreneurship (for example, by providing credit through financial institutions and expanding opportunities in supply chains) and gender equality in the workplace (for example, by improving employment opportunities and working conditions, including addressing GBV). Although the strategy covers both institutions (World Bank and IFC) and identifies areas of collaboration (for example, in knowledge generation and piloting of approaches), IFC clients are private sector firms, and the evaluation approach will take this into account. 1.11 A new gender strategy for 2024–30 will be launched in calendar year 2024. The early draft suggests that the new strategy largely builds on the previous one, with an increased focus on GBV, childcare and eldercare, digital solutions, mobility, and engagement of women as leaders (World Bank 2023c). The new strategy plans to cover sexual and gender minorities and expand attention to intersectionality. Its theory of action emphasizes innovation, financing, and acting collectively (using partnerships and stakeholder engagement). The new strategy also commits to strengthening the accountability framework by complementing the gender tag and gender flag with monitoring at implementation and closing and to enhancing gender outcome orientation in country engagement. This evaluation will focus on the FY16–23 gender strategy and the earlier period and aims at deriving findings and implications that are relevant to the forthcoming new strategy. The Independent Evaluation Group Mid-Term Review 1.12 The Independent Evaluation Group (IEG) conducted a Mid-Term Review of the FY16–23 gender strategy at about its midpoint cycle, focusing on implementation processes. The Mid-Term Review consisted of a rapid assessment of how well the implementation of the gender strategy was progressing (FY19 included), with the goal of positioning the Bank Group to better contribute to closing key gender gaps in client countries (World Bank 2021). The review adopted a qualitative approach supplemented by descriptive statistical analysis. It provided insights on commitment to country-driven approaches, coordination of gender expertise, and measurement of progress. It also analyzed what was working more and less well in supporting continuous monitoring and learning in the World Bank and IFC. 1.13 IEG’s Mid-Term Review revealed progress upstream and exposed opportunities for improvement at the level of strategy operationalization. According to the Mid-Term Review findings, Bank Group management, staff, and partners showed commitment to the gender strategy. Some visible progress included the emergence of good practices in implementing the strategy in Global Practices (GPs), industry groups, Regions, and countries and improvements in project design. This process was partially driven by 8 increased gender tagging and flagging. However, implementation actions did not consistently match this commitment, nor were they commensurate with the level of ambition of the strategy. The implementation of the country-driven approach advocated by the strategy had been hindered by competing priorities, limited familiarity with the gender gap approach, and variability in the support models across Regions. 1.14 This evaluation will complement the findings of the IEG Mid-Term Review. As the IEG Mid-Term Review mostly focused on processes, this evaluation will focus more on results, as discussed in chapter 3. 2. Objectives and Audience 2.1 The main objective of this evaluation is to assess (i) the results achieved by the World Bank and IFC in helping countries (understood as governments, the private sector, civil society, and citizens at large) address gender inequalities and (ii) the contribution of the FY16–23 gender strategy. The findings of this evaluation will complement the findings from the IEG Mid-Term Review and other relevant evaluations, such as the 2023 IEG evaluation that assessed Bank Group support in promoting women’s economic empowerment and addressing GBV in FCV contexts (World Bank 2023a). Moreover, this evaluation will be conducted in parallel with an IEG evaluation of Bank Group support to jobs and labor market reforms through IDA financing, which will assess how the Bank Group has been supporting inclusive access to jobs, with a focus on gender. The findings of this evaluation will inform future gender-related evaluative work and identify the constraints to address gender inequalities. Ultimately, this will contribute to ongoing discussions about realistic goal- and target-setting in relation to gender equality at the corporate, country, and project levels and support the operationalization of the new 2024–30 gender strategy. 2.2 This evaluation aims to inform a broad range of stakeholders who have been engaged during the preparation and implementation of the Bank Group’s new 2024–30 gender strategy: • The World Bank gender group and the IFC Gender and Economic Inclusion Solutions department (to assist them in defining and implementing the institutional approach to promoting gender equality and, more specifically, to provide evidence useful for the preparation of the new gender strategy, planned to be launched in January 2024). • The Bank Group Board of Executive Directors and Bank Group senior management tasked with making strategic decisions regarding corporate priorities, including on the evolution of the institutional gender agenda and 9 gender-related commitments, and with setting up incentives to attain those priorities. • Bank Group sector and country teams—staff and managers within units tasked with implementing the gender strategy, Operations Policy and Country Services, Country Management Units, GPs, and IFC regional and industry groups. • The IDA Deputies (the representatives of countries that contribute to IDA) and the IDA borrower countries because several IDA commitments are strategic gender commitments. • Governments officials in ministries and implementing agencies of client countries that are assisted by the Bank Group in advancing gender equality. • External stakeholders and partners, including other multilateral development banks, international organizations (including United Nations agencies), multilateral donors (such as the European Union) and bilateral donors, international nongovernmental organizations and foundations, national civil society organizations, national gender equality bodies, women’s organizations and networks, local authorities, private sector players, think tanks, and researchers (to inform them on progress being made and work to be done by the Bank Group in support of gender equality and the World Bank’s and IFC’s role and comparative advantage to inform collaborations and coordination efforts toward achieving common objectives). 2.3 MIGA, while excluded from the scope of this evaluation, will find the results of the evaluation useful as it is refining the operational plan of its Gender Strategy Implementation Plan FY21–23 (MIGA 2021), which will be included in the new 2024–30 gender strategy. 3. Evaluation of Theory of Action, Questions, and Scope 3.1 The evaluation questions and scope were derived from consultations conducted during the evaluation inception phase and from the review of strategic documents and the gender strategy’s theory of action. To identify the evaluation questions and define the scope, the team conducted semistructured interviews with representatives of the Board, Bank Group management, operational staff, and gender advisers and focal points. IEG also met with the former and current management of the gender group and the IFC gender team to ensure that the evaluation questions and design maximize learning and that the evidence generated is relevant and useful for the preparation of the new gender strategy and, more generally, for a better definition of results and increased probability of achieving them. The evaluation questions are based on the theory of 10 action of the FY16–23 gender strategy but are broad enough to apply to the whole Bank Group gender program. Theory of Action 3.2 The theory of action proposed by the FY16–23 gender strategy document is represented in figure 3.1. As discussed in chapter 1, this theory of action has some original elements, but it largely builds on previous strategic approaches. The Bank Group allocates inputs (figure 3.1, column one) to ensure that activities are undertaken at the corporate, Regional, country, industry group or GP, and project levels. These inputs consist of strategic directions and commitments on gender; technical skills strengthened through training and guidance tools; monitoring and evaluation (M&E) systems; and a gender “architecture,” including gender leads, focal points, guidance, and advise to facilitate cross-sectoral and cross-institutional collaborations. These inputs are meant to strengthen the capacity of the Bank Group to provide effective support to countries (figure 3.1, column two) in the form of better diagnosis of gender gaps; the identification and piloting of gender-smart solutions; the coordinated use of multiple instruments (strategic plans, knowledge, lending, and technical assistance); and collaboration between the World Bank and IFC and across stakeholders. This increased capacity of the Bank Group is meant to increase the uptake among countries (figure 3.1, column three), in terms of better prioritization of relevant gender gaps and definition of adequate solutions; the piloting, testing, and implementation of gender-smart solutions; and their institutionalization through scale-up and creation of conditions for sustainability. Eventually, this should lead to reducing gender gaps in countries in the four dimensions identified by the gender strategy (figure 3.1, column four). 3.3 This evaluation considers countries as the main beneficiaries of World Bank and IFC support. Countries are understood to include the public and private sectors, civil society, local authorities, and the women and men citizens of those countries (country actors). This is in line with the country-driven approach, which is the first operationalization pathway of the gender strategy. 3.4 This evaluation will assess results at the level of country actors’ adoption of relevant gender-smart solutions (figure 3.1, column three). This evaluation will focus on Bank Group achievements in improving countries’ capacity to address gender inequalities—for example, by increasing the coverage, availability, and frequency of sex- disaggregated country-level data; improving the laws and regulations that impact women’s rights and opportunities; having companies adopt gender-inclusive practices; and implementing programs that address specific gender inequalities. The evaluation will not assess results in terms of reduction of gender inequalities in the country (figure 3.1, column four), such as reduction in maternal mortality or GBV, but it will use 11 country data and statistics on specific gender inequalities to measure the relevance of Bank Group support. The evaluation will also focus on the development objectives that Bank Group projects and activities have achieved (or intend to achieve) with respect to reducing gender inequalities for the intended beneficiaries. 3.5 The evaluation will also focus on select elements of Bank Group inputs and activities. The evaluation will complement and build on some findings of the IEG Mid- Term Review, which has extensively focused on processes, and will analyze select inputs and activities (figure 3.1, columns one and two), as detailed in the evaluation questions. 3.6 From a technical standpoint, the evaluation faces data limitations and difficulties in assigning attribution and will focus instead on the contribution of the gender strategy to advancing the Bank Group approach to gender equality. Data are limited because of the relatively short implementation period of the gender strategy: many of the projects approved under the gender strategy are still being implemented and have yet to produce tangible outcomes. Moreover, shortcomings in M&E frameworks were highlighted by previous evaluations, including the Results and Performance of the World Bank Group 2015 (World Bank 2016a), World Bank Group Gender Strategy Mid-Term Review (World Bank 2021), and Addressing Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence: An Evaluation of the World Bank Group’s Support (World Bank 2023a). Assigning attribution to the gender strategy is also difficult because the Bank Group is only one of many actors supporting the country’s gender agenda. Moreover, since 2020, an important confounding factor, COVID-19, has negatively affected both gender inequalities around the world and the implementation of the gender strategy. This justifies the evaluation’s focus on contribution rather than attribution. 3.7 The evaluation period is FY12–23 recognizing that the gender strategy framework originates in the World Development Report 2012. This evaluation is not narrowly focused on assessing the results of the gender strategy but, more broadly, the results of the Bank Group gender work, which has a long history (as detailed in box 1.1). Starting the evaluation period in 2012 makes special sense because this is the year in which the World Development Report 2012: Gender Equality and Development (World Bank 2012) was published. The World Development Report framework has guided the work on gender in the subsequent years and represents the explicit foundation of the FY16–23 gender strategy, which introduces some innovative elements but has many more elements of continuity with the preexisting framework. Finally, as the main goal of the evaluation is to assess results, it is necessary to adopt a time span longer than the strategy because the results discernible today mostly originate from activities that started before 2016. 12 Figure 3.1. The Theory of Action of the Gender Strategy for Fiscal Years 2016–23 Source: Independent Evaluation Group. Note: IFC = International Finance Corporation. 13 Evaluation Questions 3.8 The main evaluation question is as follows: What are the main results achieved by the World Bank and IFC in supporting countries to address gender inequalities, and what is the contribution of the gender strategy? This overarching question comprises two questions, which are further articulated into subquestions, as specified in box 3.1. These questions are about the “what” (results) and the “how” (processes) that have characterized the support provided by the World Bank and IFC to countries with the aim of closing gender gaps or addressing gender inequalities. 5 Assessing the factors that 4F affected the achievement of results and elements of the theory of action (the “why”) will be treated as a cross-cutting area of inquiry. 3.9 The first area of investigation speaks to the achievements of the World Bank and IFC effort to promote gender equality. It encompasses questions on the relevance of the support provided to countries by the World Bank and IFC with the aim of decreasing gender gaps; on the effectiveness of this support, or the tangible gender-smart solutions, outputs, and outcomes across the four strategic objectives articulated by the gender strategy; and on the extent to which the World Bank and IFC have been able to involve and support countries to fully own these results, institutionalize them, make them sustainable, and scale them up. 3.10 The second area of inquiry focuses on the specific contribution of the FY16–23 gender strategy. This evaluation will focus on the key elements of the gender strategy, including (i) the use of knowledge to inform the country engagement and the adoption of effective solutions to reduce gender inequalities; (ii) the guidance and incentives deployed by the institution to create internal awareness, ownership, knowledge, and accountability regarding gender strategy implementation; (iii) the corporate M&E systems (including the World Bank gender tag and the IFC gender flag) to monitor progress and correct course; and (iv) the World Bank convening power and capacity of coordinating and collaborating with the other development partners in the countries. 14 Box 3.1. Evaluation Questions What are the main results achieved by the World Bank and the International Finance Corporation (IFC) in supporting countries to address gender inequalities, and what is the contribution of the gender strategy? 1. What progress have the World Bank and IFC achieved in supporting countries to address gender inequalities since the World Development Report 2012? a. To what extent has the World Bank’s and IFC’s support been relevant to countries’ priorities, and how has this changed over time? b. To what extent have the World Bank and IFC supported country actors’ adoption of effective (gender-smart) solutions to reduce gender inequalities, and how has this changed over time? c. To what extent has the World Bank supported country actors’ ownership of gender- smart solutions and their capacity to replicate them, make them sustainable, and scale them up, and how has this changed over time?a 2. What has been the contribution of the fiscal year 2016–23 gender strategy in enabling progress in supporting countries to address gender gaps? a. What has been the contribution of the fiscal year 2016–23 gender strategy in improving World Bank and IFC use of knowledge, guidance, partnerships, and monitoring and evaluation mechanisms to track and achieve results? Cross-cutting question: Which factors affected the achievement of results and elements of the theory of action? Source: Independent Evaluation Group. Note: a. This evaluation adopts a definition of inclusive country ownership, meaning participation of key actors (government leaders, public officials, legislators, civil society actors, private sector, service users and providers, academia, and citizens) in setting priorities of the development agenda (World Bank 2012). In the case of this evaluation, country ownership encompasses (i) participation in defining gender priorities and solutions to address them and (ii) buy-in on the solutions (as envisaged by the gender strategy). Evaluation Scope Areas within Scope 3.11 The scope of this evaluation has been defined along five dimensions: reference period, institutional coverage, subject focus, Regional and country coverage, and level of engagement and operationalization: • Reference period. The reference period for this evaluation is FY12–23. It spans from FY12 (the year in which the World Development Report 2012: Gender Equality and Development was published; World Bank 2012) until FY23, up to the most recent month for which information on World Bank and IFC activities and initiatives is available at the time of sampling countries and projects. The recently published Bank Group’s 10-year retrospective (World Bank 2023b) also uses the 15 World Development Report 2012 publication year (FY12) as its reference period starting point. • Institutional coverage. The evaluation will cover the International Bank for Reconstruction and Development, IDA, and IFC. It will exclude MIGA, considering the short period since the introduction of the MIGA FY21–23 Gender Strategy Implementation Plan. • Subject focus. The evaluation will focus on gender gaps under the four strategic objectives across all Practice Groups, GPs, and industry groups. Because the four strategic objectives encompass a very large share of the overall Bank Group portfolio of activities, the evaluation will rely on a mix of global portfolio analysis and sampling (for countries and projects and for other activities) to make the analysis manageable. Some parts of the analysis will be deep dives into specific Practice Groups, GPs, and IFC industry groups (for example, the analysis of the robustness of the theory of change—part of evaluation question 1b). Some specific topics included in the gender strategy are not part of the scope of this evaluation (see the Areas out of Scope section in chapter 3). • Regional and country coverage. The evaluation aims to produce findings generalizable to all countries and Regions. Part of the analysis will rely on the overall portfolio of lending, IFC investment and advisory services, and country strategies, and part on country case studies. • Level of engagement and operationalization. The evaluation questions set up the analysis at multiple, interconnected levels: corporate, GP and industry group, Regional, country, and project or activity level. At the corporate level, the evaluation will focus on strategic documents of the institution, follow-up notes and other gender guidance, and corporate monitoring indicators (especially the World Bank gender tag and the IFC gender flag). At the Regional level, the evaluation will focus on Regional strategies (including the Regional gender action plans), initiatives, and the system of incentives and support provided at this level. At the country level, the evaluation will focus on country strategy documents, country-level initiatives and platforms, analytical work, data, policy dialogue, coordination, and collaboration with other relevant actors and between the World Bank and IFC. At the project or activity level, the evaluation will focus on investment project financing, development policy financing, Program-for- Results financing, advisory services and analytics, and IFC investment services and advisory services. The evaluation will assess the use of the different Bank Group instruments and the coordination among them in supporting countries to reduce gender inequalities. 16 Areas out of Scope 3.12 This evaluation will not be able to cover some topics in depth. In addition to excluding high-level results from the assessment of efficacy, this evaluation will only consider the use, production, and dissemination of knowledge in relation to the country strategic engagement and portfolio of activities. For example, the evaluation will review impact evaluations to assess evidence of results achieved by specific interventions but will not evaluate the activity of the Gender Innovation Labs as such. Similarly, the evaluation will review trust-funded activities as part of the relevant country portfolio (evaluation question 1b) but will not undertake an assessment of the Umbrella Facility for Gender Equality or other trust funds. 3.13 This evaluation will also not cover a set of topics that, while not included in the original gender strategy document, have progressively been added to the corporate gender agenda. These topics go under the “walking the talk” umbrella—that is, they relate to how the Bank Group promotes gender equality in human resource and workplace practices within the Bank Group, Bank Group procurement of goods and services, and project implementation: • Gender equality in human resource practices and corporate culture within the Bank Group. This topic is about how the Bank Group recruits, rewards, promotes, provides benefits to, and addresses the specific needs of men and women employees and employees of diverse sexual orientation and gender identity. Moreover, the topic includes how violence and harassment are addressed in the workplace and how the physical, managerial, and cultural environment promotes gender equality. An assessment led by the Development Economics Vice Presidency analyzed salary gaps between male and female Bank Group employees between 1987 and 2015 (Das, Joubert, and Tordoir 2017). In 2022, the Bank Group earned the second-level Economic Dividends for Gender Equality certification, which is considered the global standard for gender equality in the workplace and considers equal pay for equivalent work, recruitment and promotion, leadership development, training and mentoring, flexible working arrangements, and company culture. The evidence on which the Economic Dividends for Gender Equality certification is awarded is not public, however. Because assessing human resource practices and corporate culture is an overly complex and delicate task, this topic is not covered by this evaluation and could be part of a future separate study. • Gender equality in procurement. In April 2018, the Bank Group set a goal to more than double its share of corporate procurement spent with women-owned businesses to 7 percent by 2023 (World Bank Group 2018). This commitment is 17 part of a broader commitment to inclusive sourcing. This topic will not be covered in this evaluation as it is going to be looked at in an ongoing corporate evaluation on procurement practices. • GBV and sexual exploitation, abuse, and harassment action plan. 6 Recent 5F experience indicates that GBV and sexual exploitation, abuse, and harassment can be overlooked impacts of project implementation and need to be prevented and addressed through a solid safeguarding framework, which the Bank Group adopted in 2017 (World Bank 2016b). This evaluation will not cover this topic, which will be part of an upcoming evaluation of the Environmental and Social Framework. 3.14 Table A.1 lists the various levels and dimensions of analysis corresponding to each evaluation question. 4. Evaluation Design Approach 4.1 The evaluation design comprises six main blocks of activities conducted at five main levels: corporate, Regional, GP or industry group, country, and project or activity level. Figure 4.1 lists the evaluation activities and indicates which questions they help answer and the level of analysis. 4.2 At the corporate level, the analysis will assess the information content of corporate reporting of gender results, including through the gender tag and flag (evaluation question 2a). 4.3 At the Regional level, the analysis will assess how Regional commitments through the Regional gender action plans and the guidance, advice, and coordination provided at the Regional level have contributed to the implementation of the gender strategy (evaluation question 2a). 4.4 At the GP or industry level, the analysis will assess to what extent and how the guidance and advice on gender provided to World Bank and IFC teams have contributed to the implementation of the gender strategy (evaluation question 2a). In addition, the analysis of promising gender-smart solutions (robustness of projects’ theory of change), which is part of the analysis of effectiveness (evaluation question 1b), will be conducted for select types of projects through deep dives in select GPs or industries. At this level, the evaluation will also investigate the use of knowledge on “what works” to inform promising gender-smart solutions. 18 Figure 4.1. Evaluation Activities and Level of Analysis by Evaluation Question Source: Independent Evaluation Group. 19 4.5 At the country level, the analysis will assess the relevance of the World Bank and IFC support with respect to the country context and priorities, the comparative advantage of the two institutions, and the alignment of all instruments to ensure relevant and impactful support for gender equality (evaluation question 1a). It will also assess the World Bank and IFC contribution to identifying effective (gender-smart solutions) to reduce gender inequalities in the country (evaluation question 1b) and to fostering (i) ownership of these solutions by the country and (ii) the country’s ability to scale up and make those solution sustainable (evaluation question 1c). The country-level analysis will also aim at assessing how the use of knowledge, commitments, incentives, and mechanisms (including M&E), as well as the coordination and collaboration with the other development partners, have contributed to the adoption of the country gender engagement and of relevant and effective gender-smart solutions at the country level (evaluation question 2a). 4.6 At the project or activity level, the analysis will assess the types of gender gaps that the World Bank and IFC have prioritized and how aligned those gaps are with the priorities of the gender strategy and with the solutions that have been proposed and implemented to address them. At this level, the analysis will also assess the power of the World Bank gender tag and the IFC gender flag to identify and track the project potential to close gender gaps (evaluation question 2a), as well as the robustness of M&E frameworks, the results achieved based on reporting on gender-related indicators, and the robustness of the project theory of change (evaluation question 1b). 4.7 The evaluation activities include the following: • Country case studies. Each case study includes extensive desk reviews of Bank Group projects, advisory services and analytics, and strategic documents (country strategy–related documents and regional documents); desk review of literature on gender inequalities and effective solutions to address them in the country; desk review of the country’s own strategies and policies; and desk review of documents reporting on gender agendas and gender-related activities by other relevant actors. Country case studies will involve on-site data collection, which will rely on collaboration with local M&E and gender experts and on consultations with nonstate actors, beneficiaries, and other key informants; the studies will therefore require the support of local consultants or in-country visits. • Desk review of projects. This activity—conducted on samples of investment project financing, development policy financing, and IFC investment services and advisory services—consists of extracting information from project documents on (i) the type of gender gaps (and gender inequalities more broadly) targeted by projects; (ii) solutions to address those gaps and how these solutions 20 combine; (iii) the characteristics that make solutions gender smart, including their effectiveness and subsequent replicability and scalability; (iv) the type of indicators and reporting on outputs and outcomes at various stages in the life of the project; (v) the presence of the World Bank gender tag and IFC gender flag; and (vi) other relevant topics. The desk review of projects will also gather information on projects that support the collection of (and the country capacity to collect) better country-level sex-disaggregated data and statistics. Outcomes will be coded using an inductive taxonomy based on the four strategic objectives of the FY16–23 gender strategy. 7 6F • Assessment of country strategies. This exercise will consist of a review of the last Systematic Country Diagnostic and Country Partnership Framework based on criteria derived from the evaluation questions and aligned with the quality assurance criteria for Systematic Country Diagnostics and Country Partnership Frameworks outlined by the Resource Package for Bank Group Country Teams. The desk review will be complemented by interviews to country managers to identify factors impacting the country engagement for gender equality. • Desk review of GP or industry, regional, and corporate documents. This activity consists of extracting information from strategic documents and guidelines produced at the GP or industry level (such as GP follow-up notes), regional level (such as Regional gender action plans), and corporate level (such as gender strategy documents, guidance, and IDA Replenishment documents) to analyze (i) how guidance at these levels generates awareness and produces incentives and (ii) the added value of the gender strategy. • Semistructured and structured interviews and focus groups with key informants. Key informants include Bank Group management, Bank Group task team leaders and IFC investment officers, country directors or managers and Country Management Unit and country staff, the World Bank gender group, the IFC Gender and Economic Inclusion Solutions department, program leaders, gender leads and focal points, and project implementation units; representatives of partners and relevant international and national stakeholders (civil society organizations, women’s rights organizations, academics and think tanks, media, private sector, local authorities, religious and customary institutions, community leaders, and grassroots organizations); and representatives of women and men beneficiaries. • Analysis of gender statistics. This analysis consists of compiling relevant gender statistics for Bank Group countries, measuring gender inequalities, calculating trends over time, and—based on levels and trends—grouping countries 21 according to specific needs. This activity supports both the selection of country cases and the relevance analysis. • Theory of change analysis. This activity consists of identifying the main elements and robustness of the theory of change of select interventions (type of projects) based on project documents and GP guidelines, as well as evidence provided by empirical literature. 4.8 A pilot country case study was conducted as part of the evaluation inception exercise. The pilot country case study, with mission in the field, allowed the criteria for country case selection to be validated, the tools used for data collection and analysis and the main research hypothesis to be tested, and the evaluability of the various components to be assessed. 87F 4.9 The evaluation will follow different strategies to address complexity, maximize efficiency, and facilitate triangulation. To account for complexity, the evaluation adopts a nested design with methodological components building on each other and contributing to different evaluation subquestions; this feature will achieve economies of scale and assist in triangulation. This means that cases for this evaluation will be countries and, nested within those, the projects in those countries. The desk review of projects will encompass the projects studied in depth in the country cases, but it will expand the sample to assist in generalization and increase external validity (different samples will likely be used for different types of analysis). For example, the project portfolio review will rely on a representative sample of projects, investments, and advisory services to analyze gender gaps and solutions and to assess the function of the gender tag and gender flag. In other instances, the evaluation will go deeper into select types of projects; for example, it will focus on specific sectors or industries or types of gender-smart solutions to carry out the analysis of the theory of change. To achieve efficiency, the evaluation will use, to the extent possible, evidence collected during the preparation of the IEG Mid-Term Review (World Bank 2021) and the IEG gender in FCV contexts evaluation (World Bank 2023a). Moreover, several activities are designed to be conducted in parallel, rather than sequentially, which will increase efficiency and improve triangulation. 4.10 The evaluation aims to assess both World Bank and IFC support. It will analyze how the World Bank and IFC collaborate and coordinate to close gender gaps. This interaction between World Bank and IFC will be captured at the country level in the country case studies and in interviews with key informants at the Regional, GP or industry, and corporate levels. 22 4.11 The evaluation will consider the COVID-19 pandemic as one factor impacting achievements at the country and project levels. The pandemic caused stoppages that affected operational processes and policy effectiveness in different sectors, across numerous countries, across levels of government, and across the Bank Group itself. It also required emergency support to be mobilized very quickly, without what would be considered appropriate preparation in normal times. Moreover, the pandemic directly affected gender inequalities. In answering the evaluation questions, the evaluation will consider the role that COVID-19 had in slowing down progress in the country and in requiring course corrections in World Bank and IFC activities. It will also use the lessons of the recent IEG evaluation of the World Bank response to COVID-19 to the extent possible (World Bank 2022). 4.12 More details, a description of the activities planned, and the sources of information are included in the evaluation matrix in appendix A. Country Selection 4.13 The evaluation will adopt a purposive sampling strategy to select the country cases. The evaluation will select a set of eight countries as cases. The selection criteria are listed in box 4.1. FCV countries will not be included in the case studies because IEG has just completed an evaluation of Bank Group support to FCV countries to address gender inequalities that analyzes in depth the experience of six FCV countries (World Bank 2023a). However, this evaluation will also use the material from more than 200 interviews and the project and country strategy reviews conducted in FCV countries for the IEG gender in FCV evaluation (World Bank 2023a). Box 4.1. Criteria for Selection of Country Cases Diversification: i. Location in different geographical areas (regions) ii. Inclusion of International Development Association, blend, and International Bank for Reconstruction and Development countries Programmatic: i. Significant engagement of the World Bank Group in supporting the country to advance gender equality ii. Presence of both World Bank and International Finance Corporation gender-focused interventions iii. Identification of what World Bank and International Finance Corporation corporate documents (for example, gender strategy updates, International Development Association Mid-Term Reviews), as well as interviewees, highlight “successful” cases of a 23 country-driven approach to addressing gender gaps and implementing gender-smart solutions. In addition to applying the selection criteria, the evaluation team will seek advice from gender focal points and gender experts, the World Bank gender group, and the International Finance Corporation gender team to choose among multiple equally eligible candidates. Source: Independent Evaluation Group. Challenges and Limitations 4.14 The evaluation design takes into account two main challenges: the assignment of attribution and data limitation (including shortcomings of the gender tag). These challenges emerged from the analysis of the FY16–23 gender strategy’s theory of action and have been confirmed in the interviews conducted during the scoping phase. 4.15 A first set of challenges concern the identification of the impact of the gender strategy (attribution). Because the gender strategy builds on several decades of evolving strategic approaches and institutional commitments to promote gender equality (box 1.1), it is not possible to isolate its distinct impact on advancing the gender agenda in client countries. Several institutional commitments related to gender, for example, have been introduced by successive IDA Replenishments and, earlier on, by World Bank Operational Policy 4.20; the Regional gender action plans existed before the gender strategy, as did the Gender Innovation Labs, the IFC gender flag, and an initial version of the World Bank gender tag—just to mention some of the most salient features of the gender strategy’s theory of action. Moreover, it is difficult to disentangle the various contributions of many development partners (of which the Bank Group is only one) to advancing the overall gender agenda in a country. This issue will be addressed by focusing on the contribution of the gender strategy, acknowledging that attribution cannot be determined. In addition, the evaluation will focus on the FY12–23 period, longer than the gender strategy period, to discern changes that could be associated with the introduction of the strategy. 4.16 Data limitation derives from the short gender strategy implementation period. Only a small minority of projects and activities that started after the launch of the gender strategy are closed, which makes it hard to compare results of the gender strategy against results of the longer-term gender agenda. The evaluation will address this type of data limitation by considering not only closed projects but also those still under implementation and by looking not only at results and their indicators but also at the robustness of the theory of change of gender-smart solutions (evaluation question 1b). 4.17 Another aspect of data limitation is the shortcomings of the gender tag and gender flag. The main monitoring tool—the gender tag and gender flag—is defined only at the project level, despite the gender strategy fully embracing the country-driven 24 approach. It is therefore valuable for the evaluation to explore whether there is a disconnect between the country-driven approach, at the core of the Bank Group strategic approach to gender, and the tool used to generate awareness and monitor progress—the gender tag and gender flag—which is assigned exclusively at the project level and identifies projects that “demonstrate a results chain by linking gender gaps identified in analysis to specific actions tracked in the results framework.” Even if projects are correctly gender tagged and flagged, there could be a disconnect between the specific gaps that projects aim to close and the country’s needs and priorities. The evaluation is mindful of this limitation and will assess the extent of this potential disconnect. 4.18 An additional challenge is the limited informational content of the gender tag and flag in relation to project results. The gender tag and flag is assigned at the design stage and is, according to the strategy, the main tool to identify projects that aim to close gender gaps. There is no systematic tool to follow up and further reassess the gender tag and flag during implementation or at project closing; the gender tag and flag is therefore not identifying projects that did achieve results. 9 Also, over time, the incidence of the 8F gender tag has dramatically increased, and as of today, virtually all World Bank projects are gender tagged. From an evaluative perspective, it is therefore increasingly difficult to distinguish projects that are especially gender relevant or have higher potential to close gender gaps. 10 9F 4.19 Finally, the timing of the evaluation is not synched with the timing of the upcoming gender strategy. The Bank Group is currently preparing the new gender strategy, which is planned to be launched in 2024. This challenge will be addressed by sharing emerging findings informally with the gender group to provide useful evaluative evidence to inform the strategy and its operationalization plan. 5. Quality Assurance Process 5.1 The evaluation will be subject to a rigorous quality assurance process. This Approach Paper will undergo the IEG standard internal review process and be peer- reviewed by three recognized international experts: Michael Bamberger, independent evaluator, with internationally recognized expertise in development program evaluation and gender evaluation; Elizabeth King, nonresident senior fellow at the Brookings Institution and former World Bank director of education and vice president for human development; and Deepa Narayan, author and social scientist and former World Bank poverty reduction and economic management senior adviser. A private sector expert will be added as a reviewer of the final report. 25 6. Engagement and Dissemination 6.1 The evaluation team will regularly engage with the World Bank gender group, the IFC Gender and Economic Inclusion Solutions department, and other key stakeholders throughout the evaluation process. The evaluation team has identified opportunities for collaboration and exchange with the gender teams (World Bank and IFC), which are currently undertaking several assessments aimed at informing the new 2024–30 gender strategy. 7. Resources 7.1 The evaluation will be prepared with an estimated budget of US$700,000. The team members for the evaluation are Elena Bardasi and Mariana Branco (task team leaders), Susan Caceres, Shiva Chakravarti Sharma, Giulia Ciliotta Bezada, Serena Fogaroli, Diana Goldemberg, Rima Habasch, Laura Pasquero, Maria Elena Pinglo, and consultants contributing to country case studies. Estelle Raimondo will provide methodological advice and guidance for the evaluation, and Ariya Hagh did so for the preparation of this Approach Paper. Yezena Yimer will provide administrative support. The work will be conducted under the guidance of Galina Sotirova (manager, IEG Corporate and Human Development), Theo Thomas (director, IEG Human Development and Economic Management), and Sabine Bernabe (Director-General, Evaluation). 1The 2016 gender strategy document does not cover inequalities based on sexual orientation and gender identity (SOGI). However, the World Bank Group has been developing an SOGI agenda starting from 2016, when the first global adviser on SOGI was hired in the Social Sustainability and Inclusion Global Practice. The Good Practice Note titled “Environmental and Social Framework for IPF Operations: Non-Discrimination—Sexual Orientation and Gender Identity (SOGI)” was issued in 2019 to support the implementation of the environmental and social standards (World Bank 2019) and complement the World Bank Directive on Addressing Risks and Impacts on Disadvantaged or Vulnerable Individuals or Groups (World Bank 2018). An updated version of the Good Practice Note is forthcoming in 2023. Similarly, the Economic Inclusion Program of the International Finance Corporation launched in 2020 and the International Finance Corporation’s environment and social work that also includes SOGI are not covered by this evaluation. 2 See https://sdgs.un.org/goals. 3The strategy builds on the conceptual framework of the World Development Report 2012 (World Bank 2012), which posits that households, markets, and institutions and the interactions between them influence gender equality and economic development. 26 4An earlier version of the gender tag to monitor gender integration in World Bank operations (called, at the time, “gender flag”) was introduced in 2011. The design of the new gender tag was presented in appendix D of the fiscal year 2016–23 gender strategy; detailed guidance on applying this tool was prepared after the launch of the strategy. 5This evaluation refers to “gender gaps” in keeping with the approach and language of the fiscal year 2016–23 gender strategy; however, it favors the concept of “gender inequalities,” which is more comprehensive (as it also includes those dimensions that are women- and girl-specific, such as gender-based violence, maternal mortality, and early pregnancies). The analysis will focus on gender inequalities more broadly. 6The gender-based violence and sexual exploitation, abuse, and harassment action plan was developed and introduced after the World Bank Inspection Panel investigation of the Uganda Transport Sector Development Project in 2015–16, which found evidence of sexual violence and harassment, in addition to other serious safeguard violations. 7 Taxonomy is defined here as a typology of gender gaps that is based on the four strategic objectives of the fiscal year 2016–23 gender strategy, goes to a more granular level, and is derived from the analysis of actual projects. Recent publications, such as World Bank 2023b, will be used to compare and verify this taxonomy. 8Benin was identified as the pilot case study because of its diverse portfolio. A field mission was conducted in June-July 2023. 9The project outcome indicators that refer to gender gaps obviously need to be tracked and reported in Implementation Status and Results Reports and Implementation Completion and Results Reports, similar to every project outcome indicator. 10The gender tag is a Corporate Scorecard indicator (Corporate Scorecard indicators are meant to provide snapshots of the results and performance of the Bank Group), but as the tag is defined only at entry, it does not measure results achieved. 27 References Cortez, Clifton, John Arzinos, and Christian de la Medina Soto. 2021. Equality of Opportunity for Sexual and Gender Minorities. Washington, DC: World Bank. Das, Jishnu, Clement Joubert, and Sander Florian Tordoir. 2017. “Compensation, Diversity and Inclusion at the World Bank Group.” Policy Research Working Paper 8058, World Bank, Washington, DC. World Bank. 1994. “Enhancing Women’s Participation in Economic Development.” Policy Paper 13415, World Bank, Washington, DC. World Bank. 2002a. Integrating Gender into the World Bank’s Work: A Strategy for Action . Washington, DC: World Bank. World Bank. 2002b. The World Bank Gender and Development Policy Framework—A Guidance Note. Washington, DC: World Bank. World Bank. 2005. Evaluating a Decade of World Bank Gender Policy: 1990 –99. Operations Evaluation Department. Washington, DC: World Bank. World Bank. 2009. Gender and Development: An Evaluation of World Bank Support, 2002–08. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2011. Gender Equality as Smart Economics: World Bank Group Gender Action Plan— Four-Year Progress Report (January 2007–December 2010). Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/519961468325777020/pdf/617100BR0SecM 20e0only0900BOX358354B.pdf. World Bank. 2012. World Development Report 2012: Gender Equality and Development . Washington, DC: World Bank. World Bank. 2016a. Results and Performance of the World Bank Group 2015 . Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/evaluations/results-and-performance-2015. World Bank. 2016b. The World Bank Environmental and Social Framework. Washington, DC: World Bank. https://thedocs.worldbank.org/en/doc/837721522762050108- 0290022018/original/ESFFramework.pdf. World Bank. 2017. Global Gender-Based Violence Task Force: Action Plan for Implementation. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/206731510166266845/pdf/121031-WP- PUBLIC-Gender-Based-Violence-Task-Force-Action-Plan.pdf. 28 World Bank. 2018. “Environment and Social Framework for IPF Operations: Non -Discrimination and Disability.” Good Practice Note, World Bank, Washington, DC. https://documents1.worldbank.org/curated/en/573841530208492785/Environment-and- Social-Framework-ESF-Good-Practice-Note-on-Disability-English.pdf. World Bank. 2019. “Environmental and Social Framework for IPF Operations: Non - Discrimination—Sexual Orientation and Gender Identity (SOGI).” Good Practice Note, World Bank, Washington, DC. https://thedocs.worldbank.org/en/doc/590671570796800429- 0290022020/original/GoodPracticeNoteSOGI.pdf. World Bank. 2021. World Bank Group Gender Strategy Mid-Term Review. Independent Evaluation Group. Washington, DC: World Bank. https://ieg.worldbankgroup.org/evaluations/gender-strategy-mid-term-review. World Bank. 2022. The World Bank’s Early Support to Addressing COVID -19: Health and Social Response—An Early-Stage Evaluation. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023a. Addressing Gender Inequalities in Countries Affected by Fragility, Conflict, and Violence: An Evaluation of the World Bank Group’s Support . Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2023b. Gender Equality in Development: A Ten-Year Retrospective. Washington, DC: World Bank. http://hdl.handle.net/10986/39939. World Bank. 2023c. “World Bank Gender Strategy 2024–2030: Accelerate Gender Equality for a Sustainable, Resilient, and Inclusive Future.” Consultation Draft, World Bank, Washington, DC. World Bank Group. 2015. World Bank Group Gender Strategy (FY16–23): Gender Equality, Poverty Reduction, and Inclusive Growth. Washington, DC: World Bank. https://documents1.worldbank.org/curated/en/820851467992505410/pdf/102114- REVISED-PUBLIC-WBG-Gender-Strategy.pdf. World Bank Group. 2018. Gender Responsive Procurement at the World Bank Group. Washington, DC: World Bank. https://thedocs.worldbank.org/en/doc/592061615492148577- 0090022021/original/5WBIWD2021SlidesRubinfield.pdf. World Bank Group. 2020. World Bank Group Strategy for Fragility, Conflict, and Violence 2020 –2025. Washington, DC: World Bank. http://documents.worldbank.org/curated/en/844591582815510521/World-Bank-Group- Strategy-for-Fragility-Conflict-and-Violence-2020-2025. 29 Appendix A. Evaluation Methodology Table A.1. Evaluation Matrix Questions Description Methods Sources of Information and Sampling Overarching EQ: What are the main results achieved by the World Bank and IFC in supporting countries to address gender inequalities, and what is the contribution of the gender strategy? EQ 1. What progress have the World Bank and IFC achieved in supporting countries to address gender inequalities since the World Development Report 2012? 1a. To what extent has 1. Evaluate to what extent the World 1. Country case studies: 1. Regional-level documents (all regions of the World Bank’s and Bank’s and IFC’s support to countries (i) Analysis of the priorities, goals, and actions country cases): IFC’s support been (through the various instruments to reduce the country gender gaps based on (i) Regional strategies; relevant to countries’ deployed: IPF, DPF, Program-for- desk review of World Bank and IFC strategic (ii) Regional gender action plans; priorities, and how has Results, IFC investment services and documents (at the Regional, country, and this changed over time? advisory services, ASA, and other types (iii) Regional gender-focused knowledge project or activity levels) and interviews with of engagement) reflects priorities, products (relevant to selected countries and key informants, to assess (a) consistency goals, and actions to reduce gender GPs); across activities and initiatives, (b) type and gaps that are the following: use of evidence to identify priorities and (iv) Regional relevant projects (in countries (i) Aligned across instruments; actions in response to those priorities, (c) selected as case studies). (ii) Identified and implemented in consistency between priorities and actions, 2. Country-level documents: coordination and collaboration with the and (d) consistency between priorities and (i) Bank Group country strategies (CASs, CPFs, other relevant development partners; actions and country context (as described by CENs, PLRs, CLRs, CPEs, CPSD, and IFC Country (iii) Aligned with the comparative data and statistics and analyzed in existing Strategies); advantage of the World Bank Group in gender studies); (ii) World Bank and IFC knowledge products, the country; (ii) Analysis of World Bank and IFC strategic including general diagnostics (RRAs, SCDs, job (iv) Aligned with Bank Group regional documents and the country’s own strategic diagnostics, poverty diagnostics, social strategic priorities and the regional documents and those of other key actors to inclusion diagnostics, private sector gender strategic priorities; assess consistency of World Bank and IFC diagnostics, and so on), and gender-specific priorities, goals, and actions with country (v) Aligned with country priorities (as knowledge products; priorities and those of other key development expressed in the country’s own policies (iii) Country’s own gender strategies and actors; and and strategic documents, the Bank policies; Group country strategy, and the policy (iii) Comparative analysis of the same country over time. (iv) Country data and statistics; agenda of the government, private sector, and other relevant country 2. Review of SCDs and CPFs to assess: (a) the (v) Strategic documents and platforms of key actors); identification of priorities for country actors (for example, the United Nations, other development partners, women’s rights (vi) Aligned with the local context (with engagement on gender; (b) the use of organizations, and other CSOs); and the evidence on gender gaps provided knowledge for the analysis and prioritization by data and analytical studies); and of gender gaps and the definition of policy (vi) Relevant gender studies on the country. responses; (c) the level of coordination of the 30 Questions Description Methods Sources of Information and Sampling (vii) Informed by major country-specific different initiatives and institutions around 3. Project-level documents (for a representative constraints to gender equality that the gender gaps; and (d) the focus on sample of projects): PADs, Mid-Term Reviews, Bank Group has clearly and correctly sustainability and scale-up of gender restructuring papers, ICRs, ICRRs, PPARs, and identified through analytical work. outcomes and solutions through improvement IFC investment and advisory documents. 2. Evaluate to what extent relevance has of the enabling environment. 4. Interviews with key informants of the Bank changed over time. 3. Interviews with country managers to identify Group and relevant country actors factors influencing the above-mentioned (representatives of the government, dimensions. implementing agencies, civil society, women’s organizations, donors, research community, and so on). 1b. To what extent have 1. Evaluate to what extent the solutions 1. Project-level analysis of results achieved 1. For methods 1, 2, and 3: the World Bank and IFC adopted by countries with World Bank’s (various samples): (i) desk review of gender The analysis will rely on project documents supported country or IFC’s support (using the various gaps using a typology developed by the (PADs, ISRs, ICRs, ICRRs, and IFC investment actors’ adoption of instruments identified and analyzed to evaluation based on corporate documents; (ii) and advisory documents) and similar effective (gender-smart) respond to EQ 1a) have been effective desk review of solutions based on a typology documents by implementing agencies and solutions to reduce (gender smart) in reducing gender developed by the evaluation using corporate partner donors; published articles and reports; gender inequalities, and gaps, in terms of the following: documents; (iii) desk review of project M&E and gray literature. Typologies of gender gaps how has this changed (i) Achieved results; frameworks; (iv) desk review of M&E reports of and solutions will be derived using an inductive over time? (ii) Expected results; and implementing agencies and co-financing approach (based on occurrences in projects partners, when available; (v) desk review of and corporate documents) and a deductive (iii) Robustness of the theory of change impact evaluations and other evaluations of approach (based on literature). Machine (in other words, an evidence-based the project; and (iv) desk review of studies learning will assist in coding projects after the theory of change). (including systematic reviews and meta- typologies are identified. 2. Evaluate to what extent the project analyses) on the effectiveness of similar M&E system (indicators and methods) interventions in the country or similar contexts. 2. For method 3: is adequate to track the reduction of Typologies of projects and theories of change 2. Case studies: (i) in-depth analysis of resultswill be identified based on existing gender gaps. of select interventions and the theory of Independent Evaluation Group and Operations 3. Evaluate to what extent results, change of projects (which belong to the robustness of the theory of change, and sample of projects for project-level analysis); Policy and Country Services work to classify M&E systems have changed over time. (ii) interviews with key informants and focus project outcomes. Projects will be purposefully selected based on a combination of criteria: groups with beneficiaries; and (iii) analysis of relevant projects in the case studies; projects documents of similar interventions flagged as innovative or best practice by GPs (comparators) implemented in the country. or industries (presented in Bank Group 3. Analysis of the robustness of the theory of reporting as gender-smart solutions); projects change through deep dives in select GPs or whose impact has been evaluated by the industries and for select types of projects: Gender Innovation Labs; and projects identified comparison of “models” of theories of change during interviews. A limited number of project of successful and unsuccessful World Bank and 31 Questions Description Methods Sources of Information and Sampling IFC interventions to assess completeness and types (two to three) will be selected for this GP type of elements present and absent, based on or industry deep dive. evidence from quantitative and qualitative evaluations. 4. Comparative analysis for methods 1 and 2 over time. 1c. To what extent has 1. Evaluate country actors’ ownership of 1. Country case studies: (i) desk review of 1. Project-level documents: documents the World Bank the gender-smart solutions identified in project documents referring to the gender- referring to the IPF, DPF, and ASA. supported country EQ 1b and the gender outcomes these smart solutions identified in EQ 1b to assess actors’ ownership of solutions generated: (i) who engaged in (a) the role of the government and other key gender-smart solutions the process and how (at the various country actors in identifying, implementing, and their capacity to stages: identification, prioritization, monitoring, adjusting, and evaluating the replicate them, make implementation, monitoring, course project and (b) the strategy and actions them sustainable, and corrections, and evaluation) and (ii) to pursuing ownership, sustainability, scale-up, scale them up, and how what extent the government, the and capacity development for local actors; (ii) has this changed over private sector, and other actors have interviews with the task team leaders, project time? adopted those solutions and gender implementation unit coordinators, and country outcomes. managers for triangulation purposes. 2. Evaluate the capacity of the Comparative analysis for methods 1(i)(a) and government, private sector, and other 1(i)(b) over time. relevant actors to implement the gender-smart solutions and the existing strategies and actions planned or implemented by the government and other actors (with the support of the Bank Group) to make them sustainable and scale them up. EQ 2. What has been the contribution of the FY16–23 gender strategy in enabling progress in supporting countries to address gender gaps? 2a. What has been the 1. Evaluate four key elements of the 1. GP and industry level (for select GPs and 1. The GPs for this analysis will be identified contribution of the FY16– theory of action of the gender strategy: industries): (i) desk review of follow-up notes based on the prevalence of relevant projects 23 gender strategy in (i) the use of knowledge to achieve and updates, other guidance, and web pages and theories of change (as emerging in improving World Bank results; (ii) the guidance provided by and (ii) interviews with gender advisers or focal previous EQ analysis). and IFC use of GPs and industries; (iii) the Regional (or points on the most significant results after the 2. The Regional- and country-level analysis will knowledge, guidance, country) gender action plans and the adoption of the gender strategy and on the be conducted for the country cases and their partnerships, and M&E role of the Regional or country gender role of advisory support, guidance, and Regions. mechanisms to track and guidance; (iv) the World Bank gender information in achieving those results in 3. The projects for the gender tag–gender flag achieve results? tag; (v) the IFC gender flag; and (vi) the relation to other factors. analysis will be the representative sample of World Bank and IFC collaboration, 32 Questions Description Methods Sources of Information and Sampling convening power and coordination 2. Regional and country level: (i) desk review of projects approved after the introduction of the activity for gender equality. Regional gender actions plans and updates, gender tag or gender flag. 2. Evaluate to what extent there is an the country gender action plans and updates, 4. The change over time will be based on key improvement (increased contribution of web pages and documents uploaded, and so informant interviews and on analysis of these elements) over time. on and (ii) interviews with key informants guidance, training material, and Regional (managers and gender coordinators at the gender action plans and their updates. Regional level; country managers and gender 5. The analysis of World Bank and IFC resource persons for the country case studies). convening power and coordination activity for 3. Project-level analysis of the World Bank gender equality will be carried out at the gender tag and IFC gender flag: (i) analysis of country level (in country case studies). the quality of design and M&E framework for select projects to compare tagged projects with untagged projects and (ii) interviews with key informants (gender tag advisers and task team leaders). 4. Corporate-level analysis of the World Bank gender tag and IFC gender flag: (i) desk review of guidance, training material, reports, statistics on adoption, and gender tag intranet web page and (ii) interview with key informants (gender tag and gender flag advisers and directors of operational effectiveness). 5. Analysis of World Bank and IFC convening power and coordination activity for gender equality. Cross-cutting question: Which factors affected the achievement of results and elements of the theory of action? Source: Independent Evaluation Group. Note: ASA = advisory services and analytics; CAS = Country Assistance Strategy; CEN = Country Engagement Note; CLR = Completion and Learning Review; CPE = Country Program Evaluation; CPF = Country Partnership Framework; CPSD = Country Private Sector Diagnostic; CSO = civil society organization; DPF = development policy financing; EQ = evaluation question; FY = fiscal year; GP = Global Practice; ICR = Implementation Completion and Results Report; ICRR = Implementation Completion and Results Report Review; IFC = International Finance Corporation; IPF = investment project financing; ISR = Implementation Status and Results Report; M&E = monitoring and evaluation; PAD = Project Appraisal Document; PLR = Performance and Learning Review; PPAR = Project Performance Assessment Report; RRA = Risk and Resilience Assessment; SCD = Systematic Country Diagnostic. 33 Appendix B. Prevalence of the World Bank Gender Tag and International Finance Corporation Gender Flag World Bank Figure B.1. Gender-Tagged Projects by Fiscal Year No Yes No data 2017 (N = 379) 2018 (N = 322) 2019 (N = 343) 2020 (N = 443) Year 2021 (N = 415) 2022 (N = 429) 2023 (N = 320) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: Year is fiscal year of approval. Projects approved in fiscal year 2017 and later are included. 34 Figure B.2. Percentage of Gender-Tagged Projects (2017–23) by Global Practice No Yes No data Agriculture and Food (N = 201) Digital Development (N = 52) Education (N = 189) Energy and Extractives (N = 203) Environment, Natural Resources, and the Blue Economy (N = 106) Finance, Competitiveness, and Innovation (N = 196) Governance (N = 134) Global Practice Health, Nutrition, and Population (N = 368) Macroeconomics, Trade, and Investment (N = 278) Poverty and Equity (N = 34) Social Protection and Jobs (N = 214) Social Sustainability and Inclusion (N = 60) Trade and Competitiveness (N = 3) Transport (N = 183) Urban, Disaster Risk Management, Resilience, and Land (N = 275) Water (N = 155) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: Projects approved in fiscal year 2017 and later are included. 35 Figure B.3. Gender-Tagged Projects (2017–23) by Lending Instrument No Yes No data DPF (N = 410) Lending instrument IPF (N = 2,028) PforR (N = 213) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: Projects approved in fiscal year 2017 and later are included. DPF = development policy financing; IPF = investment project financing; PforR = Program-for-Results. 36 Figure B.4. Gender-Tagged Projects (2017–23) by Region No Yes No data East Asia and Pacific (N = 357) Eastern and Southern Africa (N = 519) Europe and Central Asia (N = 321) Region Latin America and the Caribbean (N = 335) Middle East and North Africa (N = 166) South Asia (N = 356) Western and Central Africa (N = 597) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: Projects approved in fiscal year 2017 and later are included. 37 International Finance Corporation Investment Services Figure B.5. Gender-Flagged Projects by Fiscal Year No Yes No data 2019 (N = 223) 2020 (N = 264) 2021 (N = 292) Year 2022 (N = 283) 2023 (N = 295) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: For International Finance Corporation investment services, the gender flag has been systematically captured since fiscal year 2019. Percentage is calculated for projects committed from fiscal year 2019 to fiscal year 2023. 38 Figure B.6. Gender-Flagged Projects by Region No Yes No data East Asia and Pacific (N = 206) Central Asia and Türkiye (N = 76) South Asia (N = 169) Latin America and the Caribbean (N = 254) Region Africa (N = 428) Europe (N = 112) Middle East (N = 61) World (N = 51) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: For International Finance Corporation investment services, the gender flag has been systematically captured since fiscal year 2019. Percentage is calculated for projects committed from fiscal year 2019 to fiscal year 2023. 39 Figure B.7. Gender-Flagged Projects by Industry Group No Yes No data Manufacturing, Agribusiness, and Services (N = 392) Industry group Infrastructure (N = 202) Financial Markets (N = 507) Disruptive Technologies and Funds (N = 255) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: For International Finance Corporation investment services, the gender flag has been systematically captured since fiscal year 2019. Percentage is calculated for projects committed from fiscal year 2019 to fiscal year 2023. 40 International Finance Corporation Advisory Services Figure B.8. Advisory Services Projects with Allocation for Gender by Fiscal Year No Yes 2017 (N = 107) 2018 (N = 121) 2019 (N = 125) Year 2020 (N = 143) 2021 (N = 97) 2022 (N = 122) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: For International Finance Corporation advisory services, the gender flag has been systematically captured since fiscal year 2017. Percentage is calculated for projects committed from fiscal year 2017 to fiscal year 2023. 41 Figure B.9. Percentage of Advisory Services Projects with Allocation for Gender by Region No Yes Europe (N = 46) Central Asia and Türkiye (N = 33) Africa (N = 352) East Asia and Pacific (N = 138) Region World (N = 19) Latin America and the Caribbean (N = 109) Middle East (N = 65) South Asia (N = 90) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: For International Finance Corporation advisory services, the gender flag has been systematically captured since fiscal year 2017. Percentage is calculated for projects committed from fiscal year 2017 to fiscal year 2023. 42 Figure B.10. Percentage of Advisory Services Projects with Allocation for Gender by Primary Business Area No Yes Environment, Social, and Governance (N = 76) Infrastructure (N = 36) Regional Advisory (N = 171) Manufacturing, Agribusiness, and Services (N = 111) Financial Institutions Group (N = 271) Business area Transaction Advisory (N = 146) Disruptive Technologies and Funds (N = 33) Climate Business (N = 16) Other (N = 17) 0 20 40 60 80 100 Share of projects (%) Source: Independent Evaluation Group. Note: For International Finance Corporation advisory services, the gender flag has been systematically captured since fiscal year 2017. Percentage is calculated for projects committed from fiscal year 2017 to fiscal year 2023. 43