World Bank Group Support to Demand-Side Energy Efficiency © 2023 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington, DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org ATTRIBUTION Please cite the report as: World Bank. 2023. World Bank Group Support to Demand-Side Energy Efficiency. Independent Evaluation Group. Washington, DC: World Bank. COVER PHOTO Shutterstock/thinkhubstudio EDITING AND PRODUCTION Amanda O’Brien GRAPHIC DESIGN Luísa Ulhoa This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The bound- aries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissem- ination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. World Bank Group Support to Demand-Side Energy Efficiency An Independent Evaluation March 2, 2023 Contents Abbreviations v Acknowledgments vi Overview viii Background and Context������������������������������������������������������������������������������������������1 1.  Context 3 Evaluation Purpose, Questions, and Methods 9 Effectiveness, Scale-Up Challenges, and Factors of Success�����������������������������13 2.  Demand-Side Energy Efficiency Portfolio 15 Effectiveness of Interventions and Scale-Up 19 Challenges to Scaling 25 Factors of Success in Scaling 28 Coherence of World Bank Group Interventions to Support Scale-Up���������������������� 37 3.  Internal Coherence 41 External Coherence 49 Untapped Opportunities in Demand-Side Energy Efficiency������������������������������58 4.  Indirect Greenhouse Gas Emissions and Socioeconomic Outcomes 59 Embodied Carbon for the Building Industry 64 Digital Innovations 66 Financial Innovations 69 Conclusions and Recommendations�������������������������������������������������������������������� 74 5.  Bibliography��������������������������������������������������������������������������������������������������������������� 78 ii Boxes Box 2.1. Global Programs Supporting Energy Efficiency 18 World Bank Group Support for Demand-Side Energy Efficiency Market Box 2.2.  Creation in India 24 Blended Finance to Support Demand-Side Energy Efficiency Box 3.1.  across Sectors 46 The World Bank’s Support for Demand-Side Energy Efficiency in Armenia Box 3.2.  52 Greening the Energy-Transport Nexus for Reducing Indirect Emissions and Box 4.1.  Developing Horizontal Scaling Opportunities 62 Figures Figure 1.1. Energy Efficiency Supply and Demand 4 Figure 1.2. Global Annual Rate of Energy Efficiency Improvement 6 Global Annual Investment in Energy Efficiency, 2015–21: Figure 1.3.  Past Trends and Future Need 7 Mapping Evaluation Questions to Evaluation Figure 1.4.  Methodologies and Techniques 11 Distribution of World Bank Demand-Side Energy Efficiency Figure 2.1.  Portfolio by Country Income Classification 17 Share of International Finance Corporation Investment Services Projects Figure 2.2.  Targeting Primary Demand-Side Energy Efficiency Outcomes 22 Demand-Side Energy Efficiency Scale-up and Development Benefits Figure 2.3.  through IFC’s GBMTP-EDGE Certifications by Region 31 Countries with Cumulative Demand-Side Energy Figure 2.4.  Efficiency Engagements 35 Figure 3.1. The Relationship among Coherence, Scaling, and Outcomes 40 International Finance Corporation Demand-Side Energy Figure 3.2.  Efficiency Support 45 Demand-Side Energy Efficiency Cofinanciers Figure 3.3.  by Commitment for Investment Project Finance, 2011–20 51 World Bank and Energy Sector Management Assistance Figure 3.4.  Program Contributions to Vietnam’s Energy Planning 54 iii Figure 4.1. Emissions along the Supply Chain 61 Various Demand-Side Energy Efficiency Outcomes as a Percentage of Figure 4.2.  the World Bank Group Demand-Side Energy Efficiency Portfolio 63 Figure 4.3. Temporal View of a Typical Life Cycle for Buildings 65 Figure 4.4. Digitalization of Demand-Side Energy Efficiency Systems 67 Tables Vertical and Horizontal Scaling in Demand-Side Energy Efficiency Table 1.1.  8 World Bank Group Demand-Side Energy Efficiency Portfolio, Table 2.1.  Fiscal Years 2011–20 16 Multilateral Organizations’ Approaches to Demand-Side Energy Efficiency 50 Table 3.1.  Appendix Appendix A. Methodology 86 iv Abbreviations ASA advisory services and analytics CO2 carbon dioxide CTF Clean Technology Fund DPF development policy financing DPO development policy operation DSEE demand-side energy efficiency EDGE Excellence in Design for Greater Efficiencies ESCO energy service company ESMAP Energy Sector Management Assistance Program EU European Union FY fiscal year GBMTP Green Buildings Market Transformation Program GCF Green Climate Fund GEF Global Environment Facility GHG greenhouse gas GP Global Practice IEG Independent Evaluation Group IFC International Finance Corporation IPF investment project financing LEED Leadership in Energy and Environmental Design LIC low-income country Independent Evaluation Group World Bank Group    v LMIC lower-middle-income country MDB multilateral development bank MIC middle-income country MIGA Multilateral Investment Guarantee Agency MSMEs micro, small, and medium enterprises SDG Sustainable Development Goal SOE state-owned enterprise SSEE supply-side energy efficiency UMIC upper-middle-income country All dollar amounts are US dollars unless otherwise indicated. Acknowledgments This report was prepared by an Independent Evaluation Group (IEG) team led by Raghavan Narayanan, senior evaluation officer, and Victoria Alexe- eva, evaluation officer, and was conducted under the guidance and super- vision of Marialisa Motta, manager; Carmen Nonay, director; and Alison M. Evans, former Director-General, Evaluation. At the time of the Committee on Development Effectiveness meeting, the team was led by Ramachandra Jammi, senior evaluation officer, and the acting Director-General was Oscar Calvo-Gonzalez. The core evaluation team included Joy Kaarina Butscher, evaluation officer; Hiroaki Kambe, evaluation officer; Sanittawan Nikki Tan, extended-term consultant; Amshika Amar, Maria Shkaratan, and Tao Tao, consultants; and, during the Approach Paper stage, Ihsan Kaler Hurcan, consultant. Subject World Bank Group Support to Demand-Side Energy Efficiency  Acknowledgments matter expert Philippe Benoit advised the team. The team was supported by the methods advisory group of Jozef Vaessen, adviser; Estelle Raimondo, senior evaluation officer; Ariya Hagh, extended-term consultant; and Lauren Kelly and Andrew Stone, technical leads. During the Approach Paper stage, the team was supported by Zeljko Bogetic, lead economist. The country case studies were authored by Nana Sika Ababio (Uzbekistan), Amshika Amar (India, Rwanda), Wasiq Ismail (the Arab Republic of Egypt), and Maria Sh- karatan (Ghana, Indonesia; synthesis of all case studies). Dominik Naeher, consultant, and Virginia Ziulu, data scientist, prepared impact assessments of energy efficiency projects in Malawi and Mexico. Nikki Tan conducted a structured literature review. The report benefited from the IEG evaluation insight note on transport decarbonization led by Elisabeth Goller, senior evaluation officer, and portfolio-related contributions from Unurjargal Demberel, evaluation officer; Kavita Mathur, evaluation officer; Mari Noelle Lantin Roquiz, technical specialist; and Ridwan Bolaji Bello and Ebru Karam- ete, evaluation analysts. Emelda Cudilla and Romayne Pereira, program as- sistants, provided administrative support. Aarre Laakso, consultant, assisted with developmental and line editing. vi External subject matter experts served as peer reviewers: Kofi Agyarko, director of Renewable Energy, Energy Efficiency and Climate Change, Energy Commission, Ghana; Barry Bredenkamp, general manager, South African National Energy Development Institute; Daniel Kammen, distinguished professor of energy, University of California, Berkeley; and Melanie Slade, senior manager, energy efficiency, International Energy Agency. IEG is grateful to the global experts and representatives of country gov- ernments, private sector entities, international and bilateral development organizations, and academia who responded to questions and participated in interviews as part of the evaluation process. Consultations and interviews were also carried out with multiple World Bank Group staff members and managers across various Global Practices and industry groups. IEG greatly appreciates the support that country units provided for conducting virtual missions for the case studies. The IEG team also acknowledges the inputs on the Energy Sector Management Assistance Program provided by Jessica Kyle and Anne-Marie Verbeken of ICF International. The team offers special thanks to Carolyn Fischer, research manager; Vivien Foster, chief economist; Aart C. Kraay, deputy chief economist and direc- tor of development policy; Demetrios Papathanasiou, global director; and Jevgenijs Steinbuks, senior economist, for their comments on various energy efficiency–related concepts and inputs to working papers. World Bank Group staff bear no responsibility for this evaluation’s content. Independent Evaluation Group World Bank Group    vii Overview An Urgent Need to Scale Demand-Side Energy Efficiency Improving energy efficiency—using less energy to do the same amount of work—has both supply-side and demand-side aspects. Improvements in en- ergy efficiency are reductions in the energy required to maintain or improve energy services to households, businesses, and communities. Supply-side en- ergy efficiency approaches target energy generation via grid infrastructure, utilities, and power producers. Demand-side energy efficiency (DSEE) focus- es on the energy use of industries, commercial entities, and households. This evaluation focuses on the World Bank Group’s approaches to DSEE and opportunities to scale them up. It focuses on DSEE for four reasons. First, scaling up DSEE can substantially reduce energy demand, avoiding green- house gas (GHG) emissions. More than 40 percent of the reduction in GHG World Bank Group Support to Demand-Side Energy Efficiency  Overview emissions over the next 20 years could come from energy efficiency (IEA 2020a). Second, DSEE interventions face policy, institutional, market, and behavioral barriers that the Bank Group can help address. Third, the Bank Group has committed—in the World Bank Group Climate Change Action Plan 2021–25 (World Bank 2021b)—to scaling up DSEE. Fourth, other recent Inde- pendent Evaluation Group evaluations, including evaluations of Bank Group support for energy access and renewable energy, have covered supply-side issues (World Bank 2015, 2020). Investments at scale are needed to realize the untapped potential of ener- gy efficiency. The rate of improvement in global energy efficiency (the rate at which global gross domestic product per unit of energy used increases) needs to nearly double to meet the Sustainable Development Goal (SDG) 7 (energy access) target. Moreover, it needs to increase by more than 2.7 times to achieve the ambitions of the Paris Agreement. In addition, meeting SDG 7, SDG 13 (climate action), and the ambitions of the Paris Agreement—and reaping the other economic and social benefits associated with energy effi- ciency—requires closing a $500 billion annual gap in global investments. viii Furthermore, recent crises—the COVID-19 pandemic and the energy cri- sis—have highlighted the importance of DSEE and the urgency of scaling it up. These two crises have exacerbated Bank Group clients’ systemic energy challenges, especially in Europe and Central Asia. The COVID-19 pandemic highlighted the potential for DSEE to improve air quality (by reducing emis- sions, which can contribute to pulmonary diseases), create jobs, increase in- comes, and improve productivity during economically damaging lockdowns. As a result of the energy crisis caused by the Russian Federation’s invasion of Ukraine, European ambitions for energy security hinge on policy measures to maximize the use of DSEE measures in both industrial and residential settings (including in relation to heating and cooling of buildings). Addressing climate goals—including SDG 13 and the Paris Agreement— requires scaling up DSEE interventions vertically and horizontally. Scaling up DSEE refers to influencing or increasing clients’ DSEE activities, inputs, or investments across or within sectors. Vertical scaling targets multi- asset owners within a sector, whereas horizontal scaling targets entire multisectoral supply chains. The evaluation was conducted at the global, country, and intervention levels. It used a combination of methods: literature review, portfolio sampling, benchmarking, key informant interviews, country case studies, economet- ric analysis, surveys of best practices, and analysis of existing surveys. This evaluation is part of the climate change and environmental sustainability Independent Evaluation Group World Bank Group    ix theme in the Independent Evaluation Group’s work program. A Mixed Record of Demand-Side Energy Efficiency Success across the World Bank Group The World Bank’s investment project financing (IPF) and the investment services of the International Finance Corporation (IFC) have been the main instruments supporting DSEE; the Multilateral Investment Guarantee Agency (MIGA) supported DSEE through guarantees. During the evaluation period, between 2011 and 2020, World Bank IPF and IFC investment services delivered 62 percent of the DSEE projects and 70 percent of the total DSEE financing volume ($19.4 billion over 354 interventions). In this period, MIGA issued 11 guarantees across three countries under eight projects, for $1.4 billion in DSEE support, primarily for public infrastructure (hospitals). Two global, advisory-oriented programs supported Bank Group interventions in DSEE. The World Bank Energy Sector Management Assistance Program (ESMAP) and the IFC Green Buildings Market Transformation Program helped address market, institutional, and information barriers in client countries by piloting new DSEE approaches and crowding in investment. Bank Group DSEE interventions have gravitated toward middle-income countries (MICs) and industrialized lower-middle-income countries (LMICs). More than half of World Bank IPF and IFC investment services went to MICs and industrialized LMICs. Bank Group lending reached the industrial and commercial building market segments and public infrastructure (for exam- ple, public buildings, lighting), but it has not fully addressed all of the most energy-consuming and hardest-to-abate market segments. The industrial segment accounts for 38 percent of total global final energy use, followed by buildings and transport. Hard-to-abate manufacturing sectors—including cement, steel, glass, and chemicals—are therefore high priorities. World Bank Group Support to Demand-Side Energy Efficiency  Overview The World Bank’s DSEE projects were effective, but most did not scale, espe- cially horizontally. World Bank projects mostly met their outcome objectives (95 percent of closed projects were rated moderately satisfactory or above), with similar success across IPF projects and development policy operations. However, most interventions were one-off pilots, and fewer than one-quarter of the World Bank DSEE interventions specifically aimed at scaling up. Some DSEE interventions—mostly in MICs and industrialized countries (China, India, Mexico, and Türkiye)—supported vertical but not horizontal scaling. IFC investment projects had limited effectiveness, partly due to overambi- tious targets, although the pairing of IFC investment and advisory did lead to some scale-up. Based on an analysis of IFC self-evaluations and Indepen- dent Evaluation Group validation notes, IFC had varied success in achieving the two primary DSEE development outcomes of energy savings and GHG emissions reduction (62 percent for IFC advisory services; 37 percent for IFC investment services). Many investment projects attempted to demonstrate outcomes incommensurate with their design and scope. Examples include projects that, while supporting a single firm or intermediary, aimed at DSEE x market creation (such as targeting new end-user segments to reduce GHG emissions and improve energy efficiency at scale) or system-level trans- formation (such as reducing energy use across entire supply chains). Yet, when paired with advisory services, cofinanciers, development partners, and intermediary support for global standards (such as the Excellence in Design for Greater Efficiencies [EDGE] standards and certification process), some in- vestments scaled up both vertically and horizontally (for example, in South Africa and Colombia). The World Bank Group Has Proven It Can Overcome Challenges to Scaling Demand-Side Energy Efficiency The Bank Group faces three main challenges in scaling up DSEE at the country level. First, demand for Bank Group DSEE support from LMICs was limited by volatile client priorities and the need for countries to address supply-side concerns (such as increasing generation, access, and distribu- tion). Disincentives to addressing DSEE include surplus power supply (for example, in Ghana and Indonesia), surplus fossil energy sources for elec- trification, and (at the consumer level) below-cost energy prices (also in Ghana and Indonesia). Second, the Bank Group has been unable to articulate tangible DSEE benefits to clients, partly because in many countries it has lacked the opportunity to demonstrate DSEE’s potential benefits for society and the economy (such as health improvements and job creation). Third, the Bank Group has not sufficiently leveraged global programs (such as ESMAP Independent Evaluation Group World Bank Group    xi and the Green Buildings Market Transformation Program) and its convening power. An external evaluation of ESMAP (ICF 2020), for example, concluded that the program had supported World Bank energy efficiency loans but had not yet developed a global reach or reputation. Yet, the Bank Group succeeded in helping clients scale up under certain cir- cumstances. Successful scale-up was possible when (i) countries had robust policy environments for energy efficiency; (ii) clients received strong advisory and analytical work; (iii) the Bank Group targeted relevant clients, such as state-owned enterprises (SOEs); (iv) the interventions used de-risking instru- ments; and (v) clients benefited from cumulative Bank Group engagements. A robust policy environment for energy efficiency in the client country was a common factor in all successful cases. Committed MICs had more success with DSEE than other countries did. However, an optimum policy environ- ment was not a prerequisite for success. Bank Group support to improve the policy environment in parallel with DSEE investments—rather than the tra- ditional sequence of financing after policy reforms—sometimes led to suc- cessful scaling in LMICs. Whether provided in advance or in parallel, Bank Group advisory and analytical support enhanced the policy environment to facilitate successful DSEE scale-up in various industrialized countries (for example, China, India, Türkiye, and Vietnam). ESMAP, a global knowledge convener on energy, drove the World Bank’s advisory services and analytics projects for DSEE measures, accelerating investments in DSEE. Likewise, through the Green Buildings Market Transformation Program, IFC advisory services demonstrated scale-up of client commitments and early develop- ment outcomes in green buildings across several Regions. Targeting multi-asset infrastructure owners such as SOEs, using de-risking instruments, and supporting cumulative engagements have led to successful DSEE scale-up. SOEs and national development banks remain central actors in World Bank Group Support to Demand-Side Energy Efficiency  Overview the energy sectors of client countries. Scaling by targeting SOEs and national development banks has proven successful in MICs and industrialized clients because they own or operate multiple infrastructure assets that need DSEE im- provements. De-risking mechanisms (such as risk-sharing facilities) and guar- antee instruments can promote DSEE approaches and encourage onlending through local financial institutions. Similarly, repeat engagements in targeted end-user segments have led to DSEE scale-up, as in Türkiye and Vietnam. More Coherence Needed to Close the Gaps Coherent approaches lead to DSEE scale-up. Coherence refers to the extent to which Bank Group DSEE interventions support or undermine each other and the interventions of Bank Group partners in reaching SDG 13, SDG 7, and Paris Agreement alignment goals. Coherence can be assessed internal- ly (among the Bank Group institutions) and externally (between the Bank Group and other actors). Coherence includes coordination, complementary activities, and alignment with standards. Only by working consistently in xii alignment within the Bank Group and with external partners can the Bank Group scale DSEE support sufficiently to close the gaps in investment and development outcomes. A minority of World Bank energy efficiency interventions had both supply- side energy efficiency and DSEE elements. The World Bank has used various approaches (such as smart metering and time-of-use pricing) to balance supply and demand while promoting energy efficiency. However, only 25 percent of World Bank supply-side energy efficiency projects had DSEE elements. World Bank DSEE interventions were limited across the energy, water, agri- culture, and transport sectors. The Energy and Extractives Global Practice (GP) has made the most contributions toward DSEE. The Transport GP was responsible for only 6 percent of World Bank DSEE commitments ($0.8 bil- lion in four projects over 10 years), and the Water GP was responsible for only 5 percent. Transport interventions especially need to incorporate ener- gy efficiency because the transport sector is responsible for approximately 27 percent of total energy-related GHG emissions, and energy efficiency in transport offers enormous unexploited potential for mitigation. IFC’s approach to DSEE has been well coordinated across diverse sectors (industry groups and business lines). IFC has mainstreamed DSEE invest- ment projects across three industry groups—Financial Institutions Group; Infrastructure and Natural Resources; and Manufacturing, Agribusiness, and Independent Evaluation Group World Bank Group    xiii Services—and across all its business lines. IFC’s internal coherence has led to horizontal scaling. IFC has helped client governments address climate change and energy shortages at scale by bring- ing together complementary blended finance programs in various sectors and countries. Examples include Colombia and Türkiye. IFC’s Partnership for Cleaner Textile program is a good example of organizing for coherence across water and energy efficiency. Coordination units are critical to bringing internal coherence to DSEE work. The unwinding of the World Bank’s DSEE community of practice and the DSEE-focused global solutions group in recent years has limited collabora- tion across World Bank GPs to support horizontal scaling. Conversely, the IFC Climate team has been essential in enabling IFC’s coherence in DSEE across sectors. Multilateral development banks, including the Bank Group, do not have a co- herent approach to DSEE. They do not consistently link DSEE interventions to the primary DSEE development outcomes of saving energy and reducing GHG emissions or other socioeconomic benefits. Most do not have a unified approach to advancing DSEE standards or communicating the benefits of DSEE to stakeholders. The World Bank has been externally coherent with cofinanciers but less so on advisory services. The World Bank has been externally coherent onlend- ing with cofinanciers (the Global Environment Facility and the Clean Tech- nology Fund) but less coherent on advisory support to clients and knowledge diffusion goals with client governments and partners. The external coher- ence of the World Bank Energy and Extractives GP and ESMAP in Vietnam is a best practice. Both the World Bank and IFC have been coherent with global building stan- dards. The World Bank’s ESMAP and the Carbon Finance Unit have promoted World Bank Group Support to Demand-Side Energy Efficiency  Overview global building standards and the benefits of enforcement since 2008 via advisory and knowledge work. IFC is coherent with global DSEE standards, with a singular focus on green buildings by using its EDGE certification and standards process to advance DSEE priorities across market segments. MIGA applies a coherent approach to green building standards in its proj- ects. MIGA DSEE clients (from industrialized countries) have been well positioned to embrace either the Leadership in Energy and Environmental Design or EDGE global standards for buildings, the latter being a standard advocated by IFC. In a hospitality-cluster project, IFC and MIGA are part- nering across some of the subprojects (individual hotels). MIGA and IFC are further supporting the client’s adoption of the EDGE certification standard. Untapped Opportunities to Scale Demand-Side Energy Efficiency The Bank Group has not fully tapped four opportunities to improve its Paris Agreement alignment, SDG 13, and SDG 7 contributions and facilitate DSEE xiv scale-up at the country level. Although the success factors described earlier provide guidelines for succeeding with typical DSEE engagements, the eval- uation identified the following untapped opportunities for the Bank Group to scale DSEE exponentially: (i) measuring indirect emissions and socioeco- nomic outcomes in DSEE interventions’ results frameworks, (ii) adopting an embodied carbon approach in project scoping and design, (iii) incorporating digital innovations into project designs, and (iv) integrating financial inno- vations into project designs. The Bank Group can improve its alignment with the Paris Agreement goals by aiming to reduce indirect GHG emissions, opening opportunities for DSEE horizontal scale-up, and broadening socioeconomic outcomes. Most Bank Group project interventions do not tackle indirect emissions (that is, emissions that are a consequence of an organization’s activities but occur at sources owned or controlled by another entity) or the related energy efficiency measures. Tackling indirect emissions can significantly boost the Bank Group’s alignment with the Paris Agreement objectives and SDG 13. The magnitude of the emissions problem related to climate change re- quires addressing both direct and indirect emissions, as indirect emissions are often responsible for an organization’s biggest GHG impacts. Thus, one way to strengthen the link between DSEE interventions and climate objec- tives would be to include the client’s full scope of emissions in the design of new Bank Group DSEE interventions and help clients measure emissions throughout the life of a project. The shadow carbon-pricing pilots in World Bank lending and IFC investment projects are a step in the right direction. Independent Evaluation Group World Bank Group    xv In the building market segment, embodied carbon (that is, all emissions associated with the development and use of construction materials) needs to be tackled. Embodied carbon refers to all GHG emissions related to de- forestation, manufacturing, transportation, installation, maintenance, and disposal of construction materials. Embodied carbon stands in contrast to operational carbon, which refers to emissions over the course of a building’s lifetime. Reaching net zero emissions requires minimizing end users’ total carbon (embodied and operational carbon) emissions and removing any re- sidual emissions. Bank Group approaches targeting total carbon in buildings can facilitate DSEE scale-up across supply chains because various subsectors of the economy supply construction materials to the building industry. Digital and financial innovations embedded in DSEE approaches can in- crease end-user awareness and improve DSEE adoption at scale. Optimized use of digital solutions (such as blockchain, smart sensors, and digital energy management systems) in DSEE approaches has improved end-user aware- ness to change energy consumption behaviors and increased adoption of DSEE measures in developed countries. For example, Opower’s cloud-based software for the utility industry uses artificial intelligence and behavioral science to guide users in reducing energy consumption, which results in cost savings. Similarly, optimal use of blended finance, performance guarantees, and capital market solutions have led to DSEE scale-up in developed and de- veloping countries. In the Arab Republic of Egypt, for example, a credit risk guarantee mechanism for DSEE supported by a 2005 World Bank and Global Environment Facility project attracted private capital and led to a scale-up of DSEE investment. Conclusions and Recommendations DSEE is important for global sustainability, and the Bank Group has com- mitted to it. The World Bank made two overarching corporate commitments World Bank Group Support to Demand-Side Energy Efficiency  Overview for which DSEE is critical: (i) to achieve Paris Agreement alignment by 2023 (World Bank) or 2025 (IFC and MIGA), and (ii) to contribute to the achieve- ment of SDG targets, which the Bank Group has internalized in its overarch- ing poverty alleviation and shared prosperity goals. The weight of the global priorities and the limited scale-up on DSEE to date leave the Bank Group with the need to fully reorient its DSEE approaches and outcome aspirations from an energy savings focus to a broader decar- bonization focus. With this necessary pivot of DSEE approaches toward glob- al priorities as the backdrop, this evaluation proposes four near-term actions that the Bank Group should take. Recommendation 1 (Bank Group). Intensify DSEE support to MICs for decarbonization and wider socioeconomic benefits. By supporting MICs in scaling up DSEE, the Bank Group would make the most difference in closing GHG emissions gaps while also contributing to economic and social devel- opment outcomes. Intensifying scale-up in MICs requires an increased focus by the World Bank on multisectoral and horizontal scale-up approaches in xvi project design. Similarly, this recommendation entails an increased role in MICs for IFC and MIGA—including through IFC upstream interventions and MIGA business development approaches—in countries that are ready for the greening of public assets and assets of SOEs (for example, China, India), subject to client demand. Recommendation 2 (World Bank and IFC). Develop energy efficiency sector-specific approaches in a select group of LMICs that seek productivity gains alongside or via DSEE, even if energy efficiency policy reforms are in early stages. Bank Group DSEE efforts in countries with a policy environ- ment that is not conducive to energy efficiency reforms or low energy use or emissions per capita, or that have inefficient capital allocation mechanisms to energy generation (for example, fossil subsidies), are unlikely to lead to meaningful outcomes. Select LMICs, however, that are making deliberate efforts to increase firms’ productivity while also achieving DSEE are prom- ising scale-up targets for the World Bank and IFC, especially if they focus their DSEE interventions on energy-intensive sectors or subsectors, such as the industrial market segment in Uzbekistan or the commercial construction market segment in Indonesia. Parallel technical assistance and IFC upstream and advisory services can help target new client types and cumulative in- vestments, subject to client demand. Recommendation 3 (World Bank and IFC). Expand DSEE approaches by incorporating the reduction of indirect emissions (scope 3), including em- Independent Evaluation Group World Bank Group    xvii bodied and operational carbon, in DSEE project design. The current approach of designing for direct (scope 1) emissions is necessary but not sufficient for the pivot to decarbonization and for steering greater financing flows to- ward DSEE as part of the multilateral development banks’ Paris Agreement alignment approach. This recommendation entails incorporating scope 3 (and in some cases scope 2) risks for these emissions ex ante (that is, at the time of project design discussions, during post-client mandate activities, and when crafting loan agreements). This recommendation implies, for example, focusing on horizontal scaling through longer-term, repeat engagement, and multisector approaches (similar to what the Bank Group has achieved in India and Mexico) that cut across upstream and downstream activities. In this regard, IFC’s recent advisory services initiative Partnership for Cleaner Textile is promising. Recommendation 4 (World Bank and IFC). Exploit untapped DSEE op- portunities and help clients leapfrog by exploring cross–Practice Group (World Bank) and cross–industry group (IFC) approaches. This would entail integrating DSEE with untapped opportunities, such as digital and financial instrument innovations, that could support leapfrogging efforts in some cases. Examples include convening and supporting existing IFC clients (for example, firms operating in retail supply chains, top GHG-emitting firms, and firms owning and operating data centers) to incorporate digital solu- tions, such as intelligent monitoring and artificial intelligence–based energy optimization within their building portfolios; leveraging supply-side energy efficiency activities (for example, combining electricity utility upgrades with innovative guarantee mechanisms to promote DSEE); using multistakehold- er approaches to invest in local technology start-ups (as done with Negawatt in Ghana); designing behavioral policy interventions (China); and commu- nicating successful pilot cases. This recommendation would entail explor- ing integrated approaches, for example, among the Energy and Extractives; Digital Development; Macroeconomics, Trade, and Investment; and Finance, Competitiveness, and Innovation GPs, as well as among IFC’s Infrastructure World Bank Group Support to Demand-Side Energy Efficiency  Overview and Natural Resources; Manufacturing, Agribusiness, and Services; and Dis- ruptive Technologies and Funds industry groups. xviii 1 | Background and Context Highlights Energy efficiency—a reduction in the energy required to maintain or improve energy services—comprises both supply-side energy efficiency, such as the efficiency of power plants, and demand-side energy efficiency (DSEE), such as the efficiency of buildings. This evaluation focuses on DSEE for four reasons: (i) DSEE can sub- stantially reduce energy demand, avoiding greenhouse gas emis- sions that contribute to climate change; (ii) the World Bank Group is in a position to help address the policy, institutional, market, and behavioral barriers that limit DSEE interventions; (iii) the Bank Group committed in the Climate Change Action Plan to scaling DSEE; and (iv) other recent evaluations of the Independent Evalua- tion Group have addressed supply-side energy efficiency. DSEE contributes to meeting Sustainable Development Goal (SDG) 13 on combating climate change, the Paris Agreement goals, and the global energy efficiency target under SDG 7. It could also contribute to several other SDGs, including SDG 3 (health), SDG 8 (economic growth, decent work, and productivity), and SDG 11 (sus- tainable cities and communities). A serious challenge is that meeting the SDG 7, SDG 13, and Paris Agreement goals requires increasing the annual rate of improve- ment in energy efficiency globally—the rate at which global gross domestic product per unit of energy used increases—by 2.7 times by 2030. This requires increasing global investment in DSEE from all sources by $500 billion per year to $790 billion annually from now until 2030. Accelerating the rate of energy efficiency improvement enough to meet the climate goals requires scaling DSEE interventions both 1  vertically (within industries) and horizontally (across industries). The evaluation aims to assess how well the Bank Group supports client countries in achieving end-use energy savings by scaling DSEE vertically and horizontally. It addresses the effectiveness of the Bank Group’s DSEE interventions, their coherence (both within the Bank Group and with partners), and opportunities for scaling. 2   Context Energy efficiency reduces the energy required to maintain or improve ener- gy services to households, businesses, and communities (World Bank 2015). Energy efficiency improves when the same energy services require less energy or when energy services are enhanced using the same amount of energy. For example, an LED bulb that uses less energy than an incandescent bulb to pro- duce the same amount of light is considered more energy efficient (IEA 2014). Efforts to improve energy efficiency can be categorized as supply- side approaches and demand-side approaches (figure 1.1). Supply-side energy efficiency (SSEE) approaches target energy generation via grid infrastructure, utilities, and power producers. They include upgrading and retrofitting power plant turbines, efficient transmission lines, and smart- grid applications and so on, reduce energy losses and capture waste heat during energy generation. Demand-side energy efficiency (DSEE) focuses on the energy use of industries, commercial entities, and households. It includes upgrading industrial plants, equipment, and appliances; retrofitting public infrastructure and residential buildings; improving fuel efficiency; and end-user incentives and load-shape modification programs by utilities and service providers. (Cogeneration, such as rooftop solar energy, refers to renewable energy generated for the household rather than to energy efficiency.) The main market segments for DSEE are residential, commercial, industrial, and public sector energy customers. Independent Evaluation Group World Bank Group    3 The evaluation focuses on the World Bank Group’s DSEE approaches for four reasons. » First, DSEE interventions can substantially reduce energy demand, avoid- ing greenhouse gas (GHG) emissions. Emissions of carbon dioxide (CO2) and other GHGs into the atmosphere are critical culprits in climate change, and the energy sector is responsible for about three-quarters of the world’s GHG emissions (IEA 2021c). Improvements in DSEE reduce demand and save en- ergy, avoiding fossil fuel consumption and decreasing GHG emissions. More than 40 percent of the reduction in GHG emissions over the next 20 years could come from energy efficiency (IEA 2020a). Global use of LED light bulbs in 2017 alone reduced carbon emissions by 570 million metric tons, nearly 2 percent of total emissions. » Second, the Bank Group can help address policy, institutional, market, and behavioral barriers that undermine DSEE interventions. Policy and institu- tional barriers include inadequate or unfavorable regulatory and legal frame- work and limited infrastructure investment. Market failures include informa- tion asymmetries among energy producers and consumers and capital market imperfections that lead to private sector underinvestment in energy efficien- cy. Behavioral barriers include habits, social norms, lack of trust, and lack of awareness of the economic benefits of energy efficiency measures. Figure 1.1. Energy Efficiency Supply and Demand Residential customers World Bank Group Support to Demand-Side Energy Efficiency  Chapter 1 Generation Transmission Commercial customers Supply Purchase power plant retrofitting, smart grid development solutions, and so on, for supply-side energy efficiency Industrial customers Demand Purchase heating, cooling, and lighting solutions for demand-side energy efficiency Public sector customers Source: Independent Evaluation Group. 4 » Third, the Bank Group has committed to scaling up DSEE, including through the recent Country Climate and Development Reports. Scaling up DSEE refers to influencing or increasing clients’ DSEE activities, inputs, or investments across or within sectors. The World Bank Group Climate Change Action Plan 2021–25 identifies energy efficiency as one of the largest untapped oppor- tunities for energy savings and climate mitigation and explicitly calls for scaling it up (World Bank 2021b). DSEE accounts for only one-quarter of the Bank Group energy portfolio (2011–20). The relatively small emphasis on DSEE is partly due to the difficulties in reaching the multiple and fragmented market segments, the limited interest of government counterparts, and—for the International Finance Corporation (IFC)—the challenge of achieving an adequate return on investments in DSEE interventions. The Bank Group takes Nationally Determined Contributions into account in its action plans and the recent Country Climate and Development Reports. » Fourth, other recent Independent Evaluation Group (IEG) evaluations have covered supply-side issues. Evaluations on supply-side topics include assess- ments of Bank Group support for energy access and renewable energy (World Bank 2015, 2020). DSEE contributes to Sustainable Development Goal (SDG) 13 on climate change, the Paris Agreement, the target under SDG 7 on improving global energy efficiency, and several other economic and social benefits. In addition to the climate goals, DSEE could contribute to SDG 3 (health), SDG 8 (eco- Independent Evaluation Group World Bank Group    5 nomic growth, decent work, and productivity), and SDG 11 (sustainable cities and communities). SDG 7 targets doubling the global rate of improvement in energy efficiency (the rate at which global gross domestic product per unit of energy used increases) by 2030. The average annual rate of improvement in energy efficiency between 2016 and 2020 was 1.54 percent (shown in the light-blue bar in figure 1.2). Meeting SDG 7 requires an energy efficiency improvement rate of at least 3 percent annually (SEforALL 2021; 2030 SDG 7 bar in figure 1.2). Meeting the goal would require the global rate of im- provement in energy efficiency to increase by approximately 1.9 times from the 2016–20 average (line labeled “SDG 7 gap” in figure 1.2). Moreover, to achieve net zero carbon emissions by 2050, as required by the Paris Agree- ment, the global rate of improvement in energy efficiency would need to rise to 4.2 percent by 2030 (IEA 2021c). (Net zero emissions is a state in which the GHGs going into the atmosphere are balanced by their removal from the atmosphere, including residual emissions; reaching this target is necessary for halting climate change.) Meeting this goal would require the global rate of improvement in energy efficiency to increase by more than 2.7 times from the 2016–20 average (“Paris/Net zero gap” line in figure 1.2). Figure 1.2. Global Annual Rate of Energy Efficiency Improvement 5 Rate of improvement in energy efficiency (%) Paris/Net zero gap ≈ 2.7× 4 3 SDG 7 gap* ≈ 1.9× 2 1 World Bank Group Support to Demand-Side Energy Efficiency  Chapter 1 0 2016 2017 2018 2019 2020 2016–20 2030 2030 Avg. SDG 7 Rev.* Net zero† Year Sources: Inernational Energy Agency 2021a; Sustainable Energy for All 2021. Note: Avg. = average; Rev. = revised; SDG = Sustainable Development Goal. * Revised SDG 7 target. † Required by 2030 to achieve net zero by 2050 per the Paris Agreement. Meeting SDG 7, SDG 13, and the Paris Agreement—and reaping the economic and social benefits associated with energy efficiency—requires a substantial increase in investment. Global annual investments in energy efficiency have been less than $300 billion since 2015 (figure 1.3).1 To increase the rate of improvement in energy efficiency to 4.2 percent, as is required to reach net zero emissions, align with the Paris Agreement, and meet SDG 13 and SDG 7, world energy efficiency investments would need to rise from approximately $290 billion in 2021 to $790 billion annually by 2030—a yearly gap of ap- proximately $500 billion, requiring an increase in investment of more than 2.7 times (IEA 2021a). 6  lobal Annual Investment in Energy Efficiency, 2015–21: Figure 1.3. G Past Trends and Future Need 800 700 600 Investment (US$, billions) Gap ≈ US$500 billion 500 400 300 200 100 0 2015 2016 2017 2018 2019 2020 2021* 2030† Year Source: International Energy Agency 2021a. Note: An energy efficiency investment is defined as the incremental spending on new energy-efficient equipment or the full cost of refurbishments that reduce energy use. The intention is to capture spend- ing that reduces energy consumption. Under conventional accounting, part of this is categorized as consumption rather than investment. * Estimate. † Projected need. Addressing climate goals—including SDG 13 and the Paris Agreement—re- quires scaling DSEE interventions both vertically and horizontally. Vertical Independent Evaluation Group World Bank Group    7 scaling introduces DSEE solutions to reduce direct emissions from buildings, factories, vehicles, or other assets belonging to a single sector (table 1.1). An example of vertical scaling in the buildings sector would be going from improving the efficiency of air conditioning systems in a few dozen buildings to improving them in several hundred buildings. Horizontal scaling intro- duces energy efficiency solutions in multiple sectors, reducing both direct emissions and indirect emissions (emissions that are a consequence of an organization’s activities but occur at sources owned or controlled by another entity). An example of horizontal scaling would be supporting a construction company in reducing not only its own emissions but also the emissions of its customers (by constructing buildings with low energy use and emissions) and partners (such as cement and glass firms, transport and logistics firms, and banks). Both vertical and horizontal scaling can involve scaling across multiple organizations, assets (for example, multiple buildings), and countries.  ertical and Horizontal Scaling in Demand-Side Energy Table 1.1. V Efficiency Vertical Scaling Horizontal Scaling Emissions targeted Direct emissions Direct plus indirect emissions Sectors targeted Single sector Multisector Other aspects of scale Multiorganization, multi-asset, multicountry Source: Independent Evaluation Group. In addition to addressing climate goals, energy efficiency plays a key role in partly mitigating the short- and long-term effects on energy security arising from the war between the Russian Federation and Ukraine in the Europe and Central Asia Region and beyond. Roughly one-third of the gas demand of the European Union (EU) is used for heating buildings across various market segments (for example, industrial, commercial, household) and another one- third for electricity production (Det Norske Veritas 2022). Almost 20 percent World Bank Group Support to Demand-Side Energy Efficiency  Chapter 1 of EU gas demand comes from the manufacturing industry and the remain- der from the petrochemical industry. European policy makers are determined to slash the EU’s dependence on Russian gas by two-thirds by the end of 2022. European energy security ambition therefore hinges on additional pol- icy measures on both energy supply and energy demand. On demand, beyond nudging end-user (especially household) behavior toward lower energy use, there is scope for a concerted policy push for energy efficiency via multi- sectoral approaches (for example, joint energy-transport approaches) and budget support. Scaling DSEE in the Europe and Central Asia Region and be- yond will mitigate the long-term negative impact of dependence on Russian gas. Only about 1 percent of the EU’s building stock is renovated each year (OECD 2022). A rapid extension to an additional 0.7 percent, targeting the least energy-efficient buildings across various market segments (industrial, commercial, and household), would save more than 1 billion cubic meters of gas use per year and increase employment. The COVID-19 pandemic and the consequent global economic downturn have intensified the relevance and urgency of improving energy efficiency. 8 The COVID-19 pandemic has (i) reduced the ability of households to pay for DSEE upgrades, given the loss of income associated with the decline in economic activity, and (ii) slowed down the financing of DSEE solutions as national priorities have changed. As a result, even greater efforts will be needed to meet the energy efficiency savings and climate change targets in a post–COVID-19 world (IEA 2020a). During the COVID-19 pandemic, there has been greater recognition of DSEE’s role in reducing pressure on public budgets and achieving socio- economic impacts such as increases in productivity and job creation. The various analyses of (i) the ecological links among emissions reduction, air quality, DSEE adoption, COVID-19 mortality rates, and health (Wu et al. 2020); (ii) DSEE’s relationship to job creation and industrial productivity (IEA 2020b); and (iii) the relationship between energy-efficient living con- ditions through DSEE and vulnerable household incomes (Eurofound 2016) suggest that DSEE offers wide fiscal and socioeconomic benefits beyond energy-use savings and reduction of GHG emissions. Evaluation Purpose, Questions, and Methods The overall purpose of this evaluation is to assess how well the Bank Group is supporting client countries in achieving end-use energy savings by scaling DSEE vertically and horizontally. This report addresses three specific evalua- tion questions and subquestions: Independent Evaluation Group World Bank Group    9 1. How effective have the Bank Group’s DSEE interventions been in achiev- ing the development outcome of end-use energy savings? 2. How coherent are the Bank Group’s DSEE interventions (i) internally (for example, coordination and joint initiatives across World Bank Practice Groups and IFC sectors) and (ii) externally (for example, across develop- ment partners and client governments)? 3. What untapped opportunities and mechanisms exist for the Bank Group to support clients to realize their energy efficiency potential? (i) What are the untapped opportunities for Bank Group engagement to support energy efficiency across sectors? (ii) What innovative mechanisms proved effective and sustainable and can be mainstreamed to scale DSEE interventions? The evaluation was conducted at the global, country, and intervention levels. This evaluation is part of the climate change and environmental sustainabil- ity theme in IEG’s work program. The evaluation addressed the evaluation questions through a combination of methods: literature review, portfolio sampling, multilateral development bank (MDB) benchmarking, key infor- mant interviews, country case studies, and econometric analysis (figure 1.4; see appendix A for methodology). The third evaluation question was an- swered based on best practices from within the Bank Group and through a literature review and analysis of existing surveys outside the Bank Group. The evaluation team conducted a portfolio analysis of Bank Group energy ef- ficiency projects supporting (i) demand-side-only interventions and (ii) both supply-side and demand-side interventions. The team also selected a subsa- mple of both categories of projects for a deep-dive analysis of outcomes. The evaluation team at first identified 562 Bank Group energy efficiency projects. Out of this portfolio universe, the team identified 408 projects with de- mand-side-only energy efficiency components (for example, IFC investment services) or a combination of SSEE and DSEE components (for example, World Bank development policy financing [DPF]). Out of 408 projects, the World Bank Group Support to Demand-Side Energy Efficiency  Chapter 1 evaluation team could not ascertain the commitment-value equivalent for 54 World Bank advisory services and analytics (ASA) projects. The evaluation team sampled 133 projects out of 354 (408 total portfolio minus 54 ASA proj- ects) across the three institutions for a deep-dive analysis of outcomes (refer to appendix A for more details). The IFC project outcome analysis was based on IFC project self-evaluations and IEG validations. The self-evaluations are selected based on a randomly stratified sample from all approved and com- mitted projects. The evaluation team included both investment project financing (IPF) and development policy loans in the World Bank lending portfolio but account- ed for them differently. The evaluation team included in the portfolio the total number of IPFs that supported DSEE and their full loan commitment amounts. The team also included in the portfolio the total number of devel- opment policy operations (DPOs) that supported DSEE, but only the share of the loan commitment amounts specifically related to prior actions sup- porting DSEE. The share of DPOs’ commitments included in the portfolio 10 as specifically supporting DSEE prior actions was approximately $3 billion (20 percent of the DPOs’ total commitment amount of $15 billion).  apping Evaluation Questions to Evaluation Methodologies Figure 1.4. M and Techniques Portfolio sampling and review MDB benchmarking Global-level Effectiveness analysis Literature review Key informant Country-level Coherence interviews analysis Untapped Intervention-level opportunities Country case analysis studies Econometric analysis Independent Evaluation Group World Bank Group    11 Source: Independent Evaluation Group. Note: MDB = multilateral development bank. 1 Throughout the evaluation, we provide data on demand-side energy efficiency, specifically when they are available and fall back to data on energy efficiency more generally when specif- ic data on demand-side energy efficiency are not available. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 1 12 2 | Effectiveness, Scale-Up Challenges, and Factors of Success Highlights The World Bank Group’s demand-side energy efficiency (DSEE) approaches at the project level were mostly effective but did not lead to sufficient scale-up at the country level in terms of DSEE financing and outcomes. World Bank DSEE interventions supported vertical scaling most- ly in middle-income and industrialized countries, but support for horizontal scaling was limited in countries at all income levels. The International Finance Corporation achieved scaling, including hori- zontal scaling, when combining investment and advisory work. The evaluation could not assess the Multilateral Investment Guarantee Agency’s effectiveness because the Agency’s projects were being evaluated after this report was underway or had not yet met oper- ational maturity to undergo evaluation at the time this report was underway. Global programs—the World Bank Energy Sector Management Assistance Program and the International Finance Corporation Green Buildings Market Transformation Program—helped address market, institutional, and information barriers in a handful of middle-income and industrialized countries. Challenges to scaling DSEE interventions include (i) client prefer- ences for support of energy supply, (ii) volatile priorities for DSEE, (iii) the Bank Group’s inability to articulate tangible DSEE benefits to clients, and (iv) insufficient leverage of global programs and Bank Group convening power. 13  Successful scale-up was possible when (i) countries had robust policy environments, (ii) clients received strong advisory and an- alytical work, (iii) the Bank Group targeted large greenhouse gas– emitting entities such as state-owned enterprises, (iv) the interven- tions used de-risking instruments, and (v) clients benefited from cumulative engagements. 14   Demand-Side Energy Efficiency Portfolio During fiscal years (FY)11–21, the Bank Group committed to 354 DSEE operations and 54 World Bank ASA projects (table 2.1). World Bank IPF and IFC investment services represented a large share of the DSEE portfolio during the evaluation period, at 83 projects and $8.7 billion commitment volume for IPFs and 137 projects and approximately $5 billion commitment volume for IFC investment services. DPF with DSEE measures corresponded to approximately $3 billion in lending (based on the share of the DSEE- related prior actions). The World Bank approved three Program-for-Results operations (in China, India, and Serbia) over 10 years. During the evaluation period, the Multilateral Investment Guarantee Agency (MIGA) issued 11 guarantees for eight projects in Bangladesh, Djibouti, and Türkiye. MIGA has increased its energy efficiency portfolio in the past few years, but the evaluation could cover only projects closed and validated between FY11 and FY21. The Energy and Extractives Global Practice (GP) led with the largest share of the World Bank DSEE investment portfolio by number of projects (51 percent), followed by the Macroeconomics, Trade, and Investment GP (9 percent). The Manufacturing, Agribusiness, and Services industry group (65 percent) and the Financial Institutions Group (27 percent) delivered most of the IFC investments by volume. DSEE interventions reached 82 countries but gravitated toward upper- middle-income countries (UMICs) and lower-middle-income countries (LMICs). Most Bank Group lending went to LMICs and UMICs, which have Independent Evaluation Group World Bank Group    15 higher energy intensity than high-income countries (figure 2.1). The largest share of DSEE financing for both the World Bank and IFC was in Europe and Central Asia (35 percent), followed by sizable commitments across other Regions, except for the Middle East and North Africa, which had the lowest share (9 percent). By country, the largest shares of DSEE investments were in China, India, and Türkiye, driven by client demand.  orld Bank Group Demand-Side Energy Efficiency Portfolio, Table 2.1. W Fiscal Years 2011–20 No. Commitments (US$, billions) Type All projects All projects Closed projects Active projects IFC IS 137 4.97 2.57 2.40 IFC AS 80 0.21 0.06 0.14 World Bank IPF 83 8.73 2.12 6.61 World Bank DPF 43 2.90a 2.86 0.04 World Bank P4R 3 0.84 n.a. 0.84 World Bank ASA 54 n.a. n.a. n.a. MIGA guarantee 8b 1.79 0.54 1.25 Source: Independent Evaluation Group. Note: World Bank ASA is a nonlending product. Out of 408 projects, 54 World Bank ASA could not be classified into supply side or demand side, and no specific commitments value was attached to them. AS = advisory services; ASA = advisory services and analytics; DPF = development policy financing; IFC = International Finance Corporation; IPF = investment project financing; IS = investment services; MIGA = Multilateral Investment Guarantee Agency; n.a. = not applicable; P4R = Program-for-Results. a. The $2.9 billion of World Bank demand-side energy efficiency (DSEE) DPF commitments shown in the table are the share of total World Bank DSEE DPF commitments ($15 billion) proportional to the share of DSEE-related prior actions in all prior actions under DSEE-related DPF (20 percent). Given the nuances between DPF and IPF accounting methods, this evaluation does not add up portfolio details across instruments. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 b. There were 11 MIGA guarantee contracts issued for 8 MIGA projects. Closed projects in MIGA’s con- text means nonactive projects (that is, terminated, canceled, or expired). Bank Group DSEE lending mainly addressed the most energy-consuming market segments (industry and buildings), but it has not yet fully addressed the needs in the transport market segment. Globally, the industrial segment accounts for 38 percent of total global final energy use, followed by build- ings and transport. The industrial sector has an extensive carbon footprint, especially in indirect emissions. The energy that buildings use across their life cycles (for example, in manufacturing construction materials, the use of fossil fuels to generate electricity and heat, and end-of-life disposal) is responsible, directly and indirectly, for approximately 37 percent of global energy-related CO2 emissions (IEA 2021d). World Bank lending (IPF, DPF, Program-for-Results) and IFC investment services targeted the needs in industrial, public, and commercial buildings and, to a lesser extent, in res- idential buildings as well. DSEE support in the transport market segment, however, has been minimal to date. 16  istribution of World Bank Demand-Side Energy Efficiency Figure 2.1. D Portfolio by Country Income Classification 5.0 4.5 Commitment amount �(US$, billions) 4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0 LIC LMIC UMIC HIC Country income classification World Bank IPF IFC IS Source: Independent Evaluation Group. Note: This distribution analysis does not include development policy financing for accounting reasons explained in the final paragraph in the Evaluation Purpose, Questions, and Methods section in chapter 1 and in table 2.1. HIC = high-income country; IFC = International Finance Corporation; IPF = investment project financing; IS = investment services; LIC = low-income country; LMIC = lower-middle-income country; UMIC = upper-middle-income country. Independent Evaluation Group World Bank Group    17 Two global, advisory-oriented programs supported Bank Group interventions in DSEE: the World Bank Energy Sector Management Assistance Program (ESMAP) and IFC’s Green Buildings Market Transformation Program (GBMTP). The World Bank ESMAP worked closely with the lending teams, IFC investment services teams, and other World Bank–administered trust funds (such as the Global Environment Facility [GEF]). ESMAP piloted new DSEE approaches through a combination of analytical work, research papers, and market diagnostics. The IFC GBMTP crowded in investment and advisory services support programmatically along with external partners to facilitate the greening of industrial and commercial market segments (box 2.1). Box 2.1. Global Programs Supporting Energy Efficiency The Energy Sector Management Assistance Program (ESMAP) is a partnership be- tween the World Bank and 22 development actors to help low- and middle-income countries reduce poverty and boost growth through sustainable energy solutions. ESMAP’s analytical and advisory services are fully integrated within the World Bank’s country financing and policy dialogue in the energy sector. Through the World Bank Group, ESMAP works to accelerate the energy transition required to achieve Sustain- able Development Goal 7 to ensure access to affordable, reliable, sustainable, and modern energy for all. It also helps shape Bank Group strategies and programs to achieve the Bank Group Climate Change Action Plan targets. ESMAP provides grants and technical support to countries through Bank Group oper- ational units. It delivers key global knowledge products deployed for country engage- ments and develops external partnerships with international organizations, research and development institutions, and industry associations. It works with several Bank Group regional energy units and sectors (such as transport, urban, water, health, and gender) and mobilizes donor resources for World Bank–executed activities (for exam- ple, cofinancing International Bank for Reconstruction and Development and Interna- World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 tional Development Association operations). ESMAP raised $330 million for its business plan for fiscal years 2017–20. ESMAP’s recent support (beginning in fiscal year 2021) to the World Bank Climate Change Action Plan implementation includes demand-side energy efficiency–related priorities. The Zero Carbon Public Sector initiative focuses on retrofitting public buildings. Industrial Decarbonization focuses on greening industrial buildings. Efficient and Clean Cooling aims to accelerate the uptake of sustainable cooling technologies and policies. The Clean Cooking Fund supports modern energy cooking services that are clean and efficient. ESMAP also supports power system planning, which helps reduce demand. The International Finance Corporation’s Green Buildings Market Transformation Pro- gram takes a four-pronged approach to incentivize market adoption of green building practices and support greater investment in green buildings. The four prongs are (i) Excellence in Design for Greater Efficiencies (EDGE) certification, (ii) green building codes and incentives-related support to client governments and firms, (iii) investment and advisory support for the industrial and commercial segments, and (iv) investment and advisory support for commercial banks. 18 (continued) Box 2.1. Global Programs Supporting Energy Efficiency (cont.) EDGE is the core of the Green Buildings Market Transformation Program. The Interna- tional Finance Corporation created the EDGE certification system to respond to the need for a measurable and credible solution to prove the business case for building green and to unlock financial investment. EDGE is an international green building cer- tification system implemented via a software tool. For prospective clients, EDGE sets a green building standard of 20 percent or more savings on energy, water, and embod- ied carbon use in materials and rewards developers for their green building projects through its certification system. EDGE certification helps create awareness and offers a verifiable performance indicator that financiers can lend against, advancing green building practices. EDGE has recently evolved to offer support to the development of zero carbon buildings, which are at least 40 percent more energy efficient than typical buildings and are fully powered by renewable energy. Sources: Energy Sector Management Assistance Program 2022; International Finance Corporation 2019. Effectiveness of Interventions and Scale-Up World Bank DSEE interventions have been effective. The World Bank DSEE lending program has been successful (95 percent of closed projects were rat- ed moderately satisfactory or above), with similar success across IPF projects Independent Evaluation Group World Bank Group    19 and DPOs.1 The World Bank has effectively achieved energy savings and GHG emission reduction targets in its investment financing projects, where measured. IEG analyzed all closed DSEE IPFs during FY11–21 (29 projects). Of the 21 proj- ects that specifically targeted energy savings or GHG emissions reduction, 80 percent fully achieved or exceeded their targets, and a further 18 percent partially achieved their targets. Only two did not achieve their targets. Ap- proximately 30 percent of the projects (8 out of 29) did not measure energy savings or GHG emissions reduction. They either combined supply-side im- provements with support for DSEE or focused on institutional-strengthening results, such as certifying green buildings, connecting buildings to an energy consumption monitoring platform, conducting energy audits, and creating an energy-use database. DSEE projects often do not articulate development outcomes beyond climate and energy benefits. Of 133 sampled active and closed DSEE proj- ects of the World Bank and IFC, only one-quarter target socioeconomic benefits such as gender inclusion, job creation, or improvement of health and well-being. For example, women are more deliberate and conscious of GHG emissions and energy use than men, according to Bank Group house- hold surveys, but they often are not targeted in project design to increase adoption. Similarly, retrofitting buildings or greening public infrastructure, among other DSEE activities, can create net new jobs, and lowering emis- sions and air pollution (some arising during combustion) improves respi- ratory and cardiovascular health. However, jobs and health outcomes are rarely included in DSEE projects. The World Bank used DPF to a limited extent to help develop an enabling policy and regulatory environment that promotes DSEE. Forty-three DPOs supported DSEE in 26 countries (FY11–20). Of these 43 DPOs, 28 (in 15 World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 countries) had prior actions that supported direct regulatory, planning, or market-oriented measures to promote public and private investments in en- ergy efficiency. These included approving energy efficiency laws and policies, setting energy efficiency standards, establishing building codes, regulating fuel quality, and mandating energy audits. Four DPOs (in Colombia, Jordan, Poland, and Ukraine) aimed to achieve market-oriented DSEE reforms, such as establishing and operationalizing an energy efficiency financing fund that could crowd in private sector capital and foster the development of energy service companies (ESCOs).2 The Poland DPO included support for white certificates, documents that certify that energy suppliers or distributors had reduced energy consumption. Only 6 DPOs in four countries out of 28 DPOs with direct DSEE policy reforms measured energy savings or GHG emissions reduction; these 6 DPOs fully or partially achieved all of the relevant targets. The World Bank DPOs had limited impact on DSEE adoption despite progress in energy sector reforms. Nearly one-quarter of the DPOs (14) pursued tariff reforms that included increasing tariffs for end users and reducing energy or fuel subsidies, aiming to create a commercially sustainable electricity sector. 20 The underlying rationale was that tariff charges for end users that fully reflect costs would reduce energy consumption and, thus, overall demand. Reductions in demand would, in turn, reduce GHG emissions and generate positive environmental benefits. DPOs in the Arab Republic of Egypt, Jordan, Panama, Rwanda, Serbia, Uzbekistan, and Vietnam pursued electricity tariff adjustment as part of electricity sector reforms. Contrary to expectations, the evidence collected for the evaluation does not show that the tariff reforms supported by DPOs increased end users’ DSEE adoption at scale. Although IFC advisory services were mostly effective, IFC investment ser- vices had limited effectiveness in achieving explicit energy savings and GHG emission reduction targets, partly because of overly ambitious development targets. IFC has increasingly targeted the primary DSEE development out- comes of energy savings or GHG emissions reduction. In FY16–21, 94 per- cent of IFC investment services projects targeted energy savings or GHG emissions reduction, a substantial increase from 55 percent in FY11–15 (figure 2.2), yet IFC had varied success in achieving energy savings or GHG emissions reduction. Relatively low ratings for investment services (37 per- cent of 13 evaluated projects) were largely due to project business underper- formance driven by exogenous factors unrelated to DSEE-specific activities, such as country or financial sector conditions. The underperformance was also due to project design shortcomings such as setting overambitious ob- jectives at entry and not meeting them. Many projects attempted to demon- Independent Evaluation Group World Bank Group    21 strate DSEE market creation (for example, targeting new end-user segments to improve GHG emissions reduction at scale) and system-level transforma- tion outcomes (such as reducing energy use across entire supply chains) that were not commensurate with the project design and scope of supporting a single firm or a single intermediary to promote DSEE. The evaluation team could not assess the effectiveness of MIGA guarantees. MIGA issued 11 guarantees for eight projects with DSEE measures. MIGA is supporting a fertilizer manufacturing firm in Bangladesh to increase DSEE adoption in its activities and a business and finance center in Djibouti. In Türkiye, MIGA is supporting the Turkish Ministry of Health in renovating five public hospital buildings as part of the country’s health transformation program. Some of MIGA’s DSEE projects do not have specific targets for the DSEE outcomes (for example, energy savings, GHG emissions, water use savings), but they are making progress in greening the public infrastructure, especially in the health care end-user market segment. The evaluation could not assess their effectiveness, however, because the eight projects were be- ing evaluated after this report was underway or had not yet met operational maturity to undergo evaluation at the time this report was underway.  hare of International Finance Corporation Investment Figure 2.2. S Services Projects Targeting Primary Demand-Side Energy Efficiency Outcomes 100 90 80 Share of projects (%) 70 60 50 40 30 20 10 World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 0 2011–15 2016–20 (n = 11) (n = 36) Fiscal year approval cohort Source: Independent Evaluation Group. Most Bank Group DSEE interventions did not scale up, solving only a frac- tion of client countries’ needs. Many World Bank and IFC interventions with DSEE components were part of larger energy programs that did not priori- tize energy efficiency gains as primary goals. Of interventions that targeted energy efficiency priorities, 66 percent (n = 96 of 146) covered single assets or groups of repeat-client small and medium enterprises without an ambi- tion to scale. Even effective DSEE interventions with the ambition to scale often could not do so. For example, the public stock retrofits under an effec- tive World Bank project in Armenia covered only approximately 2 percent of more than 5,800 public buildings. Another effective World Bank project in Türkiye targeting public buildings (schools and hospitals) committed $150 million to improve DSEE but could cover only 0.3 percent (500 out of 22 180,000 central government–owned public buildings) of the potential mar- ket segment. Most World Bank pilot interventions, even the effective ones, were not sustained or replicated beyond project close. Although the World Bank conducted several DSEE pilots across countries and introduced innovative components in some projects during the evaluation period, most such interventions have not scaled horizontally or vertically. Many innovative pilots were completed without any further client commitments or means for scaling successful interventions. For example, a pilot DSEE project in Argentina in 2012 set up an Energy Efficiency Fund that aimed to onlend to a pipeline of subprojects to sustain and grow the market for energy efficiency services and equipment. Although the project achieved its energy use savings and GHG emissions reduction goals, the innovative approach of the government coleading the fund to stimulate a strong market-led pipeline of DSEE did not materialize. An innovative pilot DSEE project in Benin provided grants (via the International Development Association and GEF) to finance energy-efficient appliances (such as compact fluorescent light bulb upgrades and improved stoves) for the commercial and residential market segments. The project was well coordinated with multiple development partners (for example, the European Investment Bank and the Agence Française de Développement [French Development Agency]) and was a rare and successful pilot in a low-income country (LIC) client. However, it did not Independent Evaluation Group World Bank Group    23 evolve into larger projects to develop DSEE when government commitment waned, and the lessons of experiences dissipated over time. The main reasons for lack of scale from innovative pilots across countries were volatile economic conditions, shifts in energy sector priorities toward more supply- side efforts, excess power supply, and unfavorable enabling environments. The Bank Group was successful at scaling DSEE in a few middle-income countries (MICs), mostly through a combination of instruments and co- financing, including from global programs. Trust funds and grants from the Clean Technology Fund (CTF), the Green Climate Fund (GCF), and the GEF, together with support from global programs (World Bank ESMAP and IFC GBMTP), helped address market, institutional, and information barri- ers, leading to vertical and horizontal scaling in some countries. India was able to scale DSEE both vertically and horizontally by developing a market for DSEE using support from the World Bank, IFC, ESMAP, CTF, and GEF (box 2.2). Similarly, Bank Group support allowed China and Türkiye to scale DSEE vertically. A few UMICs, including Colombia and South Africa, scaled DSEE horizontally. ESMAP also facilitated the integration of DSEE into other energy sector programs in some countries and across several sectors, creating the conditions for horizontal scaling. Examples include (i) energy efficiency and urban development, regeneration, and housing programs in Argentina, China, and Côte d’Ivoire and (ii) energy efficiency and urban re- silience in the Kyrgyz Republic.  orld Bank Group Support for Demand-Side Energy Efficiency Box 2.2. W Market Creation in India World Bank operations aimed to assist India’s super energy service company (ESCO) a in accessing private sector financing to scale energy savings in the residential and public market segments. A partial risk-sharing facility—the first for demand-side ener- gy efficiency (DSEE) financing globally—started with initial financing of US$12 million from the Global Environment Facility and used a Clean Technology Fund guarantee of World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 US$25 million to attract US$127 million of private financing. The partial risk-sharing fa- cility removed market barriers to commercial financing for DSEE by providing loan-de- fault risk guarantees to commercial banks that extended loans to firms adopting DSEE. An Energy Efficiency Scale-up Program (total financing US$1.43 billion) used the World Bank’s first-ever partial risk guarantee to crowd in private finance for an energy efficiency project (commercial US$200 million; development partners US$380 million). These programs used a bulk procurement model to create economies of scale, in- creasing supply and reducing retail prices for energy-efficient appliances and thereby creating a retail market for DSEE. The International Finance Corporation provided advisory support to help design public-private partnership contracts for ESCOs and prepare documents to support ESCOs’ applications for commercial loans. The ESCOs paid for street lighting upgrades, operations, and maintenance and earned part of the energy bill savings. The International Finance Corporation’s advisory support also addressed the municipal government’s insufficient creditworthiness, lack of data on street lighting, lack of payment security mechanisms, and difficulty in linking public sector payments to the performance of private partners. The International Finance Corporation’s support 24 (continued)  orld Bank Group Support for Demand-Side Energy Efficiency Box 2.2. W Market Creation in India (cont.) triggered market transformation by creating and piloting replicable models for street lighting investment and incentivizing the private sector to scale. The advisory support is expected to mobilize US$100 million in private investment. The Energy Sector Management Assistance Program, in parallel and via technical assistance, supported an in-depth market assessment for the design of ESCOs’ con- tract models, procurement models, and measurement and verification guidelines for energy savings. Source: Independent Evaluation Group. Note: a. A super ESCO is a government entity or public-private partnership established to serve as an intermediary between the government and independent ESCOs. Challenges to Scaling There are several challenges to scaling DSEE interventions. These challeng- es—including challenges to leveraging financing—consist of the following: (i) client preferences for support of energy supply and clients’ volatile pri- orities for DSEE, (ii) the Bank Group’s inability to articulate tangible DSEE benefits to clients, and (iii) the Bank Group’s insufficient leverage of global Independent Evaluation Group World Bank Group    25 programs (for example, ESMAP) and its convening power. Client demand, which is focused on energy supply, has limited potential for DSEE scale-up. The main reason is the lack of client demand, often driven by client priorities related to improving energy access and electricity generation. Although governments are generally able and willing to develop supply-side energy access, generation, and utility reforms, they see less value in DSEE reforms. Disincentives to addressing DSEE include excess power supply (Ghana, Indonesia), country’s own energy resources, and (at the consumer level) below-cost energy prices (Ghana, Indonesia). Over the next three to five years, most client countries are likely to prioritize energy sector access and renewable energy generation rather than energy efficiency–related reforms, such as addressing use subsidies and creating DSEE laws and regulations to provide incentives for end-user energy efficiency to meet growing energy demand and to achieve gross domestic product growth (IEA 2021a). Country demand for DSEE support is volatile. Several countries were affect- ed by fragility, conflict, or violence during the evaluation period, drying up DSEE opportunities because governments had other priorities. Similarly, most LICs have insufficient energy supply and prefer to address supply-side issues before DSEE. Countries that have recently graduated to LMIC status but were LICs during the evaluation period demonstrated strong demand for energy sector reforms, including energy efficiency policy reforms. How- ever, during the evaluation period, the need to address supply-side con- cerns (increasing generation, access, and distribution) limited demand from certain LMICs for Bank Group support on DSEE. For example, Ghana was an advanced DSEE reformer within Sub-Saharan Africa, but DSEE is no longer a government priority because of excess power supply (among other rea- sons). The Bank Group has been financing investments in transmission and distribution network improvements to reduce power loss, support the main distribution utility to improve financial and operational performance, and provide electricity access to rural areas. It also produced knowledge work World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 and technical advice supporting DSEE measures, but there were no plans to follow through on the related interventions (ESMAP 2019, 2022). The Bank Group’s inability to articulate tangible socioeconomic DSEE out- comes to clients limited DSEE scale-up. Bank Group interventions give uneven attention to multiple development benefits of DSEE beyond energy savings and GHG emissions and lack synergies between climate change and development impacts at the project level. There is increasing evidence that non-energy benefits play a crucial role in the decision to invest in energy efficiency. The Bank Group portfolio broadly recognizes that improvements in energy efficiency are among the most cost-effective responses to the increase in energy costs, the unpredictability and instability of energy sup- plies, and the growing demand for energy services. DSEE can enhance energy security and address supply-side crises, as reflected in the several efficient lighting deployment programs in Sub-Saharan Africa and the DSEE programs triggered by energy security concerns for Ukraine in 2014. DSEE’s potential to generate gains for society and the economy (for example, job creation 26 with retrofit projects, health improvements with reduced emissions, en- hanced profitability and product quality, and improved public budgets) and for decoupling growth from GHG emissions is underexplored in many coun- tries. However, DSEE development outcomes are frequently not measured: only a quarter of the DSEE portfolio attempted to measure a DSEE devel- opment benefit not related to energy savings or GHG emissions. The World Bank’s recent Country Climate and Development Reports—as part of de- veloping a road map for countries to transition to carbon neutrality—might help identify DSEE investments that could support clients with scaling up. Insufficient leverage of the existing global programs and the convener on energy (ESMAP) has also limited scaling. The World Bank’s ESMAP offers a unique platform to execute collective actions with development finance institutions, governments financing trust funds for development, and the private sector. However, the World Bank has not sufficiently leveraged it. ESMAP operates with a narrow focus on promoting World Bank lending, and on this it has been successful, including in facilitating the integration of DSEE into the energy sector programs of several countries. Yet, an ex- ternal evaluation of ESMAP concluded that its efforts on energy efficiency have been limited to supporting World Bank loans and that it has not yet developed a global reach and reputation (ICF International 2020). Despite its success, ESMAP lacks the mandate to support DSEE scaling globally because the priorities of specific donors and clients drive the program. ESMAP’s work is demand driven, but demand needs also to be shaped by ESMAP’s influence Independent Evaluation Group World Bank Group    27 on global thinking; by knowledge products and events; and by funding for new tools for data collection, analysis, and modeling. In contrast to its role in energy access and renewable energy generation, in DSEE, ESMAP is more a joiner than a leader. For example, to counteract the slowing rate of energy efficiency improvements, the Three Percent Club was launched under the Climate Action Summit’s Energy Transition Track in September 2019. It is a coalition of 15 businesses and institutions committed to energy efficiency through ambitious policy measures. The Three Percent Club comprises the European Bank for Reconstruction and Development, the International En- ergy Agency, GEF, Sustainable Energy for All Energy Efficiency Accelerators and Hub, and Energy Efficiency Global Alliance, among others. ESMAP, on behalf of the World Bank, has only recently joined the group. Factors of Success in Scaling The evaluation has identified positive lessons with strategic implications for scaling up DSEE. Successful scale-up was possible when (i) countries had robust policy environments for energy efficiency; (ii) clients received strong advisory and analytical work; (iii) the Bank Group targeted relevant clients, such as state-owned enterprises (SOEs); (iv) the interventions used de-risk- ing instruments; and (v) clients benefited from cumulative engagements with the Bank Group. Factor 1: Robust Policy Environment for Energy Efficiency A robust policy environment for energy efficiency in the client country was the common factor among all successful cases. Greater success in DSEE is observed in committed UMICs than in other countries. The main country- level incentives to address DSEE include fiscal pressure from energy- sector subsidization (Egypt, India, Morocco, Rwanda, Uzbekistan); cases when power demand is above supply (Egypt, Uzbekistan) or significant investments in new capacity are needed to cover projected steep demand World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 increases (India); cases when climate mitigation is at the top of the government’s agenda (India, Morocco); countries at high fiscal risk because of reliance on imported fuels (Egypt, Morocco, Rwanda); countries with power and gas tariffs reaching cost-recovery level (Morocco); and countries with significant economic inefficiencies due to high energy intensity (India, Egypt, Uzbekistan). In these countries, the Bank Group has a clear role in providing financing and technical advice and knowledge for DSEE activities, such as support to industrial DSEE; DSEE market development through financing funds and ESCOs; wide-scale leveraging of commercial financing for DSEE; promotion of municipal-level DSEE, such as green buildings and street lighting; and funding household-level DSEE, such as clean cooking. The largest borrowers for DSEE support from the World Bank were China, India, and Türkiye. Although the magnitude and impact of the scale-up as a result of the Bank Group DSEE approaches cannot be quantified precisely because of a lack of data, the overall energy efficiency savings in China were approximately 16 exajoules during 2010–18 (20 percent more than the total production of electrical energy in the United States in 2001). Similarly, the 28 overall energy efficiency savings in India were approximately 6 exajoules during the same period (about half of the yearly production of all nuclear power plants worldwide). Parallel Bank Group support to improve the policy environment and financ- ing of DSEE investments—rather than the traditional sequence of financing after policy reforms—can lead to successful scaling in LMICs. The Bank Group found opportunities to implement a nonsequential, opportunistic approach to DSEE reform that creates the policy, regulatory, and institutional environment for DSEE in parallel with the Bank Group’s financing. Fiscal and economic gains from energy savings create solid incentives for government buy-in and future reforms, as in Mexico and Uzbekistan. The Energy Efficiency Facility for Industrial Enterprises in Uzbekistan is a notable example of developing a sustainable financing mechanism for DSEE in a nonexistent DSEE market and an economy dominated by large and highly subsidized SOEs. The World Bank project financed a credit line to large commercial banks (two state-owned, one private) for onlending on concessional terms to large industrial SOEs for equipment modernization. The World Bank scaled the credit line through additional financing based on successful investments and repayments from subprojects. The government formally recognized DSEE credit lines as a crit- ical mechanism for scaling industrial DSEE investments. Other development partners (such as the Asian Development Bank) started to invest in this area, involving additional commercial banks in Uzbekistan. Factor 2: Advisory Services and Analytical Work Independent Evaluation Group World Bank Group    29 Knowledge work and country-specific technical assistance often guide suc- cessful investments in DSEE. Bank Group advisory and analytical support enhanced the policy environment to facilitate successful DSEE scale-up in industrialized countries (China, India, Türkiye, Vietnam). Data availability and developing data platforms proved beneficial for the Bank Group client govern- ments’ decision-making and leveraging of climate finance (Ghana, Indonesia). ESMAP, a global knowledge convener on energy, drove the World Bank’s ASA projects for DSEE measures, accelerating investments in DSEE. ESMAP succeeded in broadening the geographical reach of energy efficiency and piloting energy efficiency measures in sectors outside energy. This includes analytical work to identify energy-efficient investments in transport, water supply, and lighting that helped either lead to new lending operations (as in Uzbekistan) or inform operations (as in Argentina, Pakistan, and Zanzi- bar; ICF International 2020). ESMAP initiated support for policy reforms on urban development, building regeneration, and housing programs in Côte d’Ivoire in parallel to opportunistic investments from IFC and its cofinancing partners in low-income housing development, which led to the development of housing finance platforms and crowded in commercial investors to DSEE. ESMAP’s functions in funding, knowledge, tools, expertise, hands-on adviso- ry support, and operational engagement have accelerated DSEE investments. IFC, through GBMTP, demonstrated scale-up of client commitments and early development outcomes in green buildings across several Regions. GBMTP combined primary outcomes (energy savings and GHG emissions reduction) with outcomes measuring water resource efficiency and mate- rial use efficiency. It demonstrated benefits to firms, leading to increased firm-level capital commitments to energy efficiency across several Regions (figure 2.3). GBMTP uses a holistic approach that combines advisory support, lending, and the underlying Excellence in Design for Greater Efficiencies World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 (EDGE) certification process. In Colombia, GBMTP crowded in private sector participation through the Colombian Chamber of Construction and promot- ed green building codes. Furthermore, it supported capacity building and awareness raising on green buildings for an extensive network of Colombian architects and engineers, staff working in Colombian financial institutions, and other practitioners. GBMTP enabled IFC to partner with Bancolombia and Davivienda to issue green bonds and invest in domestic green bond issu- ances. The green bonds also promoted EDGE because new project teams that wanted to qualify for green construction financing through the green bonds were required to obtain a green building certification such as Leadership in Energy and Environmental Design (LEED) or EDGE. Similarly, in Indonesia, GBMTP provided a tool kit and training that conveyed essential guidance on the mandatory green building codes to Indonesia’s Ministry of Public Works and Housing for the ministry to develop its Green Building Toolkit for the country’s 33 provinces and 98 cities. The guidance followed successful mod- els in Jakarta and Bandung. The tool kit and training accelerated the creation of a green building market. Financial institutions then partnered with IFC 30 to develop green mortgages and construction loans. In South Africa, EDGE complemented Green Building Council South Africa in providing training to enhance awareness of green finance and the benefits of green buildings. IFC invested in a fund managed by International Housing Solutions and helped build technical capacity for applying for the EDGE certification.  emand-Side Energy Efficiency Scale-up and Development Figure 2.3. D Benefits through IFC’s GBMTP-EDGE Certifications by Region Independent Evaluation Group World Bank Group    31 World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 Sources: Independent Evaluation Group; International Finance Corporation. Note: CO2 = carbon dioxide; EAP = East Asia and Pacific; ECA = Europe and Central Asia; EDGE = Excellence in Design for Greater Efficiencies; GBMTP = Green Buildings Market Transformation Program; GJ = gigajoule; IFC = International Finance Corporation; LAC = Latin America and the Caribbean; m3 = cubic meter; MENA = Middle East and North Africa; MWh = megawatt hour; SAR = South Asia; SSA = Sub-Saharan Africa; tCO2 = metric tons of carbon dioxide. 32 Factor 3: Targeting Relevant Clients Economic gains from energy savings create solid incentives for government buy-in. The Bank Group can achieve the most significant gains in a short time from DSEE investments in large industries, including in SOEs, which play a vital role in DSEE scale-up. SOEs remain central actors in the energy sectors of client countries and adjacent sectors that are critical for contri- butions to Paris Agreement alignment and SDG 7.3 SOEs are responsible for nearly 40 percent of global power investment (IEA 2020d), and direct emissions from SOEs make up at least 16 percent of total global GHG emis- sions (Clark and Benoit 2022). The importance of SOEs is about not only GHG emissions reduction but also energy use savings they can potentially undertake across their portfolios. Targeting hard-to-abate sectors such as manufacturing and heavy industries using well-funded SOEs can be an op- portunistic approach and promote DSEE scaling. State-owned development banks play an essential role in DSEE scale-up. State-owned development banks or national development banks typically aim to finance state priorities, encourage structural transformation, and pro- mote environmental sustainability, and in many client countries, they have taken on the role of advancing climate priorities. In the context of DSEE scale-up, state-owned development banks played critical roles as advocates, cofinanciers, direct clients, and sometimes project implementation units. For Independent Evaluation Group World Bank Group    33 example, the 2010–19 Financing Energy Efficiency at MSMEs (micro, small, and medium enterprises) Project in India aimed to increase demand among MSMEs for investments in energy efficiency. Among other activities, the project supported training service providers to demonstrate DSEE benefits to MSMEs and install DSEE equipment. The project estimated that its activ- ities reduced emissions by the equivalent of 16 million tons of CO2. In 2018, the state-owned Small Industries Development Bank of India, a local imple- menting partner, established a Green Climate and Sustainable Development Initiatives business line to promote energy efficiency and cleaner produc- tion by MSMEs (an indication of a robust policy environment). The GCF, a cofinancing partner, has accredited the Small Industries Development Bank of India business line as a national implementing entity to finance climate change projects in developing countries. Based on these achievements, the Small Industries Development Bank of India is creating a large new initiative to provide financing for multi-asset and multisectoral approaches across 5,000 MSMEs, an example of both vertical and horizontal scaling. Factor 4: Financial De-Risking Mechanisms and Instruments Delivery of concessional finance, grants, or guarantees can effectively demonstrate the commercial potential of DSEE financing to attract private sector investment in DSEE, creating a DSEE market. Demonstrating the commercial potential of DSEE financing adopts the model of “moving from a lending bank to a leveraging bank.” Accompanying interventions with a range of technical assistance products to leverage technical solutions, business models (including ESCO contract models and public-private part- nership models for street lighting), utility performance models, and various capacity-building activities was a vital factor for success. De-risking mechanisms and guarantee instruments were success factors in DSEE scale-ups. The lack of empirically sound, statistical data on pay- World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 ment default rates and the actual energy and cost savings achieved by DSEE investment projects causes financial institutions to assign high-risk pre- miums to DSEE investments. In this context, de-risking mechanisms (such as risk-sharing facilities and partial risk guarantee instruments) can en- courage onlending through local financial institutions, helping scale DSEE. For example, the Vietnam Scaling Up Energy Efficiency Project provided $75 million to fund a risk-sharing facility. The facility aimed to issue partial credit risk guarantees for energy efficiency loans financed by participating financial institutions to end beneficiaries (industries and ESCOs). By reduc- ing investment risk, the facility mobilized commercial financing for energy efficiency investments, a major contribution to the sustainability of energy efficiency programs in Vietnam. Factor 5: Cumulative Demand-Side Energy Efficiency Engagements Cumulative DSEE engagements in a country are a necessary condition for scaling. Cumulative engagements are multiple interventions over time that 34 build on one another, whether planned as a sequence or not. Cumulative engagements were limited mostly to UMICs during the evaluation period, both for the World Bank (figure 2.4, panel a) and for IFC (figure 2.4, panel b). Fewer than 20 countries have cumulative engagements on DSEE even when the original project outcome is successful, and horizontal scaling in LICs was very limited. Together, these two facts suggest that the Bank Group has the potential to scale DSEE interventions in UMICs and industrialized countries but has difficulty scaling them in LICs because of clients’ energy sector pri- orities and limited appetite.  ountries with Cumulative Demand-Side Energy Efficiency Figure 2.4. C Engagements a. World Bank b. International Finance Corporation China India Türkiye China Türkiye Uzbekistan Russian Vietnam Federation Brazil India Mexico Mexico Ukraine Burkina Faso Tajikistan Côte d'Ivoire Argentina Colombia Bangladesh Independent Evaluation Group World Bank Group    35 Morocco Vietnam 0 2 4 6 8 10 12 0 3 6 9 12 15 Cumulative engagements (no.) Cumulative engagements (no.) IPF DPF P4R IFC-IS Source: Independent Evaluation Group. Note: DPF = development policy financing; IPF = investment project financing; IS = investment services; P4R = Program-for-Results. 1  The evaluation distinguishes between development policy financing, which is an instrument, and development policy operations. Similarly, it distinguishes between investment project financing and investment policy financing projects. 2  Energy service companies (ESCOs) develop, design, build, and arrange financing for projects that save energy, reduce energy costs, and decrease operations and maintenance costs at their customers’ facilities. In general, ESCOs act as project developers for a comprehensive range of energy conservation measures and assume the technical and performance risks associated with a project. ESCOs are distinguished from other types of service firms that offer energy efficiency improvements for buildings in that they use the performance-based contracting methodology. When an ESCO implements a project, its compensation is directly linked to the actual energy cost savings. 3  The World Bank Group is committed to aligning its financing flows with the objectives of the Paris Agreement. The Bank Group defines alignment as providing support to clients that is consistent with pathways toward low-carbon and climate-resilient development. The World Bank plans to align all new operations by July 1, 2023, the start of fiscal year 2024. The International Finance Corporation and the Multilateral Investment Guarantee Agency plan to align 85 percent of Board of Executive Directors–approved real sector operations starting World Bank Group Support to Demand-Side Energy Efficiency  Chapter 2 July 1, 2023, and 100 percent two years later, starting July 1, 2025. To achieve this goal, both institutions are expected to begin aligning 100 percent of their projects at the concept stage well before July 1, 2023. 36 3 | Coherence of World Bank Group Interventions to Support Scale-Up Highlights “Coherence”—the extent to which the World Bank Group supports complementary approaches to scaling demand-side energy effi- ciency (DSEE)—can be assessed internally (among the World Bank, the International Finance Corporation [IFC], and the Multilateral Investment Guarantee Agency) and externally (between the Bank Group and other actors). World Bank DSEE approaches have been internally coherent in supporting vertical but not horizontal scale-up. The World Bank has been externally coherent in onlending with cofinanciers but less coherent on advisory support to clients and knowledge diffusion goals with client governments and partners. The external coher- ence of the World Bank Energy and Extractives Global Practice and Energy Sector Management Assistance Program in Vietnam is a best practice. IFC DSEE approaches have been internally coherent for not only vertical but also horizontal scaling. IFC has supported horizontal scaling through blended finance programs and multicountry and multisector approaches. IFC has also been externally coherent in using its Excellence in Design for Greater Efficiencies certification to advance DSEE priorities with its client groups in the buildings sector and in supporting client governments that aim to address climate change and energy shortages. The unwinding of the World Bank’s DSEE community of practice has limited collaboration on DSEE across sectors. The IFC Climate Business team has been critical to enabling IFC’s coherence across sectors in DSEE approaches. 37  Multilateral Investment Guarantee Agency DSEE approaches have been internally coherent. Its DSEE clients (from industrialized countries) have been well positioned to embrace either the IFC Excellence in Design for Greater Efficiencies standard and related tool kits or the US Green Building Council Leadership in Energy and Environmental Design standard. The Multilateral Investment Guarantee Agency has been externally coherent, partnering with the European Bank for Reconstruction and Development to pro- vide political risk insurance for client priorities such as health care buildings in Türkiye and adopting external green building standards. 38   Coherence refers to the extent to which Bank Group DSEE interventions support or undermine each other and the interventions of Bank Group partners in reaching SDG 13, SDG 7, and Paris Agreement alignment.1 Achieving the desired long-term outcomes of DSEE—reducing energy use and GHG emissions—requires scaling. Scaling, in turn, requires coherence because incoherent approaches cannot, by themselves, achieve the scale needed for Paris Agreement alignment and meeting SDG 13 and SDG 7 tar- gets. Only by working consistently in alignment internally and with external partners can the Bank Group hope to reach the scale that is necessary to have an impact. Coherence has several aspects. Coherence can be internal (within the Bank Group) or external (between the Bank Group and other actors). In this report, internal coherence refers to (i) the complementarity of Bank Group support for addressing clients’ SSEE needs (such as utility upgrades and reforms) and support for addressing DSEE needs and (ii) the complementarity of interven- tions across sectors (including the complementarity of World Bank inter- ventions across the GPs and of IFC interventions across industry groups). External coherence refers to (i) the complementarity of Bank Group inter- ventions with those of client governments and partner development institu- tions and (ii) the alignment of Bank Group interventions with international standards. Evidence of coherence includes coordination, joint initiatives, cofinancing, consistency, avoidance of duplication, synergies among inter- Independent Evaluation Group World Bank Group    39 ventions, and alignment with international standards. All aspects of coherence are important, but one stands out as particular- ly important: internal coherence across sectors, meaning that the Bank Group aligns its DSEE approaches in different sectors. Internal coherence across sectors is fundamental because horizontal scaling implies, by defi- nition, scaling across multiple sectors. Therefore, coherence across sec- tors contributes directly to horizontal scaling (dashed arrow in figure 3.1). Horizontal scaling is essential for reducing energy use (SDG 7) and GHG emissions (SDG 13 and Paris Agreement alignment) at scale, yet it has been elusive in the portfolio. Figure 3.1. The Relationship among Coherence, Scaling, and Outcomes Coherence Internal coherence External coherence Coherence between Coherence with partners SSEE and DSEE and clients Coherence Coherence with across sectors global standards Scaling Horizontal scaling Vertical scaling World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 Outcomes Saving energy Reducing GHG emissions Other development outcomes Source: Independent Evaluation Group. Note: DSEE = demand-side energy efficiency; GHG = greenhouse gas; SSEE = supply-side energy effi- ciency. 40 Internal Coherence Coherence between Supply-Side and Demand-Side Energy Efficiency A minority of World Bank energy efficiency interventions had coherent SSEE and DSEE elements. Of a sample of 231 World Bank lending projects with SSEE, 25 percent (58 projects) had DSEE elements. For example, the $72 million credit to Malawi via the Energy Sector Support Project (P099626; 2015–18) aimed to increase the reliability and quality of electricity supply in the country and supported complementary SSEE and DSEE interventions. The SSEE interventions included rehabilitating underground transmission cables to reduce the amount of electricity lost because of inadequate insu- lation. The DSEE interventions (3 percent of the total project cost) included installing automated meter readers that allowed large commercial customers to better monitor and manage their energy consumption while allowing the utility a real-time (remote) monitoring of meter performance on customer sites to better detect losses due to theft. The cable rehabilitation, automat- ed meter readers, and other activities that the project supported reduced electricity losses per year from 25 percent to 17 percent. Reducing losses increases energy efficiency because overcoming losses requires generating excess energy to meet end-user demand. (See Naeher, Narayanan, and Ziulu 2021 for more information about this project.) The World Bank has taken various approaches to coherently addressing Independent Evaluation Group World Bank Group    41 SSEE and DSEE needs. In addition to the approaches taken in the Malawi project, the World Bank has supported coherent supply- and demand-side interventions using smart metering (for example, in Burkina Faso and Poland), time-of-use pricing, and energy demand–based pricing for large consumers (for example, in Rwanda and Vietnam).2 These mechanisms are both supply-side and demand-side interventions because they provide information to utilities (supply) and consumers (demand) to maintain supply-demand equilibrium. Doing so promotes both system reliability and energy efficiency. For example, time-of-use pricing is a demand-response technique that adjusts the price of electricity based on when it is used. During hours of typically high demand, the cost of using electricity is higher, reflecting the cost of supplying electricity at that time. The price signal reduces electricity demand when supply is tight. The practice benefits both suppliers and consumers (for example, by improving system reliability and thus avoiding blackouts) in addition to increasing energy efficiency (Eid et al. 2016). Although limited in number (four projects), these interventions have increased access to and reliability of electricity by reducing peak load while simultaneously improving energy conservation by inducing people and firms to use less energy.3 Coherence across Sectors World Bank DSEE interventions had limited coherence across the energy, water, agriculture, and transport sectors. In principle, many World Bank GPs, not only the Energy and Extractives GP, could address DSEE. However, in practice, GPs outside of Energy and Extractives have made limited con- tributions to DSEE. For example, the Transport GP was responsible for only 6 percent of total World Bank DSEE commitments ($0.8 billion in 4 projects over 10 years) and the Water GP for only 5 percent. In the water sector, a small share of energy efficiency interventions (18 projects with $500 million World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 in lending commitments over 10 years) included upgrading and modernizing water treatment plants and pumping stations to help reduce energy costs and contribute to GHG emission reduction. DSEE requires coherence in design, measures, and actions between the energy sector and other sectors to achieve parallel reforms and manage spillover effects. The Jordan First and Second Programmatic Energy and Water Sector Reforms development policy loans (2015–18) are a good ex- ample of a programmatic intervention designed to coherently address the water-energy nexus for horizontal scale-up. Jordan is among the countries most affected by water scarcity, and its energy and water sectors are highly interdependent. Energy use for water pumping and treatment accounts for half of the costs in the water sector, which uses 15 percent of Jordan’s total electricity generation and is the largest energy consumer. Consequently, policy and operational changes in one sector directly affect the other, re- quiring careful coordination among them. As the government aimed at cost recovery through tariff increases in both the energy and water sectors, the 42 DPOs supported the government of Jordan in reducing inefficiencies in the water sector, particularly those related to energy use. The projects involved a combination of energy cost-reduction measures, including through energy efficiency in pumping operations, the improvement of water supply systems to reduce physical losses and energy input, and the use of renewable energy. This helped increase energy savings (achieved 84 gigawatt hours versus the target of 50 gigawatt hours) in the water sector by 2017, beyond the targets set by the Ministry of Water and Irrigation (World Bank 2019b). However, steep increases in electricity tariffs by the National Electric Power Company and charged to the Water Authority of Jordan since 2018 outpaced any ener- gy efficiency gains achieved earlier without corresponding increases in water tariffs by the Water Authority of Jordan, diminishing the effectiveness of the parallel reforms. Furthermore, a lack of continuous World Bank engagement and cumulative investments limited spillover effects from the energy sector to other sectors, including water. The water sector in Jordan still struggles with the heavy burden of an average 24 percent increase in electricity tariffs since 2013 and from the additional pressure of hosting 1.3 million Syrian refugees (IMF 2022). Transport interventions especially need to aim at increasing energy efficiency. The transport sector was responsible for approximately 27 percent of total en- ergy-related GHG emissions in 2019 (IEA 2021b), and there are enormous un- exploited mitigation potentials from energy efficiency in transport. However, the Transport GP did little on energy or fuel efficiency during the evaluation Independent Evaluation Group World Bank Group    43 period. Measures to improve vehicles, fuels, and energy efficiency in trans- port facilities constituted only a fraction of the transport sector interventions. Moreover, typical interventions (such as the modernization of bus fleets) did not measure fuel efficiency. World Bank transport projects also did not explore digital services that could contribute to transport decarbonization (such as in- telligent transport management that includes, for example, route optimization and shipping programming; Dominioni and Englert 2022). Designing transport projects to demonstrate energy efficiency and climate co-benefits would help scale DSEE, including by mobilizing green finance and demonstrating climate abatement opportunities to developers. The World Bank’s DSEE community of practice and DSEE solutions group have been dormant in recent years, limiting collaboration on DSEE across sectors. During the first half of the evaluation period, the World Bank’s DSEE community of practice—which included representatives of the Energy and Extractives GP and the Climate Change Global Solutions Group of the Sustainable Development Practice Group—coordinated efforts with ESMAP. Together, they created a coherent blended DSEE approach, combining GCF commitments and World Bank lending for a new sustainable cooling initiative that cut across the energy and transport sectors. (It supported, for example, improvements in the efficiency of refrigeration services in both buildings and trucks.) Such forward-looking initiatives led to identifying cooling-related projects (such as cold chain logistics and warehousing)4 in Bangladesh, North Macedonia, Panama, and Sri Lanka. The approaches also led to innovative analytical work, such as The Cold Road to Paris (World Bank 2021a), needs assessment in food safety, and health sector interventions that require cooling solutions for food and vaccine transport. However, because the community of practice has been dormant in recent years, the World Bank has lacked an alternative centralized group or mechanism to facilitate the coherence of DSEE interventions across sectors. The inactivity of the community of practice and a limited focus on emerging DSEE issues have constrained cross–Practice Group collaboration and integrated approaches World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 to DSEE projects. IFC’s approach to DSEE has been coherent across diverse sectors (industry groups and business lines). IFC has mainstreamed DSEE investment projects across three industry groups: the Financial Institutions Group; Infrastruc- ture and Natural Resources; and Manufacturing, Agribusiness, and Services (figure 3.2, panel a). Similarly, IFC advisory services mainstreamed DSEE across all its business lines (figure 3.2, panel b). During the second half of the evaluation period, Manufacturing, Agribusiness, and Services took the lead on DSEE in IFC investment services ($1.9 billion for FY16–20), driven by increased demand for green buildings. In the recent two-year period, Infra- structure and Natural Resources has been building its pipeline on DSEE by targeting publicly owned and private airports and developing opportunities for green public infrastructure. IFC’s Partnership for Cleaner Textile program is a good example of a coherent water and energy efficiency approach. Part- nership for Cleaner Textile is a horizontally scalable program that supports the entire textile value chain—spinning, weaving, wet processing, and gar- ment production—in adopting cleaner production practices.5 The program 44 engages with technology suppliers, industrial associations, financial institu- tions, and governments to bring about systemic and positive environmental change for the Bangladesh textile sector and contribute to the sector’s long- term competitiveness and environmental sustainability. nternational Finance Corporation Demand-Side Energy Figure 3.2. I Efficiency Support a. International Finance Corporation investment services demand-side energy efficiency commitments by industry group 2.0 Commitment amount� 1.5 (US$, billions) 1.0 0.5 0.0 CDF FIG INR MAS Industry group FY11–15 FY16–20 b. International Finance Corporation advisory services demand-side energy efficiency funds by primary business line Economics and Private Sector Development Transaction advisory Access to finance IFC primary business line Cross-cutting advisory solutions Independent Evaluation Group World Bank Group    45 Disruptive Technologies and Funds Equitable Growth, Finance, and Institutions Financial Institutions Group Infrastructure Manufacturing, Agribusiness, and Services Sustainable business advisory Trade and Competitiveness 0 10 20 30 40 50 60 Total funds managed by IFC (US$, millions) FY11–15 FY16–20 Source: Independent Evaluation Group. Note: CDF = Disruptive Technologies and Funds; FIG = Financial Institutions Group; FY = fiscal year; IFC = International Finance Corporation; INR = Infrastructure and Natural Resources; MAS = Manufacturing, Agribusiness, and Services. IFC has used blended finance to improve coherence on DSEE across sectors. Combining concessional loans with commercial loans for DSEE addresses several priorities and barriers at once: targeting critical economic actors operating in different sectors (for example, small and medium enterprises); providing financial instruments that are popular in one or more sectors (for example, leasing for small and medium enterprises operating in the ener- gy, agriculture, and service sectors); removing barriers for domestic DSEE adoption (for example, by providing long-term financing in local financial markets where it is absent or limited); and targeting energy-intensive client firms (such as big importers of fuels). IFC’s successful blended finance model for DSEE in Türkiye is an example (box 3.1).  lended Finance to Support Demand-Side Energy Efficiency Box 3.1. B across Sectors The International Finance Corporation (IFC) Commercializing Sustainable Energy Finance program in Türkiye coherently focused on multiple energy-intensive sectors, such as textile manufacturing and metal production, by increasing demand-side ener- World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 gy efficiency financing through local commercial banks. IFC financed the program with roughly US$21 million from the Clean Technology Fund (a World Bank–administered financial intermediary fund) blended with almost US$100 million of IFC’s own funds. IFC developed a leasing model for demand-side energy efficiency (figure B.3.1.1), which was nonexistent in Türkiye. IFC provided blended concessional loans to three Turkish leasing companies. The leasing companies, in turn, provided leases to firms operating in energy-intensive industries that wanted to finance demand-side ener- gy efficiency projects and industrial equipment that met specific energy efficiency parameters (for example, reducing energy consumption by at least 15 percent). Once a customer decided to acquire energy-efficient equipment, one of the leasing com- panies would purchase the equipment using Commercializing Sustainable Energy Finance funds and provide the equipment through a lease to the customer. In 2015, IFC reduced its concessions to local banks and began lending on commercial terms because the market had transformed. The use of blended finance enabled a leasing model that allowed companies in several industries to improve energy efficiency, planting the seed for horizontal scale-up. The ensuing market transformation (local banks lending on commercial terms) reinforced the potential for scale-up. 46 (continued)  lended Finance to Support Demand-Side Energy Efficiency Box 3.1. B across Sectors (cont.) Figure B.3.1.1 International Finance Corporation Blended Finance Approach to Demand-Side Energy Efficiency End-use customers Firms in energy-intensive industries International Iron and steel Finance Corporation Cement Equipment Loan Leasing leases company Chemicals and Climate Purchases equipment petrochemicals Investment once end-use customer Funds agrees to lease Aluminum Textiles Sources: Clean Technology Fund, https://www.climateinvestmentfunds.org/topics/clean-technolo- gies; Independent Evaluation Group. Some of IFC’s recent DSEE approaches specifically aim for coherence across sectors. Over the past three years, IFC’s DSEE approaches have evolved be- yond targeting the energy efficiency of individual firms’ buildings to coher- Independent Evaluation Group World Bank Group    47 ently targeting energy efficiency across supply chains of energy-intensive industries. For example, EDGE certified 49 stores of a retail food chain in an Eastern European country.6 IFC and the chain’s parent company have since discussed greening all of its more than 100 stores and other assets global- ly, including the logistics fleet. Doing so would involve greening cooling solutions in cold supply chains (for meat, for example) and storing goods in warehouses that use green building materials, minimizing heating and cooling needs, and taking advantage of natural lighting. Because this ap- proach incorporates the transport and logistics sectors in a retail chain (or backward linkages),7 it is a promising example of coherence across sectors, even though it is too early to assess whether it will be effective. Another example of coherence across sectors is IFC’s support to an Argentina-based firm in the limestone industry. The IFC project aimed to implement energy efficiency measures to reduce the GHG emissions and embodied carbon of the limestone firm and firms in related industries. The project included sup- port for the client to transfer its know-how and technology about improving energy and water efficiency to firms in other industries—such as mining and steel production—that could contribute to reducing the GHG emissions of the client’s buildings. The IFC Climate Business team demonstrates that a centralized unit that supports coordination across sectors is critical for the coherence of DSEE interventions. IFC approached DSEE using a climate lens during the evaluation period, channeling most of its DSEE investments through the IFC Climate Business team as the central coordinating unit across IFC industry groups. The industry groups originate deals, and the Climate Business team, together with industry specialists and advisory teams, provides the groups with knowledge, technical, and industry specialist support. The Climate Business team also provides the industry groups with input on clients’ needs and collects feedback on global DSEE standards from industry actors and Bank Group colleagues. The Disruptive Technologies and Funds industry group initiated the IFC TechEmerge program in close collaboration with World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 the Climate Business team and industry teams to provide concessional and grant funding to pilot projects across sectors. Examples include pilots for temperature-controlled logistics firms (the transport sector), cooling technologies in supermarkets (the retail sector), and resource efficiency and cooling solutions for hotels (the hospitality sector). MIGA applies a coherent approach to green building standards in its proj- ects. MIGA DSEE clients (from industrialized countries) have been well po- sitioned to embrace either the LEED or EDGE global standards for buildings, the latter of which is a standard advocated by IFC. In the hospitality-cluster project, where IFC and MIGA are partnering across some of the subprojects (individual hotels), MIGA and IFC are jointly supporting the client’s adop- tion of the EDGE certification standard. 48 External Coherence Coherence with Partners and Clients MDBs as a group do not have a coherent approach to DSEE. MDBs, including the Bank Group, are incoherent in linking DSEE interventions to the primary DSEE development outcomes of saving energy and reducing GHG emissions. In addition, most MDBs do not have a unified approach to advancing DSEE standards. For example, the industrial and building segments have multiple national and international green building standards around the world in ad- dition to EDGE: the US Green Building Council’s LEED, the UK government’s Building Research Establishment Environmental Assessment Method, and national standards, such as Building Energy Performance—Türkiye. The MDBs (including the Bank Group) do not have coherent approaches to advancing DSEE standards and deploy different standards for different interventions. MDBs, including the Bank Group, are also incoherent in linking DSEE ap- proaches to more comprehensive socioeconomic benefits and communi- cating the benefits of DSEE to stakeholders. Table 3.1 shows how the major MDBs approach central aspects of DSEE. Most consider DSEE a key priori- ty. Some have cross-sectoral approaches and results frameworks for DSEE interventions. However, none track the socioeconomic benefits of DSEE interventions (related to health or gender, for example) or have a clear Independent Evaluation Group World Bank Group    49 communication strategy to catalyze end-user adoption of DSEE at scale. In this incoherent environment, it is impossible for the Bank Group to consis- tently support interventions that complement those of partner development institutions and other multilateral actors. Yet, the Bank Group has missed opportunities to exhibit leadership in coordinating with other MDBs to in- crease external coherence in DSEE by, for example, setting green standards for public infrastructure, establishing precedents for using SOEs as project implementation units, improving links to DSEE co-benefits, and communi- cating benefits clearly to end users. The recently prepared joint MDB as- sessment framework for Paris Alignment (2022), which aims at collaborative work toward low-emission and climate-resilient development, is a step in the right direction.  ultilateral Organizations’ Approaches to Demand-Side Table 3.1. M Energy Efficiency Criterion ADB AfDB AIIB EBRD IDB IsDB World Bank Group DSEE key Y Y Y Y Y — Y priority Results frame- Y Y Y — — Y Y work for DSEE interventions Socioeconom- — — — — — — — ic links Cross-sectoral Y Y Y Y Y — — approaches Communica- — — — — — — — tion strategy on DSEE Source: Independent Evaluation Group. Note: Analysis is based on desk reviews of annual reports, MDB energy sector strategies, select country program documents of MDBs, and key informant interviews. — = no; ADB = Asian Development Bank; AfDB = African Development Bank; AIIB = Asian Infrastructure Investment Bank; DSEE = demand-side energy efficiency; EBRD = European Bank for Reconstruction and Development; IDB = Inter-American Development Bank; IsDB = Islamic Development Bank; MDB = multilateral development bank; Y = yes. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 The Bank Group has been externally coherent with cofinanciers. The Bank Group has crowded in various funding sources (including MDBs, other de- velopment finance institutions,8 and donors) for DSEE lending, illustrating its coherence with these cofinanciers (figure 3.3). The amount of trust fund financing supporting World Bank lending for DSEE grew in the second half of the evaluation period through cofinancing arrangements with the CTF, the Global Infrastructure Facility, the GCF, and the ESMAP. However, the World Bank has been less coherent on advisory support to clients and knowledge diffusion goals with clients and partners. World Bank ASA and ESMAP support (which focus on World Bank lending) have limited coherence related to crowding in global knowledge (for example, on Paris Agreement alignment needs or Net Zero Consortium efforts) and limited alignment with green standards and global consortiums. Recent World Bank research on green maritime transport (Englert et al. 2021) is promising.9 50  emand-Side Energy Efficiency Cofinanciers by Commitment Figure 3.3. D for Investment Project Finance, 2011–20 Foreign multilateral institutions (unidentified) Germany: Kreditanstalt für Wiederaufbau European Investment Bank Cofinanciers (ranked) Asian Infrastructure Investment Bank Asian Development Bank OPEC Fund European Development Fund United States Agency for International Development 0 50 100 150 200 250 300 350 400 Cofinancing amount (US$, millions) Source: Independent Evaluation Group. Note: OPEC = Organization of the Petroleum Exporting Countries. External coherence is important for greening public infrastructure at scale. Independent Evaluation Group World Bank Group    51 The World Bank is ideally placed to decarbonize public infrastructure (in- cluding public buildings) as part of pursuing the two primary DSEE out- comes (GHG emissions reduction and energy-use savings). However, it can only do so if its actions are externally coherent. Reducing energy use in publicly owned buildings (such as schools and hospitals) requires building external coherence with cofinanciers (such as GEF and CTF) and eventually local financial institutions. A World Bank project in Armenia illustrates the point: it partially achieved the greening of public infrastructure, but it could have achieved more by improving external coherence with other partners (box 3.2).  he World Bank’s Support for Demand-Side Energy Efficiency Box 3.2. T in Armenia The World Bank Energy Efficiency Project in Armenia (2011) financed energy efficiency up- grades to eligible public and social buildings to reduce energy consumption. A World Bank– administered Global Environment Facility grant of US$1.82 million financed the project to demonstrate a replicable and sustainable model for energy efficiency investments in Arme- nia’s public sector with cofinancing from the implementing agency, the Renewable Resourc- es and Energy Efficiency Fund. This fund was set up under a previous World Bank project (2005) as a nonprofit organization to promote renewable energy and energy efficiency. In this project, the fund acted as the implementation agency and channeled the World Bank and Global Environment Facility lending to demand-side energy efficiency (DSEE) using energy performance contracts. This approach of using a state-operated fund as an implementation agency to cofinance and deliver DSEE aimed to increase awareness of the economic bene- fits of energy efficiency and stimulate demand for energy efficiency financing. The 2011 DSEE project financed insulation of walls, basements, and attics; repair or replacement of external doors and windows; window optimization; reflective surfacing of walls behind radiators; improvements or replacement of boilers and heating sys- World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 tems; replacement of mercury vapor lamps with high-pressure sodium vapor lamps (or LEDs); and replacement of incandescent bulbs with compact fluorescent lamps. Investments in schools and universities, hospitals and medical centers, penitentiaries, street lighting, and theaters resulted in lifetime energy savings for the end users of about 540.2 gigawatt hours and a greenhouse gas emissions reduction of approxi- mately 145 metric kilotons of carbon dioxide, exceeding the project’s targets. Yet the project addressed only a small fraction of the country-level needs in greening the public sector in Armenia; the 124 public buildings upgraded through the project amount to only about 2 percent of more than 5,800 public buildings. Moreover, there were no spillover effects or continuity beyond the initial results. There was no concert- ed effort to build a more comprehensive network of cofinanciers, such as domestic financial institutions or development finance institutions willing to take on commercial risk or partner with donor organizations. Doing so would have required greater external coherence from the World Bank, working with development partners and the private sector to engage with the government of Armenia, identify the right project implemen- tation unit, and catalyze DSEE investments at scale. 52 Source: World Bank 2019a. The external coherence of the World Bank Energy and Extractives GP and ESMAP in Vietnam can be a model for interventions in other industrializing countries. The World Bank’s ESMAP offers a unique pathway for collective action with cofinancing partners such as development finance institutions, governments, and agencies financing trust funds, including the GEF and the CTF. Coherent ESMAP technical assistance support was at the heart of the government of Vietnam’s energy planning, which included concurrent and sequential interventions on energy efficiency, energy subsidies, renewable energy, and governance (figure 3.4). Three factors were critical for success in Vietnam. The first was ESMAP’s status as a preferred partner on energy sector development. The second was coherent sequencing of ESMAP inter- ventions, such as developing the overall government policy framework ( World Bank 2010b) before supporting scale-up in the industrial sector (GCF proposal on energy efficiency for industrial enterprises in 2017) and the real sector (energy efficiency improvement in wastewater in the Vietnam Scaling Up Energy Efficiency project, 2018). The third factor of success was aligning and linking activities across ESMAP’s cross-cutting programs. For example, ESMAP complemented efforts to reduce subsidies under its Energy Subsi- dy Reform Facility by using its Annual Block Grants program to finance a software model for assessing the impacts of such pricing changes. ESMAP’s coherence in Vietnam has contributed to substantial results: the govern- ment of Vietnam is promoting more aggressive energy efficiency targets; the 2019–30 energy savings program aims to reduce the total nationwide energy Independent Evaluation Group World Bank Group    53 consumption by 8 to 10 percent, up to 2.5 exajoules (equal to approximately 78 times the daily electricity production of all the nuclear power plants in the world). IFC’s DSEE approaches are coherent with the policies and strategies of select client governments that aim to address climate change and energy short- ages. In Colombia, for example, IFC responded to a government request for support in reducing GHG emissions in the construction sector by promoting energy efficiency and water conservation in new buildings. In Indonesia, IFC aligned with the government policy in both its investment and advisory services to reduce GHG emissions from buildings through the enforcement of mandatory regulations. Between 2012 and 2015, the government passed 13 separate pieces of legislation promoting green buildings and supporting climate-smart investments. IFC’s DSEE approaches assisted with establishing green building codes using incentives (such as green mortgages and home insurance) to attract residential buyers and encourage private sector uptake (for example, by commercial and residential real-estate property developers). IFC addressed the government of South Africa’s issue of high peak loads and a need to reduce power consumption to stop frequent rolling blackouts by pro- viding advisory services to facilitate the adoption of green building standards.  orld Bank and Energy Sector Management Assistance Figure 3.4. W Program Contributions to Vietnam’s Energy Planning Renewable energy interventions (2014–18) Governance, markets, and planning interventions (2015–18) Energy subsidy reforms (2016) GCF Proposal—EE EE Improvement World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 Improving EE in for Industrial in (Waste-) Water Vietnam (2016) Enterprises (2017) (2018) Eighth Power Development Plan EE interventions (2016–18) 2021–30 (2019) Source: ICF International 2020. Note: EE = energy efficiency; GCF = Green Climate Fund. Coherence with Global Standards Mandating building standards via the construction sector, when the standards are formulated and aligned with global standards, continuously updated, and enforced, contributes to scaling DSEE. Mandating building energy efficiency standards (for example, building codes for roofing or windows) has proven to be effective in developed and industrialized countries by reducing energy use (IEA 2020a). Furthermore, an IEG literature review suggests that mandatory global standards can initiate a positive feedback loop of government-led enforcement, supply of technologies and 54 materials from the private sector, development of compliance capacity at enforcement agencies, and alignment with investor expectations that is reinforced over time. Review of advisory and knowledge work suggests that the Bank Group has been coherent in advocating for global building standards in client countries. The World Bank’s ESMAP and the Carbon Finance Unit have actively pro- moted global building standards and the benefits of enforcement since 2008 via advisory and knowledge work. The World Bank demonstrated coherence of its work with global DSEE standards in three different ways. First, through its knowledge products (for example, Mainstreaming Building Energy Efficien- cy Codes in Developing Countries [World Bank 2010a]), the World Bank eval- uated and showcased global experiences with building standards, extracted good practices in implementing building standards, and developed a carbon finance methodology for supporting programs and projects that invest in creating more energy-efficient buildings. Second, through the promotion of the United Nations’ Clean Development Mechanism, the World Bank has fur- ther advocated for global building standards in the past decade. Finally, the World Bank ESMAP has advocated for IFC’s global certification and standard processes (via EDGE) in its advisory programs. IFC is coherent with global DSEE standards with a singular focus on green buildings. IFC has been coherently using its EDGE certification and stan- dards process to advance DSEE priorities with its client groups in the Independent Evaluation Group World Bank Group    55 buildings sector. IFC uses the certification as an entry point to identify investment opportunities for greening buildings across subsectors in the six Bank Group Regions. IFC communicates consistently with its clients and partners and has prioritized EDGE standard adoption. In addition, EDGE is supported by several development partners, including the GEF and ESMAP. MIGA has been externally coherent with other development partners and DSEE-related building standards. MIGA has exhibited external coherence in contributing to energy efficiency by supporting investments in green buildings for health care facilities and offices in Türkiye via cofinancing arrangements with the European Bank for Reconstruction and Development. For example, MIGA’s provision of political risk insurance for a European Bank for Reconstruction and Development bond in Türkiye boosted the bond’s rating; the improved rating increased the bond’s attractiveness to investors, mobilizing funding from international and local banks and institutional investors (Rosca 2016). Moreover, MIGA was coherent with external green building certifications. The five subprojects for the health sector in Türkiye obtained external green building certifications, including LEED and Building Energy Performance—Türkiye. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 3 56 1  This evaluation uses the definition of coherence from the Development Assistance Commit- tee Network on Development Evaluation of the Organisation for Economic Co-operation and Development (https://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentas- sistance.htm#coherence-block). 2  Demand-based pricing is a demand-response technique that adds a surcharge to end-user utility bills based on their highest period of energy use in a month. 3  Peak load is the greatest amount of energy that consumers draw from the grid in a set period of time (for example, one day). 4  A cold chain is a low-temperature-controlled supply chain. An unbroken cold chain is an unin- terrupted series of refrigerated production, storage, and distribution activities, along with asso- ciated equipment and logistics, which maintain quality via a desired low-temperature range. 5  Cleaner production means using resources, such as raw materials, electricity, and water, more efficiently to minimize waste. 6  Excellence in Design for Greater Efficiencies certifies buildings as “green,” meaning that they efficiently use resources, such as energy, water, and materials. 7  Backward linkages characterize the relationship of an industry or institution with its supply chain. An industry has significant backward linkages when its production of outputs requires substantial intermediate inputs from many other industries. 8  A development finance institution is any financial institution that provides risk capital on a Independent Evaluation Group World Bank Group    57 noncommercial basis for economic development. A multilateral development bank is a finan- cial institution created by a group of countries to provide financing and professional advice for enhancing international economic development. As such, a multilateral development bank is a type of development finance institution. Other types of development finance institutions include national development banks, community development banks, and certain microfi- nance institutions, among others. 9  The World Bank has commissioned an external consultant and academics (UMAS: University Maritime Advisory Services) to study the potential for green bunker fuels, such as ammonia and hydrogen, and to promote decarbonization in the shipping and maritime industry. The second volume of the study will focus on liquefied natural gas as a transition fuel. 4 | Untapped Opportunities in Demand-Side Energy Efficiency Highlights The World Bank Group has not fully tapped four opportunities to create horizontal scaling opportunities to improve its alignment with Sustainable Development Goals 7 and 13 and the Paris Agree- ment goals: Including and tracking in its demand-side energy efficiency (DSEE) interventions indirect greenhouse gas emissions and socioeco- nomic development outcomes Addressing embodied carbon in all DSEE market segments that require building materials and facilitating backward linkages with other sectors related to construction, such as transport and manu- facturing Introducing in its DSEE interventions digital innovations that can increase end-user awareness of the importance of energy efficien- cy, improve end-user adoption, and reduce information asymmetry among energy service providers and users Promoting innovative financial solutions to support DSEE scaling, such as via blended finance, venture capital investments, and capi- tal market solutions 58   The Approach Paper for this evaluation was approved with the agree- ment that the evaluation team would identify forward-looking ways to scale energy efficiency interventions that are not based only on the Bank Group’s experience. At the time of the Approach Paper meeting, the evalu- ation team was authorized to identify DSEE best practices that were not just based on Bank Group work. The evaluation team did so by comparing Bank Group work at the country and intervention levels with global innovations in energy efficiency using software-aided content analysis of (i) external reports on DSEE public sector innovations, (ii) private sector DSEE innova- tions, (iii) regional and Practice Group strategy documents, (iv) innovative examples within the Bank Group, and (v) staff and client interviews. The analysis identified four main untapped opportunities to scale DSEE. Although the success factors described in the Factors of Success in Scal- ing section in chapter 2 provide guidelines for succeeding at typical DSEE engagements, the evaluation also identified the following four untapped opportunities for the Bank Group to approach DSEE differently and scale ex- ponentially: (i) measuring indirect emissions and socioeconomic outcomes in DSEE interventions’ results frameworks, (ii) adopting an embodied carbon approach in project scoping and design, (iii) incorporating digital innova- tions into project designs, and (iv) integrating financial innovations into project designs. Indirect Greenhouse Gas Emissions and Socioeconomic Outcomes Independent Evaluation Group World Bank Group    59 Indirect Greenhouse Gas Emissions The Bank Group has an opportunity to improve its alignment with SDG 13 and the Paris Agreement objectives by tracking and aiming to reduce in- direct GHG emissions. The 38 percent of Bank Group DSEE interventions that measure GHG emissions mostly measure direct emissions, not indirect emissions, limiting the Bank Group’s alignment with the Paris Agreement objectives and SDG 13. One way to strengthen the link between DSEE inter- ventions and climate objectives would be to include the client’s full scope of emissions in the design of new interventions and measure it throughout the life of projects. The Greenhouse Gas Protocol (the world’s most widely used GHG accounting standards)1 classifies the GHGs that a public or private or- ganization emits directly (for example, by running its factories and vehicles) as scope 1 emissions (figure 4.1). It classifies the GHGs that an organization emits indirectly as scope 2 or scope 3 emissions. Scope 2 emissions are indi- rect emissions from the generation of purchased energy (for example, when a firm buys electricity for heating and cooling buildings). Scope 3 emissions are all indirect emissions not included in scope 2 that occur in a firm’s sup- ply chain. Scope 3 emissions include, for example, emissions resulting from suppliers manufacturing inputs and customers using outputs. Of sampled Bank Group interventions that measure GHG emissions, 40 percent of the World Bank’s interventions and 92 percent of those by IFC measure only di- rect (scope 1) emissions. Few cover scope 2 emissions, and none tackle scope 3 emissions. Indirect emissions, and particularly scope 3 emissions, are often responsible for an organization’s biggest GHG impacts (Carbon Trust 2019). Designing interventions that tackle direct and indirect emissions—especially scope 3 emissions—and facilitating clients’ measurement of them open up opportunities for horizontal scaling. At the country and regional levels, the Bank Group could scale development outcomes horizontally across entire World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 supply chains (for example, hospitality, retail, manufacturing, construction, and shipping) if the DSEE interventions were designed to tackle both direct and indirect emissions. This would entail using carbon pricing for all Bank Group interventions,2 tracking implementation of national and subnation- al carbon-crediting mechanisms based on the interventions (World Bank 2020), and tracking the three types of scope emissions. Adopting internal carbon pricing would have four benefits. First, it would allow the Bank Group to facilitate client-level measurement of projects’ current GHG emissions versus GHG emissions estimates in the future and track them better. Sec- ond, it would more broadly signal to clients the need to effectively measure direct and indirect emissions and the implicit carbon price. Third, it would create incentives to develop multisectoral approaches to DSEE (for exam- ple, energy-transport, or energy–macrofiscal and trade; box 4.1). Finally, it would facilitate a holistic view of DSEE outcomes to address commitments to Paris Agreement alignment, SDG 13, and SDG 7. In this regard, implementing shadow carbon pricing pilots and mainstreaming this requirement in Bank Group lending and IFC investment projects are steps in the right direction. 60 Figure 4.1. Emissions along the Supply Chain Scope 3 Scope 3 Indirect Indirect Scope 1 Direct Leased assets Scope 2 Indirect Purchased goods Employee Transportation Investments and services commuting Company and distribution Purchased facilities electricity, steam, heating, and cooling for own use Capital goods Business travel Processing of Franchises sold products Company vehicles Fuel and Waste generated energy-related in operations Use of sold Leased assets activities products Transportation and End-of-life treatment of distribution sold products Upstream (suppliers) Firm/organization Downstream (consumers) Sources: Independent Evaluation Group; World Resources Institute and World Business Council for Sustainable Development 2011. Independent Evaluation Group World Bank Group    61  reening the Energy-Transport Nexus for Reducing Indirect Box 4.1. G Emissions and Developing Horizontal Scaling Opportunities The potential for horizontal scaling remains untapped at the nexus of the energy and transport sectors. For example, building or expanding airport infrastructure would typically result in a significant increase in air travel that would lead to increases in greenhouse gas emissions. Climate mitigation actions focused on reducing direct and indirect emissions implemented by the airport could somewhat counter such an im- pact and help make a case for Paris Agreement alignment of investments that aim to expand airports. World Bank Group teams can support airport sector clients in reduc- ing their greenhouse gas footprints by identifying and measuring indirect emissions in their operations. Similarly, for firms in the shipping and maritime logistics sectors, the International Finance Corporation–Global Environment Facility’s Green Shipping In- vestment Platform targets energy efficiency gains. Horizontal scaling in such scenarios would require designing multicountry, multisector interventions that could involve all three Bank Group institutions and aim to reduce direct and indirect emissions. Source: Independent Evaluation Group. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 Socioeconomic Outcomes of Demand-Side Energy Efficiency Interventions The Bank Group could scale DSEE by targeting and measuring development benefits beyond energy savings and GHG emissions. The Bank Group broadly recognizes that DSEE improvements are among the most cost-effective means of saving energy and reducing GHG emissions. Although Bank Group DSEE interventions reviewed during the evaluation period did not track energy savings and direct emissions reductions as much as they could have (as discussed in chapter 2), they nevertheless prioritized these two primary DSEE development outcomes over socioeconomic and other benefits (figure 4.2). Some interventions aimed at market creation, institutional strengthening, and capacity building. However, fewer than 30 percent of sampled Bank Group DSEE projects in the evaluation period measured socioeconomic benefits. For example, only 24 percent of sampled projects differentiated between men and women in terms of achieving or benefiting 62 from energy efficiency outcomes. According to Bank Group surveys, women are more deliberate and conscious of emissions and energy use than men are, and, if appropriately targeted, they can maximize the benefits of DSEE for themselves, their families, and the environment. Only 15 percent of the sampled Bank Group DSEE interventions measured job creation. Retrofitting buildings and greening public infrastructure tend to create net new jobs because the energy industry has a lower job intensity than the construction industry, and the effects extend over time as energy savings reduce utility bills for years, redirecting spending away from energy to other sectors with higher job intensities. Moreover, only 2 percent of sampled projects measured health or well-being, although lowering emissions and air pollution (some arising during combustion) is key to improve respiratory health. Including these outcomes in Bank Group operation design and measuring them throughout implementation would support scaling by demonstrating the total value of DSEE interventions to clients and partners.  arious Demand-Side Energy Efficiency Outcomes as a Figure 4.2. V Percentage of the World Bank Group Demand-Side Energy Efficiency Portfolio Greenhouse gas emissions Energy savings Institutional strengthening and capacity building Market creation Independent Evaluation Group World Bank Group    63 Gender issues Financial improvement Outcome Job creation Service delivery Policy or regulatory reform Affordability and expenditure savings Environmental mitigation Access and supply-side integration Health and well-being Energy security Property value 0 10 20 30 40 50 60 Share of projects (%, n = 126) Measured Intended but not measured Source: Independent Evaluation Group. Embodied Carbon for the Building Industry Embodied carbon—that is, all emissions associated with the development and use of construction material—needs to be tackled and measured. Em- bodied carbon refers to the GHG emissions associated with the manufactur- ing, transportation, installation, maintenance, and disposal of construction materials (Carbon Leadership Forum3). In other words, embodied carbon corresponds to scope 3 emissions for buildings. Embodied carbon can be calculated as a metric or indicator relating to global warming potential and expressed in CO2-equivalent units. Embodied carbon stands in contrast to operational carbon, which refers to emissions generated by fuel consump- tion for heating, cooling, lighting, and powering machines and other devices over the course of a building’s lifetime (scope 1 and scope 2 emissions). Total carbon is the sum of embodied carbon and operational carbon. To reach net zero emissions, the total carbon emissions of a building must be addressed, not merely carbon emissions during the building’s lifetime. Addressing embodied carbon is urgently needed to meet climate targets. Op- erational carbon currently accounts for 28 percent of global GHG emissions. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 Though embodied carbon accounts for only 11 percent currently, embodied and operational carbon emission levels will be the same by 2050 because of the projected increase in construction (SPOT UL 2020). Emissions reduced now are more critical than emissions reduced later because of the carbon lock-in principle.4 Hence, reducing embodied carbon is as important as—or more important than—reducing operational carbon. Additionally, many of the most impactful decisions related to embodied carbon happen in the early stages of a building project (and, by association, a Bank Group DSEE inter- vention). Addressing embodied carbon is particularly important for reaching SDG 13 and Paris Agreement climate targets, because these emissions will likely be “front-loaded”—accrued before end users have an opportunity to save energy—in both greenfield construction and brownfield upgrades over the next 10 years, unlike annual operational carbon or ongoing emissions. For greenfield projects with long construction periods, design choices made today will lock in emissions for a building that may not open for another 5 to 10 years. The Bank Group cannot afford to fully address operational carbon before addressing embodied carbon but must do so in parallel. 64 The embodied carbon approach addresses the backward linkages to buildings and related DSEE efforts concentrated in retrofits and energy end users. As shown by the embodied carbon emissions over a building’s life cycle, the building industry influences most major sectors of global GHG emissions, including transport, manufacturing, and forestry management (figure 4.3). Figure 4.3. Temporal View of a Typical Life Cycle for Buildings Embodied Embodied Embodied Embodied CO2 GHG Emissions Time Extract raw Transport Manufacture Transport materials to factory products to site Embodied Embodied Embodied Embodied Embodied Operating CO2 GHG Emissions Time Construct Use and Demolish Haul away Landfill or the building maintain the building waste recycle the building materials Source: Winters-Downey 2021. Independent Evaluation Group World Bank Group    65 Note: CO2 = carbon dioxide; GHG = greenhouse gas. An embodied carbon approach uses an ex ante method to minimize GHG emissions before a building is constructed. The Bank Group has been fo- cused on operational carbon issues in the context of DSEE. The typical Bank Group approach focuses on ex post techniques (such as replacing light bulbs or installing smart meters) to reduce direct (scope 1) GHG emissions after a building has been constructed. The more ambitious embodied carbon ap- proach focuses on designing buildings from the start to minimize their total GHG emissions to reduce indirect (scopes 2 and 3) GHG emissions through- out the buildings’ life cycle. The embodied carbon approach also addresses industry emissions that are hard to abate. Construction generates emissions that are hard to abate because several of its essential inputs (for example, cement, steel, alumi- num, and glass) are manufactured in industrial processes that require high temperatures, which clean energy cannot yet reliably provide. Furthermore, many of the inputs to construction must be delivered by heavy-duty trans- port (such as shipping, trucking, and aviation), which also produces emis- sions that are hard to abate. Processing-related emissions from construction and fuel transformation processes release CO2 and other pollutants directly into the air. For example, approximately 60 percent of emissions from ce- ment production are process emissions that cannot be reduced through fuel switching alone. Therefore, addressing industrial emissions tied to the con- struction industry is vital if the world is to meet SDG 13 and the Paris Agree- ment objective of limiting global warming to well below 2 degrees Celsius by 2050 (Hobley 2020). Embodied carbon approaches can help address health and related socio- economic outcomes. Building materials have a direct impact on communi- ty health because their supply chains rely on manufacturing facilities and World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 power plants that release heavy metals, toxic chemicals, and particulate matter into nearby communities’ water, air, and food sources, causing short- and long-term health problems. Therefore, addressing embodied carbon approaches for buildings can reduce impacts on communities in two ways: (i) by mitigating the global impacts of climate change (as it relates to the Bank Group’s Paris Agreement alignment and Climate Change Action Plan) and (ii) by decreasing the local environmental and health impacts from industrial pollution (as it relates to SDG 7 contributions and related SDGs).5 Digital Innovations Digitalization is multiplying the opportunities for scaling energy efficien- cy. Digitalization offers the potential to increase energy efficiency through technologies that gather and analyze data and then use the information to make changes to the physical environment to optimize energy efficiency. Technologies such as sensors and smart meters collect data on energy use and conditions affecting energy use (such as weather; figure 4.4). Data are 66 processed into useful information through data analysis technologies such as artificial intelligence algorithms. Finally, the processed information is sent to devices that can effect physical changes to optimize energy use. Some devices require human action to optimize energy use. For example, a smartphone app can suggest an energy-efficient route to work, but the commuter must act on that advice. Other devices (such as switches in a large building’s cooling system or robots on a production line) are capable of opti- mizing energy efficiency more autonomously. Figure 4.4. Digitalization of Demand-Side Energy Efficiency Systems Data gathering Data analysis Action Sensors Algorithms Automation Meters Artificial intelligence Controls Interfaces Digital simulations 3D printing Interfaces Source: International Energy Agency 2020a. Digital technologies are applicable in all energy end-use sectors. Increasing- Independent Evaluation Group World Bank Group    67 ly, residential and commercial buildings are equipped with smart appliances and intelligent energy management systems. The connectivity benefits of digitalization allow digital technologies to increase both DSEE and the effi- ciency of the entire energy system more broadly. Digitalization could provide opportunities to optimize energy use in all energy-consuming sectors and address the lack of information that leads to unsound decision-making at the end-user level. For example, at the building level, smart lights can optimize consumption based on requirements and usage patterns. Energy use during the construction of buildings could also be significantly reduced by apply- ing digital tools and technologies and providing more accurate and timely information across the supply chain. This is known as real-time construction management, which brings together all on-site information on one plat- form, improving productivity and reducing costs. Similarly, at the industry level, digital technologies could change the way industries produce, process, and deliver products. Industry is responsible for approximately 38 percent of global final energy consumption and 24 percent of total CO2 emissions, and it is estimated that optimization enabled by digitalization could help achieve energy savings of at least 10 to 20 percent (United Nations Environ- ment Programme).6 This would be in addition to energy savings that could be achieved if the building itself were to be digitalized. Digital transformation of the energy system can bridge many gaps at the system level while catalyzing new opportunities through a deeper transfor- mation of how the devices, systems, and participants connect and commu- nicate. Some key benefits are better connectivity, trust, and transparency: digitalization of the energy system can provide better connectivity with cus- tomers, suppliers, and other partners to achieve better DSEE outcomes. Dig- italization can significantly improve trust among these various participants and enable an open, transparent, competitive, more resilient, and nondis- criminatory energy market, resulting in many benefits for the economy and society. Open data across the system can also enable better decision-making for businesses and policy makers and spark innovations and inventions. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 Another key benefit of digitalizing DSEE is improving supply chain manage- ment: digital technology solutions can unlock significant value for indus- try participants across the entire energy supply chain. The supply chain of variable renewable energy technologies needs to be robust to leverage its full potential, and big data, machine learning, and advanced data analytics allow various renewable energy sources to be managed with maximum flexibility and optimization. Digitalization can streamline various processes across the entire energy supply chain and significantly improve speed and cost while providing improved visibility and real-time insights into the processes. Digital solutions can increase DSEE awareness, improve adoption, and reduce information asymmetry between energy service providers and users. Digitalization of energy systems has successfully increased DSEE aware- ness and adoption in developed countries, UMICs, and LMICs. For example, Opower’s cloud-based software for the utility industry provides end users with more information about their energy efficiency using artificial intelli- gence and behavioral science. Opower’s rate coaching service uses weekly 68 customized email tips to guide users through time-of-use pricing plans to reduce both their costs and their consumption. Billing improvements include personalized context such as benchmarking a customer’s relative level of energy efficiency—relative to both similar homes and a target zone—in a simple, graphical way. Blockchain is another powerful tool for energy efficiency innovations. Blockchain is a secure online ledger that records all transactions through a peer-to-peer network. It is used in the energy system because of its incor- ruptibility, its ability to eliminate intermediaries, and its potential to reduce costs even for very small transactions that otherwise would not be eco- nomically viable. The Brooklyn microgrid is a leading example of a block- chain-based market that allows residents with solar panels to sell excess energy to their neighbors in peer-to-peer transactions, helping to manage end-user demand on the main grid (Mengelkamp et al. 2018). Another po- tential application of blockchain would empower end users to directly mon- etize their energy efficiency (Khatoon et al. 2019). The system would require each user to achieve a certain level of energy savings (as is already required in several EU countries). Users who exceeded the required energy savings would be able to use a blockchain-based system to sell their surplus energy savings to users unable to meet the energy savings requirement, allowing the buyers to avoid paying hefty fines. The potential for directly monetizing energy efficiency would motivate users to increase their efficiency. In both cases, blockchain-based automated trading is an efficient way of delivering Independent Evaluation Group World Bank Group    69 price signals to consumers about the cost of energy and the corresponding monetary value of energy efficiency, which stimulates efficient consumption and reduces costs. Despite all these benefits, the Bank Group’s DSEE approaches have not yet embraced digitalization opportunities. The World Bank has acknowledged in energy sector work the need to incorporate digital components in energy efficiency interventions (IEA et al. 2022). However, it has not yet done so. Financial Innovations The Bank Group has a comprehensive tool kit of financial instruments, in- cluding blended finance, that is underused for DSEE. Both the World Bank and IFC have significant experience, in many cases with successful development outcomes, in creating DSEE blended financing facilities. The core objectives of such facilities have been to attract private capital to scale DSEE investments, in turn making the facilities self-sustaining and creating a DSEE market—that is, an ecosystem of energy efficiency–related products and services created by a set of interventions that reduce energy use and improve firm and house- hold productivity in the client countries. These facilities, however, have been deployed toward DSEE approaches in only a few countries. Egypt, China, and India are successful examples of the Bank Group’s blended finance support to DSEE that can be replicated in other MICs. In Egypt, a cred- it risk guarantee mechanism for DSEE supported by a 2005 World Bank–GEF project attracted private capital and led to a scale-up of DSEE investment. The IFC-GEF China Utility-Based Energy Efficiency Finance Program created an environment for commercial banks, private companies, and government agen- cies to jointly design sustainable financing models for energy efficiency and renewable energy. The program provides marketing, engineering, project de- velopment, and financing services to commercial, industrial, and multihouse- hold residential energy users. The World Bank used China’s Utility-Based Energy Efficiency Finance Program approach in India, where it supported World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 DSEE by deploying several instruments, including a Program-for-Results proj- ect, an International Bank for Reconstruction and Development partial risk guarantee blended with private capital and technical assistance in part fund- ed by ESMAP. The World Bank’s interventions supported a range of different types of contract agreements for ESCOs with different risk and responsibility sharing options and auditing and procurement standards. The program, which is still ongoing, is promising in terms of DSEE financing scale-up, including through mobilization plans to crowd in domestic investors and commercial financiers supporting it via ESCOs. Untapped opportunities exist to implement similar approaches in several more countries. The use of early-stage grant and venture capital investments can contribute to DSEE development. In 2020, IFC and the UK government’s International Climate Finance jointly launched TechEmerge, a matchmaking program that helps innovative technology start-ups build commercial relationships with end-user firms. For example, TechEmerge’s Sustainable Cooling Innovation program in Latin America offers a pool of $1.5 million in grant funding to 70 support pilot implementation of cooling solutions. The DSEE start-ups were competitively selected by TechEmerge and its expert panel of independent ad- visers and then matched with top end-use companies in Colombia and Mexico to support DSEE approaches. Like blended finance, early-stage venture invest- ment for DSEE is underused and can be significantly expanded across supply chains (for example, for building materials and manufacturing technologies). Innovative financial solutions to support DSEE scale-up also exist outside of the Bank Group (for example, via capital market solutions). The Bank for International Settlements Innovation Hub Centre Hong Kong SAR and the Hong Kong Monetary Authority joined forces with the technology industry on Project Genesis to build a prototype digital infrastructure that enables energy transition investments, improves transparency on the use of pro- ceeds, and thereby helps meet regional and global environmental and sus- tainability goals. As the first energy transition project, Genesis will explore the tokenization of green bonds enabling investment in small denomina- tions toward DSEE, combined with real-time tracking of environmental outputs and outcomes. Such models could be facilitated through joint DSEE approaches from the World Bank’s Energy and Extractives, and Finance, Competitiveness, and Innovation GPs. Performance guarantees to finance deep retrofitting are an untapped op- portunity for scaling DSEE and facilitating leapfrogging. The government of the Netherlands created the Energiesprong program in 2010 to increase the energy efficiency of new and existing buildings. In contrast with shallow Independent Evaluation Group World Bank Group    71 retrofits, such as installing heat pumps or basic isolated hardware upgrades, Energiesprong introduces a systemwide retrofit that increases energy sav- ings by more than 50 percent, reduces maintenance costs, and introduces attractive designs with upgraded features, which increase the value of the buildings for the end users. Prefabricated insulating panels are fastened to a building’s exterior. The panels are manufactured based on digital scans of the building rather than drawings and specifications and can be installed in less than a week. A key feature of the Energiesprong program is its financing through a 30-year performance guarantee: renovations and new buildings are funded by future savings in energy, maintenance, and repair costs over those 30 years. In housing associations, tenants pay into an energy service plan the same amount they used to pay directly to their energy supplier. The housing association uses this income stream to partly fund the renovation, and the performance of the retrofit is guaranteed by the service company. Adoption of such innovations can help select client countries (for example, LMICs) to achieve leapfrogging without following the development trajecto- ry of UMICs and highly industrialized countries. Performance guarantees to finance deep retrofitting require legislative changes but have significant potential for DSEE scale-up. Typically, local or municipal legislation needs to be amended to allow monthly energy bills to be converted into monthly energy service fees for housing associations. These legislative changes entice suppliers to invest in the manufacturing of the components needed for such house makeovers, meeting mass customi- zation and industrialization objectives in addition to the adoption of DSEE quality and cost and pricing standards for residential buildings. More than 5,000 homes in the Netherlands have been retrofitted with Energiesprong since 2010. In 2018, the first 10 homes were retrofitted in the United King- dom as part of its Energiesprong pilot program. Energiesprong has also been introduced in France, Canada, and California and New York in the United States and piloted in Rwanda (a case study country) and South Africa. World Bank Group Support to Demand-Side Energy Efficiency  Chapter 4 72 1  See the Greenhouse Gas Protocol at https://ghgprotocol.org. 2  Carbon pricing is an instrument that captures the external costs of greenhouse gas emis- sions—the costs of emissions that the public pays for, such as damage to crops, health care costs from heatwaves and droughts, and loss of property from flooding and sea-level rise— and ties them to their sources through a price, usually in the form of a price on the carbon dioxide emitted. A price on carbon helps shift the burden for the damage from greenhouse gas emissions back to those who are responsible for it and can avoid it. Instead of dictating who should reduce emissions where and how, a carbon price provides an economic signal to emit- ters and allows them to decide to either transform their activities and lower their emissions or continue emitting and paying for their emissions. 3  See https://carbonleadershipforum.org. 4  Carbon lock-in occurs when fossil-fuel–intensive systems delay or prevent the transition to low-carbon alternatives—a situation that can seriously imperil climate action. 5  Sustainable Development Goals 3 (health), 8 (economic growth, decent work, and productivi- ty), and 11 (sustainable cities and communities). 6  See https://www.unep.org/explore-topics/energy/what-we-do/digitalisation-energy. Independent Evaluation Group World Bank Group    73 5 | Conclusions and Recommendations DSEE is important for global sustainability, and the Bank Group has com- mitted to it, but scaling it presents numerous challenges. The World Bank recently made two overarching corporate commitments: (i) to achieve Paris Agreement alignment by 2023 (World Bank) or 2025 (IFC and MIGA), and (ii) to contribute to the achievement of SDG targets, which the Bank Group has internalized as part of its overarching poverty alleviation and shared prosperity goals. DSEE is critical for Paris Agreement alignment and contrib- uting at scale to several SDGs, including SDG 13 on climate change, SDG 7 on energy efficiency, SDG 3 on health, SDG 8 on growth and productivity, and SDG 11 on sustainable communities. Yet scaling DSEE—especially horizon- tally—presents numerous challenges, including limited and volatile country demand, difficulty in articulating tangible outcomes, and the complexities of leveraging global programs. The Bank Group has been mostly effective at the intervention level but unable to scale DSEE. World Bank DSEE projects have been effective, but the World Bank has been unable to scale DSEE approaches beyond a select group of UMICs because of a variety of internal and external constraints. At the in- dividual investment project level, IFC has been only partially effective, most- ly because of overly ambitious targets for individual projects. IFC’s limited scale-up has occurred mostly thanks to the green buildings programmatic approach. MIGA DSEE approaches are limited to date, and their effectiveness could not be evaluated. The Bank Group has supported coherent sector-level DSEE approaches, but it has exhibited limited coherence across the three institutions and with devel- opment partners. The Bank Group is internally coherent within the energy sector, within GPs (World Bank), and within industry groups (IFC). The Bank Group has weak coherence across the three institutions, except for program- matic coordination in creditworthy MICs (for example, China and India). The 74   Bank Group has operated coherently with World Bank–administered trust funds (for example, GEF) but not with other development partners (bilateral, multilateral, or development finance institutions). Given the growing importance of DSEE for global sustainability, yet recog- nizing the hurdles facing the Bank Group in supporting DSEE, the evaluation proposes a pivot for the Bank Group DSEE approaches and associated out- comes toward the decarbonization agenda. Although a decarbonization focus may temporarily slow the rate of growth of DSEE-only lending commitments in some countries, it can enable the Bank Group to reach SDG 7 and Paris Agreement targets more effectively. Such a pivot places greater emphasis on reducing GHG emissions (including both direct and indirect emissions, especially scope 3 emissions across supply chains) and broadening DSEE outcomes to include socioeconomic benefits (such as health, jobs, productiv- ity, and security). The weight of the global priorities and the limited scale-up on DSEE to date require the Bank Group to fully reorient its DSEE approaches and outcome aspirations from an energy savings focus to a broader decarbonization focus. With this necessary pivot of DSEE approaches toward global priorities as the backdrop, this evaluation proposes four near-term actions. Recommendation 1 (Bank Group). Intensify DSEE support to MICs for decarbonization and wider socioeconomic benefits. By supporting MICs in scaling up DSEE, the Bank Group would make the most difference in closing Independent Evaluation Group World Bank Group    75 GHG emissions gaps while also contributing to economic and social devel- opment outcomes. Intensifying scale-up in MICs requires an increased focus by the World Bank on multisectoral and horizontal scale-up approaches in project design. Similarly, this recommendation entails an increased role in MICs for IFC and MIGA—including through IFC upstream interventions and MIGA business development approaches—in countries that are ready for the greening of public assets and assets of SOEs (for example, China, India), subject to client demand. Recommendation 2 (World Bank and IFC). Develop energy efficiency sector-specific approaches in a select group of LMICs that seek productivity gains alongside or via DSEE, even if energy efficiency policy reforms are in early stages. Bank Group DSEE efforts in countries with a policy environment that is not conducive to energy efficiency reforms, inefficient capital allocation to energy generation (for example, fossil subsidies), or low emissions per capita are unlikely to lead to meaningful outcomes. Select LMICs, however, are promising scale-up targets for the World Bank and IFC, especially if they focus their DSEE interventions on energy-intensive sectors or subsectors, such as the industrial market segment in Uzbekistan or the commercial construction market segment in Indonesia. In this context, productivity gains refer to firm-level productivity gains—that is, the amount of output a firm can produce with a given set of inputs. Scale up parallel technical assistance and IFC upstream and advisory services targeting new client types and cumulative investments, subject to client demand. Recommendation 3 (World Bank and IFC). Expand DSEE approaches by incorporating reduction of indirect emissions (scope 3), including embod- ied and operational carbon, in DSEE project design. The current approach of designing for direct (scope 1) emissions is necessary but not sufficient for the pivot to decarbonization and for steering greater financing flows toward DSEE as part of the multilateral development banks’ Paris Agreement alignment approach. This recommendation entails incorporating scope 3 (and, in some cases, scope 2) risks for these emissions ex ante (that is, at the time of project World Bank Group Support to Demand-Side Energy Efficiency  Chapter 5 design discussions, during postclient mandate activities, and when crafting loan agreements). This recommendation does not mean every project needs to track scope 3 emissions but suggests designing World Bank operations and IFC projects differently. It will imply, for example, focusing on horizontal scaling through longer-term, repeat-engagement, and multisector approaches (similar to what the Bank Group has achieved in India and Mexico) that cut across up- stream and downstream activities. In this regard, IFC’s recent advisory services initiative Partnership for Cleaner Textile is promising. Recommendation 4 (World Bank and IFC). Exploit untapped DSEE oppor- tunities and help clients leapfrog—that is, develop innovative approaches that adopt and adapt digital and financial solutions from developed coun- tries by exploring cross–Practice Group (World Bank) and cross–industry group (IFC) interventions and approaches. This would entail integrating DSEE with untapped opportunities, such as digital and financial instrument innovations, that could support leapfrogging efforts in some cases. Examples include the following: convening and supporting existing IFC clients (for 76 example, firms operating in retail supply chains, top GHG-emitting firms, and firms owning and operating data centers) to incorporate digital solu- tions, such as intelligent monitoring and artificial intelligence–based energy optimization within their building portfolios; leveraging SSEE activities (for example, combining electricity utility upgrades with innovative guarantee mechanisms to promote DSEE); using multistakeholder approaches to invest in local technology start-ups (for example, Negawatt in Ghana); designing behavioral policy interventions (China); and communicating successful pilot cases. This recommendation would entail exploring integrated approaches, such as between the Energy and Extractives GP and the Digital Development GP; between the Energy and Extractives GP and the Macroeconomics, Trade, and Investment GP; between the Energy and Extractives GP and the Finance, Competitiveness, and Innovation GP; and among IFC’s Infrastructure and Natural Resources; Manufacturing, Agribusiness, and Services; and Disrup- tive Technologies and Funds industry groups. Independent Evaluation Group World Bank Group    77 Bibliography ACEEE (American Council for an Energy-Efficient Economy). 2011. “How Does Ener- gy Efficiency Create Jobs?” Fact Sheet, ACEEE, Washington, DC. ACEEE (American Council for an Energy-Efficient Economy). 2017. “Energy Efficien- cy—Jobs and Investments.” Fact Sheet, ACEEE, Washington, DC. Batlle, Carlos, and Pablo Rodilla. 2009. “Electricity Demand Response Tools: Current Status and Outstanding Issues.” European Review of Energy Markets 3 (2): 27. Carbon Trust. 2019. “New Framework to Help Businesses Transition to a Net Zero Economy.” Carbon Trust, London. https://www.carbontrust.com/news-and- insights/news/new-framework-to-help-businesses-transition-to-a-net-zero- economy. Clark, Alex, and Philippe Benoit. 2022. Greenhouse Gas Emissions from State-Owned Enterprises: A Preliminary Inventory. New York: Columbia School of International World Bank Group Support to Demand-Side Energy Efficiency  Bibliography and Public Affairs Center on Global Energy Policy. Dominioni, Goran, and Dominik Englert. 2022. Carbon Revenues from International Shipping: Enabling an Effective and Equitable Energy Transition. Technical Paper. Washington, DC: World Bank. Det Norske Veritas. 2022. Energy Transition Outlook 2022. Høvik: Det Norske Veritas. https://www.dnv.com/energy-transition-outlook/download.html. Eid, Cherrelle, Elta Koliou, Mercedes Valles, Javier Reneses, and Rudi Hakvoort. 2016. “Time-Based Pricing and Electricity Demand Response: Existing Barriers and Next Steps.” Utilities Policy 40 (June): 15–25. Englert, Dominik, Andrew Losos, Carlo Raucci, and Tristan Smith. 2021. The Poten- tial of Zero-Carbon Bunker Fuels in Developing Countries. Washington, DC: World Bank. ESMAP (Energy Sector Management Assistance Program). 2019. Ghana Public Build- ings Energy Efficiency Report. Washington, DC: World Bank. 78 ESMAP (Energy Sector Management Assistance Program). 2022. Ghana Sustainable Cities Report. Washington, DC: World Bank. Eurofound. 2016. “European Quality of Life Survey 2016—Data Visualization.” Eurofound, Dublin. https://www.eurofound.europa.eu/data/european-quali- ty-of-life-survey. Hobley, Anthony Robert. 2020. “Tackling the Harder-to-Abate Sectors: Join the Con- versation on 7 July.” World Economic Forum, July 1, 2020. https://www.weforum. org/agenda/2020/07/tackling-the-hard-to-abate-sectors-join-the-conversation. ICF International. 2020. External Evaluation of the Energy Sector Management Assis- tance Program (ESMAP): Final Evaluation Report. Reston: ICF International. IEA (International Energy Agency). 2014. Energy Efficiency Indicators: Fundamentals on Statistics. Paris: Organisation for Economic Co-Operation and Development/IEA. IEA (International Energy Agency). 2019. Multiple Benefits of Energy Efficiency. Paris: IEA. IEA (International Energy Agency). 2020a. Energy Efficiency 2020. Paris: IEA. IEA (International Energy Agency). 2020b. Sustainable Recovery Plan. Paris: IEA. IEA (International Energy Agency). 2020c. Uzbekistan Energy Profile. Country Report. Paris: IEA. IEA (International Energy Agency). 2020d. World Energy Investment 2020. Paris: IEA. Independent Evaluation Group World Bank Group    79 IEA (International Energy Agency). 2021a. Energy Efficiency 2021. Paris: IEA. IEA (International Energy Agency) , IRENA (International Renewable Energy Agen- cy), UNSD (United Nations Statistics Division), World Bank, and WHO (World Health Organization). 2022. Tracking SDG 7: The Energy Progress Report 2022. Washington, DC: World Bank. IEA (International Energy Agency). 2021c. Net Zero by 2050: A Roadmap for the Glob- al Energy Sector. 4th ed. Paris: IEA. IEA (International Energy Agency). 2021d. Tracking Buildings 2021—Analysis and Key Findings. Paris: IEA. IEA (International Energy Agency). 2021b. “Greenhouse Gas Emissions from Ener- gy: Overview.” IEA, Paris. https://www.iea.org/reports/greenhouse-gas-emis- sions-from-energy-overview. IFC (International Finance Corporation). 2019. Green Buildings: A Finance and Policy Blueprint for Emerging Markets. Washington, DC: IFC. IMF(International Monetary Fund). 2022. “Jordan: Third Review under the Extended Arrangement under the Extended Fund Facility and Request for Modification of Performance Criteria—Press Release; Staff Report; and Statement by the Execu- tive Director for Jordan.” Country Report 22/4, IMF, Washington, DC. Khatoon, Asma, Piyush Verma, Jo Southernwood, Beth Massey, and Peter Corcoran. 2019. “Blockchain in Energy Efficiency: Potential Applications and Benefits.” Energies 12 (17): 3317. https://doi.org/10.3390/en12173317. Leichenko, Robin, and Julie A. Silva. 2014. “Climate Change and Poverty: Vulnerabil- ity, Impacts, and Alleviation Strategies.” WIREs Climate Change 5 (4): 539–56. Mengelkamp, Esther, Johannes Gärttner, Kerstin Rock, Scott Kessler, Lawrence Orsi- World Bank Group Support to Demand-Side Energy Efficiency  Bibliography ni, and Christof Weinhardt. 2018. “Designing Microgrid Energy Markets: A Case Study: The Brooklyn Microgrid.” Applied Energy 210 (January): 870–80. https:// doi.org/10.1016/j.apenergy.2017.06.054. Naeher, Dominik, Raghavan Narayanan, and Virginia Ziulu. 2021. “Impacts of Energy Efficiency Projects in Developing Countries: Evidence from a Spatial Differ- ence-in-Differences Analysis in Malawi.” Policy Research Working Paper 9842, World Bank, Washington, DC. https://documents.worldbank.org/en/publication/ documents-reports/documentdetail/360541636133634224/Impacts-of-Ener- gy-Efficiency-Projects-in-Developing-Countries-Evidence-from-a-Spatial-Dif- ference-in-Differences-Analysis-in-Malawi. OECD (Organisation for Economic Co-operation and Development). 2022. Decar- bonising Buildings in Cities and Regions. Paris: OECD. https://doi.org/10.1787/ a48ce566-en. Rosca, Olga. 2016. “Unique Financing Model as EBRD Backs Elazig Hospital PPP Project in Türkiye.” European Bank for Reconstruction and Development, No- vember 30, 2016. https://www.ebrd.com/news/2016/unique-financing-model- 80 as-ebrd-backs-elazig-hospital-ppp-project-in-turkey-.html. SEforALL (Sustainable Energy for All). 2021. SEforALL Analysis of SDG7 Progress—2021. Vienna: SEforALL. https://www.seforall.org/data-stories/ seforall-analysis-of-sdg7-progress-2021. Skoczkowski, Tadeusz. 2017. “White Certificate System in Poland: Lessons Learned.” Paper presented at the International Energy Agency, Energy Saving Policies, and the Regulatory Assistance Project Market-based Instruments for Energy Efficiency Policy Choice and Design for the Energy Transition meeting, Brussels, January 24, 2017. SPOT UL. 2020. “Embodied vs Operational Carbon.” SPOT (blog), August 16, 2020. https://spot.ul.com/blog/embodied-vs-operational-carbon. SuM4All (Sustainable Mobility for All). 2021. Sustainable Electric Mobility: Building Blocks and Policy Recommendations. Washington, DC: World Bank. UN (United Nations). n.d.-a. “The Climate Crisis—A Race We Can Win.” UN75: 2020 and Beyond. UN, New York. https://www.un.org/en/un75/climate-crisis-race- we-can-win. UN (United Nations). n.d.-b. “Goal 13: Climate Action.” Sustainable Develop- ment Goals. UN, New York. https://www.un.org/sustainabledevelopment/cli- mate-change. Winters-Downey, Erika. 2021. “The Elephant in the Room: Embodied Carbon Reduc- tion in Practice.” McCownGordon, June 9, 2021. https://mccowngordon.com/ Independent Evaluation Group World Bank Group    81 the-elephant-in-the-room-embodied-carbon-reduction-in-practice. WMO (World Meteorological Organization). 2020. WMO Statement on the State of the Global Climate in 2019. Geneva: WMO. https://library.wmo.int/index. php?lvl=notice_display&id=21700. World Bank. 2010a. Mainstreaming Building Energy Efficiency Codes in Developing Countries. Washington, DC: World Bank. World Bank. 2010b. Vietnam: Expanding Opportunities for Energy Efficiency. Asia Sus- tainable and Alternative Energy Program. Washington, DC: World Bank. World Bank. 2015. World Bank Group Support to Electricity Access, FY2000–2014. In- dependent Evaluation Group. Washington, DC: World Bank. World Bank. 2019a. “Armenia—Energy Efficiency Project.” Project Performance Assessment Report 135626, Independent Evaluation Group, World Bank, Wash- ington, DC. World Bank. 2019b. “Jordan—First and Second Programmatic Energy and Water Sector Reforms DPL.” Implementation Completion and Results Report Review ICRR0021535, Independent Evaluation Group, World Bank, Washington, DC. https://documents1.worldbank.org/curated/en/532471561641076061/pdf/Jor- dan-JO-Energy-Water-Sector-Reforms-DPL.pdf. World Bank. 2020. Renewable Energy: Evaluation of the World Bank Group Support for Electricity Supply from Renewable Energy Sources. Independent Evaluation Group. Washington, DC: World Bank. World Bank. 2021a. The Cold Road to Paris: Mapping Pathways toward Sustainable Cooling for Resilient People and Economies by 2050. Washington, DC: World Bank. World Bank. 2021b. World Bank Group Climate Change Action Plan 2021–25: Support- ing Green, Resilient, and Inclusive Development. Washington, DC: World Bank. World Bank Group Support to Demand-Side Energy Efficiency  Bibliography World Bank. n.d. “Corporate Action for Putting a Price on Carbon.” Background Note, World Bank, Washington, DC. https://www.worldbank.org/content/dam/World- bank/document/Climate/background-note_corporate-action-carbon-pricing. pdf. WRI (World Resources Institute) and WBCSD (World Business Council for Sus- tainable Development). 2011. Corporate Value Chain (Scope 3) Accounting and Reporting Standard: Supplement to the GHG Protocol Corporate Accounting and Reporting Standard. Washington, DC, and Geneva: WRI and WBCSD. Wu , Xiao, Rachel C. Nethery, M. Benjamin Sabath, Danielle Braun, and Francesca Dominici. 2020. “Air Pollution and COVID-19 Mortality in the United States: Strengths and Limitations of an Ecological Regression Analysis.” Science Ad- vances 6 (45): eabd4049. 82 Independent Evaluation Group World Bank Group    83 APPENDIX Independent Evaluation Group World Bank Group Support to Demand-Side Energy Efficiency Appendix A. Methodology Evaluation Questions The evaluation’s overarching question is, How well is the World Bank Group supporting client countries to scale demand-side energy efficiency (DSEE) to achieve development outcomes? Energy efficiency is defined as a reduction in the amount of energy required to maintain or improve energy services to households, businesses, and communities. Demand-side approaches focus on making the energy use of industries, commercial entities, and households more efficient, and supply-side approaches target the efficiency improve- ment in energy generation via grid infrastructure, utilities, and power pro- ducers. The three specific subquestions are the following: 1. How effective have the Bank Group’s DSEE interventions been in achiev- ing development outcomes related to (i) achieving end-use energy sav- ings, (ii) supporting market transformation, and (iii) attaining spillover World Bank Group Support to Demand-Side Energy Efficiency  Appendix A benefits (such as increased return on investments, greater integration with supply-side approaches, and improved service delivery)? 2. How coherent are the Bank Group DSEE interventions (i) internally (for example, coordination and joint initiatives across World Bank Practice Groups and International Finance Corporation [IFC] sectors) and (ii) externally (for example, across development partners and other energy efficiency actors)? 3. What untapped opportunities and mechanisms exist for the Bank Group to support clients to realize their energy efficiency potential? (i) What are the untapped opportunities for Bank Group engagement to support energy efficiency across sectors? (ii) What innovative mechanisms proved effective and sustainable and can be mainstreamed to scale DSEE interventions? Overview of Methodological Design This evaluation was conducted at three levels: global, country, and 86 intervention. It addressed the evaluation question through a combination of methodologies and techniques: literature review, content analysis using NVivo, portfolio sampling, multilateral development bank (MDB) benchmarking, key information interviews, country case studies, and econometric analysis. These methods are summarized in figure A.1 and further discussed in the following section.  apping Evaluation Questions with Evaluation Methodolo- Figure A.1. M gies and Techniques Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    87 Note: MDB = multilateral development bank. Bank Group effectiveness in achieving energy efficiency outcomes (evalua- tion question 1) was studied by multiple methods. First, causal models were developed to connect World Bank–IFC lending and nonlending interventions to the three outcomes: achieving end-use energy savings, supporting mar- ket transformation, and attaining and demonstrating multiple co-benefits of energy efficiency improvements. Portfolio review and analysis (PRA) was used to extract evidence (for example, from project completion reports, expanded supervision reports, and validation notes) on how World Bank–IFC interventions achieved key energy efficiency performance indicators. At the country and intervention levels, software-aided content analysis was applied to (i) country cases and (ii) intervention cases. The case-based analysis was based on semistructured interviews with staff, development partners, and stakeholders and desk reviews of portfolio data and gathered evidence on achievement of the development objectives. Software-aided content analysis of these data was used for some cases to discern patterns of causality across countries and across interventions. The empirical analysis at the intervention level used geospatial data to as- sess whether Bank Group support improved energy efficiency in two coun- tries, Malawi and Mexico. Projects in Malawi and Mexico were implemented in such a way that only some of the households in the project area benefited. The expected impact of these projects would differ between the households that benefited and those that did not. Using geospatial data to associate household location with energy-related outcomes, this analysis estimated the effectiveness of selected Bank Group projects in increasing energy effi- ciency. Data limitations (for example, World Bank geotags) were a constraint to extending the analysis beyond the two selected countries with high con- centration of DSEE approaches. The coherence of Bank Group approaches (evaluation question 2) was as- sessed using PRA, key informant interviews, literature review, and bench- World Bank Group Support to Demand-Side Energy Efficiency  Appendix A marking methods comparing corporate strategies (at the global level) to evaluations (at the country and intervention levels). At the global portfolio level, PRA and software-aided content analysis of project and country docu- ments were conducted to gather evidence on World Bank–IFC coordination on intervention design and implementation. At the country and interven- tion levels, review of (i) evaluation and validation documents and (ii) sem- istructured interviews with staff, development partners, and stakeholders was conducted to gather evidence on what worked and what did not, as well as the extent to which World Bank and IFC activities were consistent with corporate strategies and the activities of development partners in supporting energy efficiency improvements. Untapped opportunities and mechanisms for the Bank Group to help clients realize their energy efficiency potential (evaluation question 3) were as- sessed by comparing Bank Group work at the country and intervention levels with global innovations in energy efficiency. At the global portfolio level, literature review of (i) reports on global innovations, (ii) corporate strategy documents, (iii) regional and Practice Group strategy documents, and (iv) 88 staff interviews was conducted to assess untapped opportunities and where mechanisms exist to scale. At the country and intervention levels, review of (i) country case studies, (ii) evaluated and nonevaluated projects, and (iii) semistructured interviews with staff, development partners, and stakehold- ers was conducted to gather evidence on potential engagement gaps, con- straints to scaling energy efficiency initiatives, risks, and opportunities. Literature Review The structured literature review examined the current research in the field of energy efficiency on policy effectiveness, explored the factors that influence the adoption of DSEE technologies, and showed why some countries are able to foster energy efficiency successfully but others fail or are slower to do so. One of the goals of this literature review was to study the impact of incen- tives and information asymmetry on energy efficiency. Moreover, literature on country-specific factors that could influence a country’s readiness for DSEE uptake was also reviewed. These factors include government policies and reform agenda and intrinsic country characteristics, such as natural en- dowment, economic development, and so on. Portfolio Sampling Method In addition, a sampling approach was used to extract a representative sam- ple of energy efficiency operations (supply- and demand-side) to conduct Independent Evaluation Group World Bank Group    89 PRA and intervention-level study. The portfolio universe was sampled to be statistically representative using five criteria: Region, project status, income level, institution, and instrument. The minimum sample size was first cal- culated with a 90 percent confidence interval and 5 percent margin of error. Then, the stratified sampling was done using Neyman optimal allocation. Minimum sample size was selected to have a sample representative at the highest level and then increase the number of observations to be sampled within each subsegment in several iterations until the restrictions for all the segments were met. Finally, there was a forced inclusion of all World Bank demand-side and both-side projects. Multilateral Development Banks Benchmarking Method The benchmarking exercise identified industry benchmarks for the Bank Group in terms of energy efficiency. It compared seven MDBs: Asian Devel- opment Bank, African Development Bank, Asian Infrastructure Investment Bank, European Bank for Reconstruction and Development, Inter-American Development Bank, Islamic Development Bank, and the Bank Group. The seven MDBs were compared using five criteria: key priority, results frame- work, socioeconomic links, cross-sectoral approach, and communication strategy, based on policies, strategies, and directional documents of the energy sector. These criteria were selected based on a literature review of MDB strategies and the main drivers of MDB support in a particular sector or theme that allows for scale-up. The benchmarking exercise also took a deep dive into IFC’s Green Build- ings Market Transformation Program and explored it at three levels: global, country, and intervention. At the global level, it studied the gap between investment opportunities and IFC’s investment commitments. At the coun- World Bank Group Support to Demand-Side Energy Efficiency  Appendix A try and intervention levels, it reviewed how the Green Buildings Market Transformation Program had worked in Colombia, Indonesia, and South Af- rica. Development outcomes from applying Excellence in Design for Greater Efficiencies (EDGE) were examined at all three levels. Key Informant Interview Method The staff interviews were conducted to gather evidence on potential en- gagement gaps, constraints to scaling energy efficiency initiatives, risks, untapped opportunities, and where mechanisms exist to scale. On the IFC side, questions regarding these aspects were asked of IFC Climate Business, the Cities Initiative, advisory services, and IFC industry groups. Multilateral Investment Guarantee Agency (MIGA) strategy and industry groups were also interviewed. Country Case Study This evaluation also took a deep dive into six country case studies, which were selected using four indicators. One indicator is the level of Bank Group 90 intervention in the country; case studies with both high and low levels of Bank Group intervention were selected to allow for adequate variation in portfolio size and diversity. The second indicator is based on the Bank Group Regulatory Indicators for Sustainable Energy, which evaluate countries by their sustainable energy performance in three areas: electricity access, energy efficiency, and renewable energy; countries with both high and low Regulatory Indicators for Sustainable Energy scores were selected to allow for adequate variation in portfolio size and diversity. A third indicator is based on the level of industrialization in the country, as determined by the United Nations Competitive Industrial Performance Index1; countries with both high and low levels of industrialization were selected to allow for country coverage across income levels and geographies and concentration of DSEE portfolio. The final indicator is country income level; the case selection prioritized lower-middle- income and low-income countries because they are the countries with the most significant challenges in meeting Sustainable Development Goals 7, 8, and 13. Nevertheless, some upper-middle-income countries were included for comparison purposes. The application of the four filters resulted in the identification of the six country case studies: the Arab Republic of Egypt, Ghana, India, Indonesia, Rwanda, and Uzbekistan (see table A.1). Table A.1. Sampling Criteria in Country Case Studies World Bank RISE Energy Industrialization Client Coun- Independent Evaluation Group World Bank Group    91 and IFC Efficiency Score Score try Lending Countries Intervention (Avg., 5 years) (Avg., 5 years) Group Egypt, Arab High Low High LMIC Rep. Ghana Low Low Low LMIC India High High High UMIC Indonesia High Low Low UMIC Morocco Low High High LMIC Rwanda High Low Low LIC Uzbekistan Low High Low LMIC Source: Independent Evaluation Group. Note: Industrialization score is based on the United Nations Competitive Industrial Performance Index. Avg. = average; IFC = International Finance Corporation; LIC = low-income country; LMIC = lower-mid- dle-income country; RISE = Regulatory Indicators for Sustainable Energy; UMIC = upper-middle-income country. Spatial Econometric Analysis This spatial econometric analysis at the intervention level was part of the assessment of effectiveness. It used geospatial data to assess whether Bank Group support increased energy efficiency in Malawi and Mexico (see Overview of Methodological Design earlier in this appendix). Some energy projects may have been implemented in such a way that only some of the households in the project area benefited. The expected impact would differ between the households that benefited and those that did not, which allows the use of a spatial difference-in-differences (DiD) model to study the im- pacts of a large-scale development intervention aimed at improving energy efficiency in Malawi. One area of effectiveness assessment is to understand the interactions between supply-side energy efficiency (SSEE) and DSEE components within the same intervention and to test the effectiveness of DSEE approaches. DSEE outcomes are largely measured using energy savings use and emissions reduction, and SSEE outcomes can be measured using indicators such as improved access, fewer blackouts, and so on. Using a com- bination of remote-sensing (satellite) data and national household survey World Bank Group Support to Demand-Side Energy Efficiency  Appendix A data, this analysis estimated the impact of energy-efficient interventions on electricity access, blackouts, the choice of energy for lighting, and costs of electricity. Content Analysis Using NVivo This evaluation also used content analysis with NVivo to analyze the proj- ect development objectives (PDOs) and key development objectives or key development impact goals mentioned in approval documents during the Independent Evaluation Group PRA. In this content analysis, PDOs were imported into NVivo to find the frequency of energy efficiency–related keywords and the focus of energy efficiency project objectives to achieve (i) environmental outcomes, such as energy use savings and emissions reduc- tion, and (ii) socioeconomic outcomes, such as productivity increase, firm competitiveness, health improvements, and gender mainstreaming. 92 Description of Methods Literature Review Objective and Scope The goal of the structured literature review is twofold. First, reviewing relevant literature helps the evaluation team understand the current stage of research in the field of energy efficiency, especially the effectiveness of a variety of policies that were implemented to boost the adoption of energy- efficient technologies. Additionally, given that demand-side interventions have numerous and fragmented beneficiaries, it is important to understand why some communities or countries are able to incorporate energy-efficient technologies and others have failed or are slow to do so. Second, the review aims to answer two specific questions, which provide background for the team to use when answering evaluation questions. The two guiding questions for the review are as follows: 1. At the micro level, what are the role of incentives and information asym- metry in an individual’s and firm’s decision to use more energy-efficient technologies? 2. At the macro level, what are country-specific factors that influence a country’s readiness for DSEE uptake? (We paid close attention to govern- Independent Evaluation Group World Bank Group    93 ment policies and reform agendas and intrinsic country characteristics such as the level of economic development, culture, and geography.) In the Approach Paper titled World Bank Group Support to Energy Efficiency: An Independent Evaluation of Demand-Side Approaches, energy efficiency is defined as “a reduction in the amount of energy required to maintain or im- prove energy services to households, businesses, and communities” (World Bank 2021, 1). Although the scope of the evaluation and the literature re- view is limited to demand-side approaches to energy efficiency, some of the works that we reviewed also include supply-side approaches. Demand-side approaches focus on making the energy use of industries, commercial enti- ties, and households more efficient, and supply-side approaches target the efficiency improvement in energy generation via grid infrastructure, utilities, and power producers. A conventional wisdom in energy economics holds that the demand-side interventions face market failures—namely, information asymmetries, positive externalities, and capital market imperfections—that limit the scale of the interventions. Hence, this structured literature review aims to explore how scholars and researchers address these barriers and whether there are policy recommendations to overcome them. We are also interested in probing how country-specific characteristics affect demand-side interventions and whether there are patterns that we can generalize. Search Strategy Relevant literature was identified through (i) the EconLit publications data- base, (ii) the World Bank Open Knowledge Repository, and (iii) Google Schol- ar. We used EconLit and the Open Knowledge Repository as the main sources of academic literature and supplemented the searches with Google Scholar because the latter may capture papers that were missed in the former data- bases. We used a combination of inclusion and exclusion criteria to arrive at the World Bank Group Support to Demand-Side Energy Efficiency  Appendix A final list of literature. Scholarly works that are included in this review must have been published between 2010 and 2020, which is the period of the eval- uation. We also believe that limiting the search to the past 10 years ensures that we review the most up-to-date works in the field. Slightly different criteria were used on the three databases, as follows: » EconLit. We collected the number of citations for all the results and calculat- ed the third quartile if the number of the results is not large. We adjusted the quartile to second or first if the pool of the results was very small. In the final step, we reviewed the paper abstracts to further eliminate false positives. About 10 papers were included for each question. » Open Knowledge Repository. We collected the number of downloads for all the results and calculated the first quartile for the first review question because the result pool was small. We did not apply the download number criterion to the second question because the yielded results were very small (between two and four results). Instead, we reviewed all results. In the final step, we reviewed the paper abstracts to further eliminate false positives. We included nine pa- 94 pers for the first question and two papers for the second question. » Google Scholar. We reviewed the abstract of each result from the top of the page sorted by relevancy. If the abstract was relevant to the question, we included it in our list. We stopped the search when we had five papers. The rationale is that we used Google Scholar to supplement literature that EconLit did not capture. Based on our observation from running more than 10 different search strings on Google Scholar, we found that much of the most relevant literature was redundant to that found on EconLit. To strike a balance between relevancy and comprehensiveness, we decided to stop the search when we had five papers that were not present in our EconLit results. After applying these criteria to the intermediate result list, we ended up with a list of 62 scholarly works. We identified 23 papers for the first question and 39 papers for review to answer the second question. Extraction and Synthesis In our extraction process, we paid close attention to the findings of each scholarly work that could help answer the review questions. The collected literature was organized by questions. One caveat was that some works may answer more than one question, and each work may be mentioned more than once. Table A.2 is an example of our literature review findings: a breakdown of types of information barriers and their examples. Table A.2. Subcategories and Examples of Information Barriers Subcategory Example Source Independent Evaluation Group World Bank Group    95 Lack of information on » There is low information diffusion. Cagno et al. 2013 costs and benefits » There is a lack of proper regulation of classes of energy efficiency perfor- mance of particular technologies. » Suppliers of the technologies are not updated on the latest energy-effi- cient technologies. Unclear information » Energy-efficient technology suppliers Cagno et al. 2013 from technology sup- lack communication skills. pliers » There is a lack of proper regulation of classes of energy efficiency perfor- mance of particular technologies. (continued) Subcategory Example Source Trustworthiness of the » Energy-efficient technology suppliers Cagno et al. (2013) information source and have scarce communication skills to information issues on promote energy efficiency. energy contracts » Energy-efficient technology suppliers lack interest in providing clear and detailed information to their custom- ers. Information asymmetry » Energy-efficient technology suppli- Cagno et al. (2013) ers and consumers have access to different levels of information. High cost of informa- » Information about energy savings or Klemick et al. (2015) tion or cost to access access to such information is difficult information or costly. Sources: Cagno et al. 2013; Klemick et al. 2015. Portfolio Sampling Method To reduce the amount of work on PRA, a smaller but representative sample was extracted from the energy efficiency portfolio universe (supply- and de- World Bank Group Support to Demand-Side Energy Efficiency  Appendix A mand-side). The first step in this portfolio sampling process was to calculate the minimum number of projects in the sample (which is sample size, denot- ed by n). It was determined by using the following formula: In the formula, » N represents the total number of projects from World Bank and IFC. 2 » Z α represents the Z score directly tied to confidence interval α. Confidence level α refers to the percentage of different samples that would repeat the result of the portfolio. For example, if you created 10 separate samples and if 9 out of 10 times you get the same result, you have a 90 percent confidence level. The usual confidence level in research is 95 percent, which is what we used in this sampling process. 96 » e is the margin of error, which is the degree of error in results received from random sampling. A lower margin of error in statistics indicates more reli- able results. » p is the sample proportion, which is further discussed in the next paragraph. In our energy efficiency portfolio, the total number of projects from the World Bank and IFC is 615. We sampled our portfolio based on a 90 percent confidence interval and 5 percent margin of error. The estimator of p is , where x is the number of sampled projects. When projects are independent, this estimator has a (scaled) binomial distribution (and is also the sample mean of data from a Bernoulli distribution). The maximum variance of this distribution is , which occurs when the true parameter is p = 0.5. In practice, the maximum variance is often used for sample size assessments. Based on these inputs and random sampling, 130 projects were sampled to form a statistically representative sample at the portfolio level. However, because these projects were randomly sampled, the sample could be skewed and clustered in one Region or one instrument. Thus, the next step would be a stratified sampling based on five criteria. To further ensure our sample was statistically significant in each subgroup defined by Region, status, income level, instrument, and institution, a strat- ified sampling was done by using Neyman’s optimal allocation to calculate the size of each stratum. The formula is given below: Independent Evaluation Group World Bank Group    97 In the formula, » n h is the sample size for stratum h. » n is the total sample size (130 in our case). » N h is the population size for stratum h. » Sh is the standard deviation for stratum h. Once the sample size of each stratum was determined, we increased the number of observations to be sampled within each stratum in several itera- tions by random sampling until the required stratum size was met. Table A.3 summarizes the number of projects within each stratum of our final sample. Table A.3. Sample Breakdown Breakdown Sample (no.) Portfolio (no.) Share of Portfolio (%) Region       AFR 41 128 32 EAP 39 110 35 ECA 58 152 38 LAC 30 85 35 MENA 10 31 32 SAR 25 79 32 Other 11 30 37 Total 214 615 35 Status       Active 70 300 23 Closed 143 295 48 Dropped 0 2 0 Pipeline 0 2 0 n.a. 1 16 6 Total 214 615 35 County income category       World Bank Group Support to Demand-Side Energy Efficiency  Appendix A HIC 8 16 50 LIC 25 87 29 LMIC 82 245 33 UMIC 76 200 38 NA 23 67 34 Total 214 615 35 Instrument       IFC AS 27 124 22 IFC IS 42 178 24 World Bank ASA 52 60 87 World Bank DPF 34 65 52 World Bank IPF 59 182 32 World Bank P4R 0 6 0 Total 214 615 35 Institution       IFC 69 302 23 World Bank 145 313 46 Total 214 615 35 Source: Independent Evaluation Group. Note: AFR = Africa; AS = advisory services; ASA = advisory services and analytics; DPF = development 98 policy financing; EAP = East Asia and Pacific; ECA =Europe and Central Asia; HIC = high-income country; IFC = International Finance Corporation; IPF = investment project financing; IS = investment services; LAC = Latin America and the Caribbean; LIC = low-income country; LMIC = lower-middle-income country; MENA = Middle East and North Africa; n.a. = not applicable; P4R = Program-for-Results; SAR = South Asia; UMIC = upper-middle-income country. Table A.4 reflects the sample breakdown by institutions and instruments in the first column and the curated portfolio approach undertaken: the sample gen- erated using the formula from above was further streamlined to study in-depth projects with only DSEE and projects with both supply-side and demand-side components and to extract relevant trends and generalizable patterns.  isaggregated View of Energy Efficiency Projects (number of Table A.4. D projects) Universe Universe Sample Sample Instrument S/D/B D/B S/D/B D/B IFC IS 150 137 30 28 IFC AS 121 80 26 18 World Bank IPF 162 83 73 40 World Bank DPF 61 43 52 36 World Bank P4R 6 3 3 3 World Bank ASA 54 n.a. n.a. n.a. MIGA guarantee 8 8 8 8 Total 562 354 192 133 Source: Independent Evaluation Group. Note: AS = advisory services; ASA = advisory services and analytics; B = both; D = demand; DPF = devel- opment policy financing; IFC = International Finance Corporation; IPF = investment project financing; IS = investment services; MIGA = Multilateral Investment Guarantee Agency; n.a. = not applicable; P4R = Program-for-Results; S = supply. The findings of the evaluation report were largely derived from the analysis Independent Evaluation Group World Bank Group    99 of the portfolio of demand-side and both-side (supply and demand) opera- tions (see table A.5). There were 354 demand- and both-side operations, with 285 demand-side and 69 both-side operations. Operations with both supply- and demand-side interventions represented 20 percent of the portfolio.  isaggregated View of Active and Closed Projects, Excluding Table A.5. D Advisory Services and Analytics (number of projects) Universe S/D/B Universe D/B Type Active Closed Total Active Closed Total IFC IS 68 82 150 60 77 137 IFC AS 65 56 121 44 36 80 IPF 102 60 162 53 30 83 DPF 1 60 61 1 42 43 P4R 6 n.a. 6 3 n.a. 3 MIGA 5 3 8 5 3 8 Total 247 261 508 166 188 354 Source: Independent Evaluation Group. Note: AS = advisory services; B = both; D = demand; DPF = development policy financing; IFC = Interna- tional Finance Corporation; IPF = investment project financing; IS = investment services; MIGA = Multilat- eral Investment Guarantee Agency; n.a. = not applicable; P4R = Program-for-Results; S = supply. Multilateral Development Banks Benchmarking Method World Bank Group Support to Demand-Side Energy Efficiency  Appendix A This benchmarking analysis compared MDBs in terms of energy efficiency. Subjects for this analysis are the following seven MDBs: the Asian Develop- ment Bank, the African Development Bank, Asian Infrastructure Investment Bank, European Bank for Reconstruction and Development, Inter-American Development Bank, Islamic Development Bank, and the Bank Group. The seven MDBs were compared based on the five criteria of key priority, results framework, socioeconomic links, cross-sectoral approach, and communica- tion strategy, based on policies, strategies, and directional documents of the energy sector listed in table A.6, as well as supplementary information. » Key priority. This criterion is used to assess whether DSEE is explicitly priori- tized in policies and strategies developed by the MDBs. » Results framework. This criterion is used to assess whether results frame- works with indicators relevant to DSEE are set by the MDBs, in accordance with indicators that are globally recognized: Global Reporting Initiative, Impact Reporting and Investment Standards, and Sustainable Development Goals. 100 » Socioeconomic links. This criterion is used to assess whether links among energy efficiency and socioeconomic issues such as health and energy poverty (both access and quality) are recognized in results frameworks by the MDBs. » Cross-sectoral approach. This criterion is used to assess whether a cross-sectoral (policy-linked) approach is adopted by the MDBs at strate- gy level. Such an approach comprehensively responds to energy efficien- cy across various sectors, such as transport and manufacturing, including through policy-based lending, although a single-sector approach primarily focuses on the energy sector and resources allocated as a corporate priority. » Communication strategy. This criterion is used to assess whether a commu- nication strategy for DSEE targeting beneficiaries such as residential energy consumers and industrial firms is developed and mainstreamed within sector approaches and projects by the MDBs. Table A.7 shows the summary of the benchmarking analysis based on these five criteria. The analysis revealed, first, that all seven MDBs have prioritized DSEE or set indicators relevant to DSEE in their policies and strategies, al- though they have not recognized links between energy efficiency and socio- economic issues in the results frameworks. Second, the Bank Group has not adopted a cross-sectoral approach at the strategy level, although most of the MDBs respond to energy efficiency across various sectors. Finally, developing and mainstreaming a communication strategy for DSEE has been a challenge Independent Evaluation Group World Bank Group    101 for all seven MDBs.  olicies, Strategies, and Directional Documents of the Energy Table A.6. P Sector Institution Title Year Objective ADB Energy Policy 2009 » Provide reliable, adequate, and affordable energy for inclusive growth in a socially, economically, and environ- mentally sustainable way. Draft Energy Policy: Support- 2021 » Guide ADB’s energy sector ing Low Carbon Transition in (draft) operations to help DMCs Asia and the Pacific develop sustainable and resilient energy systems. (continued) Institution Title Year Objective AfDB Energy Sector Policy of the 2012 » Provide all of their popula- AfDB Group tions and productive sectors with access to modern, af- fordable, and reliable energy infrastructure and services. » Develop a socially, econom- ically, and environmentally sustainable energy sector. The Bank Group’s Strategy 2017 » Achieve universal access to for The New Deal on Energy electricity by 2025—100 per- for Africa 2016–2025 cent access in urban areas, 95 percent access in rural areas, and sufficient uninter- rupted energy supply to cov- er demand needs for those who are grid connected. AIIB Energy Sector Strategy: Sus- 2017 » Provide the framework, tainable Energy for Asia principles, and operational modalities to guide the AIIB’s energy sector engagement, including the development of its project pipeline and World Bank Group Support to Demand-Side Energy Efficiency  Appendix A future subsectoral lines of business. EBRD Energy Sector Strategy 2018 » Promote secure, affordable, 2019–23 and sustainable energy through the transition to a market-oriented low-carbon energy sector. IDB Energy Sector Framework 2018 » Increase LAC countries’ Document access to efficient, sustain- able, reliable, and afford- able energy in a diversified and secure manner, while reducing poverty, promoting improved quality of life, and fostering competitiveness and economic growth and development. IsDB Energy Sector Policy: Sus- 2018 » Sustainable energy for em- tainable Energy for Empow- powerment and prosperity. erment and Prosperity (continued) 102 Institution Title Year Objective World Bank Toward a Sustainable Energy 2013 » Secure the affordable, reli- Group Future for All: Directions able, and sustainable energy for the World Bank Group’s supply needed to end ex- Energy Sector treme poverty and promote shared prosperity. Sources: Independent Evaluation Group; ADB 2009, 2021; AfDB 2012, 2017; AIIB 2017; EBRD 2018; IDB 2018; IsDB 2018; World Bank 2013. Note: ADB = Asian Development Bank; AfDB = African Development Bank; AIIB = Asian Infrastructure Investment Bank; DMC = developing member country; EBRD = European Bank for Reconstruction and Development; IDB = Inter-American Development Bank; IsDB = Islamic Development Bank; LAC = Latin America and the Caribbean. Table A.7. Summary of Benchmarking Analysis World Bank Criterion ADB AfDB AIIB EBRD IDB IsDB Group Key priority Y Y Y Y Y — Y Results framework Y Y Y — — Y Y Socioeconomic — — — — — — — links Cross-sectoral Y Y Y Y Y — — approach Communication — — — — — — — strategy Independent Evaluation Group World Bank Group    103 Source: Independent Evaluation Group. Note: Analysis is based on desk reviews of annual reports, MDB energy sector strategies, select country program documents of MDBs, and key informant interviews. — = no; ADB = Asian Development Bank; AfDB = African Development Bank; AIIB = Asian Infrastructure Investment Bank; EBRD = European Bank for Reconstruction and Development; IDB = Inter-American Development Bank; IsDB = Islamic Develop- ment Bank; MDB = multilateral development bank; Y = yes. Key Informant Interviews Key informant interviews were also used to gather evidence on potential engagement gaps, constraints to scaling energy efficiency initiatives, risks, untapped opportunities, and where mechanisms exist to scale. The questions are summarized in this section. Key Informant Interviews with International Finance Corporation Managers and Staff Outline of Questions Climate Business Department and Industry-Climate Focal Points 1. IFC is targeting 85 percent of Board of Executive Directors–approved real sector operations to be aligned with the Paris Agreement on July 1, 2023, and 100 percent on July 1, 2025; this means that 100 percent of projects will be aligned at the concept stage very soon. To what extent is DSEE included in this plan, and in which sectors or cross-sectoral areas do you World Bank Group Support to Demand-Side Energy Efficiency  Appendix A plan to increase focus? [For follow-up questions: planned contributions are to (i) agriculture, food, water, land; (ii) cities; (iii) transport; and (iv) manufacturing.] 2. Within mitigation, across sectors, the current focus appears to be on re- newable energy support and utility support and SSEE. How do you envis- age DSEE playing a more prominent role in the future? Also, how does the recent emphasis on increasing climate change adaptation financing affect DSEE inclusion in operations? Is DSEE receiving even less attention? 3. Do you have any vision of creating an incentive program for the existing IFC client base to upgrade on DSEE considering the Bank Group climate objectives overall and the DSEE objectives specifically? How can the Bank Group Board of Executive Directors (Board) support this vision? 4. Are there any challenges to IFC’s work on green bond and loan markets, transition bonds in the energy sector, and green bonds in manufacturing and agriculture? What would reduce the challenges? How can the Board support you? 104 5. To what extent is IFC planning to work on carbon markets and emission trading schemes or support through existing capital markets initiatives? Can such efforts be part of DSEE initiatives? To what extent do collabora- tion mechanisms exist within the Bank Group to facilitate such efforts? 6. If there is no client demand in certain aspects of DSEE, how can IFC help stimulate this demand (for example, upstream, capital markets, venture capital)? 7. To what extent can lower-middle-income and low-income countries em- brace a decarbonization agenda at scale? What are the barriers, and what would reduce such barriers? Cities Initiative, State-Owned Enterprises, and Advisory Services 8. Within the Cities Initiative (which is estimated to have the largest cli- mate investment opportunity for IFC), what are the main constraints to expanding green buildings, EDGE, or both to broader climate-smart urban planning? What would reduce these constraints? What are the potential downsides to green buildings (for example, greenwashing, payment de- faults, reratings)? [For follow-up questions: green mortgages, green loans, green bonds, Breathe Better Bonds.] 9. To what extent do you think sustainability-linked loans can help expand Independent Evaluation Group World Bank Group    105 the green buildings footprint? How can advisory programs build on exist- ing EDGE clients? 10. Does the new green cities tool, Advanced Practices for Environmental Excellence in Cities, include DSEE measures? 11. To what extent do collaboration mechanisms exist to promote green buildings for the public sector and expand to state-owned enterprises and municipal infrastructure (World Bank clients)? Infrastructure and Natural Resources; Manufacturing, Agribusiness and Ser- vices; and Financial Institutions Group; 12. In the transport sector, is there a specific plan to invest in energy-efficient equipment and infrastructure (in ports and airports)? Are there any chal- lenges in this area, and what would reduce them? 13. In manufacturing, is there a specific plan to expand investment in green- house gas abatement measures and in related innovative technologies? In which manufacturing sectors? Are there any challenges in this area, and what would reduce them? 14. In water supply and sanitation, is there a plan to promote energy and water efficiency (for example, through nonrevenue water reduction, water source management, operations optimization [including through digita- lization], and wastewater collection and treatment infrastructure)? Are there any challenges in this area, and what would reduce them? [For follow-up questions: IFC’s Utilities for Climate Initiative; Latin America and the Caribbean.] 15. Within the IFC venture capital portfolio, is there room to promote DSEE, including through innovative approaches, specifically, within IFC’s Startup World Bank Group Support to Demand-Side Energy Efficiency  Appendix A Catalyst or TechEmerge initiatives? [For follow-up questions: IFC’s venture capital special initiatives: (i) IFC’s Startup Catalyst and (ii) TechEmerge.] Key Informant Interviews with Multilateral Investment Guarantee Agency Strategy and Industry Groups Questions 1. The MIGA is targeting 85 percent of Board-approved real sector operations to be aligned with the Paris Agreement on July 1, 2023, and 100 percent on July 1, 2025; this means that 100 percent of projects will be aligned at the concept stage very soon. How does MIGA plan to support this transition through private sector–focused instruments? Public sector–focused instruments? To what extent is DSEE included in the MIGA strategy, and in which sectors? [For follow-up questions: Most MIGA guarantees are in the banking sector, 106 and DSEE projects are mostly in banking and infrastructure and services sectors. Other MIGA sectors: agribusiness, capital markets, chemicals, construction, education, extractives, financial markets, financial services, infrastructure, leasing, manufacturing, mining, oil and gas, power, renewable energy, services, solid waste, telecommunications, tourism, transport, water supply, and sanitation.] 2. Within mitigation, across sectors, the current focus in operations is on renewable energy and SSEE. To what extent will DSEE play a more prom- inent role in the future? What are MIGA’s limitations to expanding on DSEE cross-sectorally or through portfolio or cluster guarantees? In MIGA, the strategy emphasizes support to clients in MIGA’s largest sector—bank- ing—in phasing out coal and promoting renewable energy, but DSEE does not seem to be mentioned. Also, how does the recent emphasis on increas- ing climate change adaptation financing affect DSEE inclusion in opera- tions—is DSEE receiving even less attention? 3. Within its urban investments, is MIGA planning to expand beyond green buildings, EDGE, or both? Are there any challenges in this area, and what would reduce them? 4. In the transport sector, is there a specific plan to invest in energy-efficient equipment and infrastructure (in ports and airports)? Are there any chal- lenges in this area, and what would reduce them? 5. In manufacturing, is there a specific plan to expand investment in green- Independent Evaluation Group World Bank Group    107 house gas abatement measures and in related innovative technologies? In which manufacturing sectors? Are there any challenges in this area, and what would reduce them? 6. In water supply and sanitation, is there a specific plan to promote energy and water efficiency (for example, through nonrevenue water reduction, water source management, operations optimization [including through digitalization], and wastewater collection and treatment infrastructure)? Are there any challenges in this area, and what would reduce them? 7. Is MIGA considering working on instruments to support green bond and loan markets, transition bonds in the energy sector, or green bonds in manufacturing and agriculture? To what extent can MIGA develop new products or instruments to promote carbon markets or emissions trading programs? Informant Breakdown Table A.8 presents the number of informants by type and country. Table A.8. Informant Breakdown Informant Egypt, Type Arab Rep. Ghana India Indonesia Rwanda Uzbekistan Country staff 1 4 0 6 0 0 Government 0 7 0 3 9 2 officials Private sector 0 0 1 4 0 6 clients Headquarters 5 0 10 0 2 0 staff Think tanks, 3 0 0 2 0 4 CSOs Academics 0 0 0 2 0 0 World Bank Group Support to Demand-Side Energy Efficiency  Appendix A Donor partners 0 3 0 1 0 0 Total 9 14 11 18 11 12 Source: Independent Evaluation Group staff calculation. Note: CSO = civil society organization. Country Case Study Country Case Study Selection Reliable energy efficiency indicators can guide us to look at an evidence- based demonstration of success or failure of energy efficiency measures around the world. Countries with proven track records of success using energy efficiency reforms can be good cases to deep dive into, and they can be compared with cases showing little evidence of success, especially in emerging market economies. The availability of comparable data on outcome indicators for both groups of case study countries will be crucial to the selection process. 108 For the sake of evidence-based learning, it will be important to isolate proj- ects specifically targeting energy efficiency because many sector reforms have been structured in relation to access, capacity, energy security, and efficiency, often making it difficult to measure energy savings. Salient Points » Time series data of energy savings to identify inflection points in the course of evolution of energy efficiency interventions » Regulatory landscape analysis » Regional integration aspect: European Union, regional energy market being developed in Middle East and North Africa » State-owned enterprises, corporatization of energy efficiency institutions, enhancing bankability of energy service companies Scope of the Country Case Study Each country case was examined through eight dimensions: market-related factors; industrialization; institutional and regulatory quality; infrastructure development; financial development; private capital flows; energy efficiency indicators; and environmental, social, and governance indicators. Detailed data and sources are shown in table A.9. Independent Evaluation Group World Bank Group    109 Table A.9. Domestic Environment Indicators Dimension Indicator Description I. Market- Growth potential GDP per capita growth (annual %) related (time series—evalua- Source: WDI factors tion timeline) (continued) Dimension Indicator Description II. Industrial- II.a Competitive industrial Denotes ability of countries to produce ization performance score and export manufactured goods com- (time series—evalua- petitively tion timeline) Source: UNIDO CIP Index II.b Industrialization Indicator of the complexity of produc- Intensity Index tion processes Source: UNIDO CIP Index II.c Emission intensity Inverse of emission efficiency (On (CO2 emitted per unit average, countries with the lowest of manufacturing levels of industrialization and the least value added) competitive industries are those with the highest emission intensities.) Source: UNIDO CIP Index II.d Impact of the Share of the economy in global manu- economy on world facturing value added manufacturing value Source: UNIDO CIP Index added III. Institu- III.a National energy Indicator of the country’s performance tional and efficiency planning in energy efficiency legislation and regulatory score (time se- national and sectoral targets quality ries—2010–17) Source: ESMAP energy efficiency indicators World Bank Group Support to Demand-Side Energy Efficiency  Appendix A III.b Energy efficiency Indicator of a country’s public and entities score (time private energy efficiency institutions, series—2010–17) evaluation of energy efficiency pro- grams based on public consultation, and certification or accreditation pro- grams mandated for energy efficiency activities Source: ESMAP energy efficiency indicators III.c Consumer informa- Indicator of a country’s access to infor- tion on electricity mation of electricity usage for residen- usage score (time tial, commercial, and industrial users, series—2010–17) and the quality of available information Source: ESMAP energy efficiency indicators III.d Energy efficiency Indicator of a country’s electricity rate incentives from structure, demand charges (large cus- electricity rate struc- tomers), and time-of-use tariffs tures score (time Source: ESMAP energy efficiency series—2010–17) indicators III.e Incentives and Indicator of a country’s energy efficien- mandates: industrial cy incentives and mandates for large and commercial end consumers, SMEs users score (time Source: ESMAP energy efficiency 110 series—2010–17) indicators (continued) Dimension Indicator Description III.f Financing mech- Indicator of a country’s financing anisms for energy mechanisms available in each sec- efficiency (time tor, including but not limited to tax series—2010–17) incentives, credit lines, green energy efficiency bonds, and so on Source: ESMAP energy efficiency Indicators III.g Minimum energy ef- Indicator of a country’s energy effi- ficiency performance ciency performance standards and standards score (time enforcement mechanisms series—2010–17) Source: ESMAP energy efficiency Indicators III.h Carbon pricing and Indicator of a country’s carbon pricing monitoring (time mechanisms such as carbon tax, emis- series—2010–17) sions trading schemes, and monitoring and verification of these mechanisms Source: ESMAP energy efficiency indicators III.i Overall RISE score A snapshot of a country’s policies and regulations in the energy sector, organized by the three pillars of the SEforALL initiative: energy access, energy efficiency, and renewable energy Source: ESMAP energy efficiency indicators IV. Infrastruc- Logistics Logistics Performance Index, overall ture develop- score ment Independent Evaluation Group World Bank Group    111 V. Financial Financial depth Liquid liabilities (% of GDP) development VI. Private Foreign direct invest- Foreign direct investment, net inflows capital flows ment (% of GDP) VII. Energy VII.a National energy MJ/unit of national output efficiency intensity Source: Tracking SDG indicators indicators VII.b CO2 emissions by Source: IEA source (continued) Dimension Indicator Description VIII. ESG VIII.a Access to clean fuels Source: WDI indicators and technology for cooking (% of popu- lation) VIII.b Access to electricity Source: WDI (% of population) VIII.c CO2 emissions (tons Source: WDI per capita) VIII.d Electricity production Source: WDI from coal sources (% of total) VIII.e Energy intensity level Source: WDI of primary energy (MJ/US$2011 PPP GDP) VIII.f Government effec- Source: World Bank ESG Data Portal tiveness: estimate Source: Independent Evaluation Group. Note: CO2 = carbon dioxide; CIP = Competitive Industrial Performance Index; ESG = environmental, social, and governance; ESMAP = Energy Sector Management Assistance Program; GDP = gross domestic World Bank Group Support to Demand-Side Energy Efficiency  Appendix A product; IEA = International Energy Agency; MJ = megajoule; PPP = purchasing power parity; RISE = Reg- ulatory Indicators for Sustainable Energy; SDG = Sustainable Development Goal; SEforALL = Sustainable Energy for All; SMEs = small and medium enterprises; UNIDO = United Nations Industrial Development Organization; WDI = World Development Indicators. Furthermore, an analytical framework was developed to facilitate cross-country analysis and develop (i) generalizable findings and (ii) contex- tual findings. Framework criteria. The framework criteria included (i) government interest in DSEE action, (ii) the current status of DSEE reform, and (iii) incentives (red text) and disincentives to implement DSEE (description of incentives and disincentives is given in parentheses in figure A.2). 112  ramework Matrix with Anecdotal Examples from Country Figure A.2. F Case Studies Source: Independent Evaluation Group. Note: DSEE = demand-side energy efficiency; O&M = operation and maintenance; SOE = state-owned enterprise. Spatial Econometric Analysis Spatial DiD analysis is used to study the impacts of a large-scale develop- ment intervention aimed at improving energy efficiency in Malawi. The estimation strategy takes advantage of the geographical variation in the im- plementation of different project components and is based on a combination of remote-sensing (satellite) data and national household survey data. Independent Evaluation Group World Bank Group    113 The strategy for estimating the impacts of the energy efficiency intervention on the considered outcome variables relies on the geographical variation in the implementation of different project components. Ideally, we would like to compare the outcomes across two identical groups of households that differ only with respect to whether or not they benefited from the project. Because it is impossible to observe the same household (at the same point in time) both with and without a treatment, counterfactual analysis seeks to identify a suitable comparison (the “control group”) for the beneficiaries (“treatment group”). In our setting, this identification is based on the geo- graphical locations and associated benefits of project components, which were distributed across different areas of the country, benefiting primarily those households living in these areas. Specifically, the DiD approach esti- mates the impact of a project by comparing the difference between before and after the start of the project for households that were able to benefit from the project and households that were unable (or less able) to benefit. The regression equation of the household-level DiD estimator in Malawi can be written as follows: = 0 + 1 + 2 + 3( × ) + + (1) where is the outcome of household in district at time . is a dum- my time variable that equals 1 for the postprogram year (2019) and 0 for the preprogram year (2013). is a dummy treatment variable that equals 1 for households residing in districts with project components and 0 other- wise. The variables and are interacted to estimate the coeffi- cient 3, which is the main coefficient of interest. is a vector of control variables that are included to account for potential imbalances in character- istics between treated and untreated households that might be correlated with the outcome. Some specifications also include district fixed effects. The estimated coefficients from the model in equation (1) can be interpret- ed in the following way. The coefficient 1 captures the change in outcome World Bank Group Support to Demand-Side Energy Efficiency  Appendix A between the pre- and postprogram years for households residing in areas without project components (the control group). The coefficient 2 captures the difference between households benefiting from the project and house- holds in the control group before the start of the project (preprogram differ- ence). The coefficient of interest is 3, which captures the difference in the change in for the benefiting households and those in the control group. If the assumptions underlying the DiD approach (see the next paragraph) hold, then 3 can be interpreted as the change in that the households in the treatment group experienced because they benefited from the project—that is, the project’s impact on over the considered time period. The DiD strategy relies on the assumption that any preexisting difference between the households benefiting from the project and those forming the control group would be constant over time in the absence of the project (parallel trends assumption). One of the advantages of DiD is that it can account for systematic differences between the treatment and control group that prevailed before the start of the intervention. The existence of such differences is to be expected in the context we study because the allocation 114 of project locations was not determined randomly. The estimation of the program effect 3 in equation (1) will be robust as long as these differences are constant over time in the absence of the project (that is, in the absence of the project, the households residing in project areas would have featured the same time trend as the control group). There is no statistical test to verify (or reject) this assumption. However, a notion in the literature is to use data on the time before the start of the intervention to visually verify whether trends appear to be parallel. Since the Living Standards Measurement Study data are available for Malawi for multiple years (2010, 2013, 2016, and 2019), we can follow this approach. In addition, it is possible that the parallel trends assumption holds only after conditioning on relevant covariates. Because our data allow us to control for a rich set of household characteristics, it will be sufficient for our identification strategy if trends are parallel when condi- tioning on these covariates. In addition to estimating equation (1) based on household survey data for two years (pre- and postprogram implementation), we use grid-level data from satellites (that are available annually for 2012–20) to estimate a more general spatial DiD specification of the form: = 0 + ∑ + ∑ + 3( × ) + (2) where is the nighttime radiance in grid cell at time , is a set of dummy variables for each year (that is, year fixed effects), is a set of dummy variables for each cell (that is, grid cell fixed effects), and is a treatment indicator that equals 1 for grid cells corresponding to proj- Independent Evaluation Group World Bank Group    115 ect areas and 0 otherwise. is defined analogously to equation (1) as a dummy time variable that equals 1 for the years after the start of project implementation (2015 in Malawi) and 0 for all earlier years. The variables and are interacted to estimate the coefficient 3, which is the main coefficient of interest capturing the impact of the project on nighttime radiance at the grid level. Two alternative treatment indicators are considered in the model in equation (2) to capture potential spillover effects of project components on the areas surrounding each project location. Many of the (supply-side) project com- ponents may have effects on wider areas around the subdistrict (city) where the component was located (for example, upgrading a power plant or part of a distribution network in a particular city may also affect the quality of electricity supply in the surrounding area outside the city). The most disag- gregated level of information we have about project locations is the sub- district level (administrative level 2). To capture potential spillover effects, we consider two alternative approaches to defining the treatment indicator in the model in equation (2). In the first approach, only those grid cells are coded as 1 for the treatment indicator that corresponds to project areas at the most disaggregated level of information about project locations (administra- tive level 2). We refer to this treatment indicator as “Treat (Admin. 2).” In the second approach, denoted “Treat (Admin. 1),” those grid cells are also coded as 1 for the treatment indicator that falls into the next higher level of ad- ministrative division (administrative level 1, that is, districts) of each project location. For example, if a project component was implemented in city A of district D1, then Treat (Admin. 2) will equal 1 for the grid cells belonging to city A (and 0 otherwise), whereas Treat (Admin. 1) will equal 1 for all grid cells belonging to district D1 (including the cells belonging to city A). Content Analysis Using NVivo World Bank Group Support to Demand-Side Energy Efficiency  Appendix A Content analysis is the study of documents and communication artifacts, which might be texts of various formats, pictures, audio, or video. The PRA team collected project-level data by manually extracting certain characteristics from project documents on a sample of projects from the portfolio. One piece of information that the team collected is PDOs, which are rich in textual content. We collected 168 entries of PDOs, key develop- ment objectives, and key development impact goals from projects across Bank Group, IFC, and MIGA. This information helps the evaluation team understand the broad goals of energy efficiency–related projects. The PRA team used NVivo, a software known for text analytics functionality, to compute and rank the frequency of each word that appears in the collect- ed PDOs and equivalent statements in approval documents. Most common words in English, such as a, an, and on, are called “stop words” and are ex- cluded from the analysis. This is because these words are so ubiquitous that they will rank highly in the frequency list, but they do not provide useful in- sights on the objectives of these projects. Once word frequency of all unique words in the objectives was computed, NVivo generated a word cloud (shown 116 in figure A.3). The size and color of each word in the word cloud shows its frequency. The words in orange are those that appear most frequently in the PDO entries. It is not surprising that words such as energy and efficiency appear in the word cloud. However, we also see that PDOs often use words such as electrici- ty, power, health, climate, sustainable, emissions, and environmental. This could imply that energy efficiency projects primarily aim for environmental bene- fits such as greenhouse gas emission reduction or sustainable development. Figure A.3. Project Development Objectives Word Cloud Source: Independent Evaluation Group. Independent Evaluation Group World Bank Group    117 References ADB (Asian Development Bank). 2009. “Energy Policy.” Policy Paper, Asian Develop- ment Bank, Mandaluyong City. https://www.adb.org/sites/default/files/institu- tional-document/32032/energy-policy-2009.pdf. ADB (Asian Development Bank). 2021. “Energy Policy: Supporting Low Carbon Tran- sition in Asia and the Pacific.” Draft Paper for Consultation, Asian Development Bank, Mandaluyong City. https://www.adb.org/sites/default/files/institution- al-document/699206/energy-policy-draft-consultation.pdf. AfDB (African Development Bank). 2012. Energy Sector Policy of the AfDB Group. Abidjan: AfDB. https://www.afdb.org/fileadmin/uploads/afdb/Documents/Poli- cy-Documents/Energy_Sector_Policy_of_the_AfDB_Group.pdf. AfDB (African Development Bank). 2017. The Bank Group’s Strategy for The New Deal on Energy for Africa 2016–2025. Abidjan: AfDB. https://www.afdb.org/fileadmin/ uploads/afdb/Documents/Generic-Documents/Bank_s_strategy_for_New_ Energy_on_Energy_for_Africa_EN.pdf. AIIB (Asian Infrastructure Investment Bank). 2017. Energy Sector Strategy: Sustain- able Energy for Asia. Beijing: AIIB. https://www.aiib.org/en/policies-strategies/ strategies/sustainable-energy-asia/.content/index/_download/energy-sec- tor-strategy.pdf. Cagno, Enrico, Ernst Worrell, Andrea Trianni, and Giacomo Pugliese. 2013. “A Novel Approach for Barriers to Industrial Energy Efficiency.” Renewable and Sustain- able Energy Reviews 19 (March): 290–308. EBRD (European Bank for Reconstruction and Development). 2018. Energy Sector Strategy 2019–23. London: EBRD. https://www.ebrd.com/power-and-energy/ World Bank Group Support to Demand-Side Energy Efficiency  Appendix A ebrd-energy-sector-strategy.pdf. IDB (Inter-American Development Bank). 2018. Energy Sector Framework Docu- ment. New York: IDB. https://idbdocs.iadb.org/wsdocs/getdocument.aspx- ?docnum=EZSHARE-715548541-11. IsDB (Islamic Development Bank). 2018. Energy Sector Policy: Sustainable Energy for Empowerment and Prosperity. Jeddah: IsDB. https://www.isdb.org/sites/default/files/ media/documents/2019-04/IsDB_Energy%20Sector%20Policy.pdf. World Bank. 2013. Toward a Sustainable Energy Future for All: Directions for the World Bank Group’s Energy Sector. Washington, DC: World Bank. https://documents1. worldbank.org/curated/en/745601468160524040/pdf/Toward-a-sustainable-en- ergy-future-for-all-directions-for-the-World-Bank-Groups-energy-sector.pdf. World Bank. 2021. World Bank Group Support to Energy Efficiency: An Independent Evaluation of Demand-Side Approaches. Approach Paper. Independent Evalua- tion Group. Washington, DC: World Bank. 118 1  United Nations Industrial Development Organization’s Competitive Industrial Performance Index 2020: Country Profiles. 2  All Multilateral Investment Guarantee Agency demand-side energy efficiency projects were included in our portfolio universe but not sampled here. Independent Evaluation Group World Bank Group    119 The World Bank 1818 H Street NW Washington, DC 20433