Acknowledgements This report was prepared by Christina Wieser (Senior Economist, Poverty and Equity Global Practice, World Bank Group); Simon Franklin (Senior Lecturer, School of Economics at Queen Mary University of London); and Wondimagegn Mesfin Tesfaye (Economist, Poverty and Equity Global Practice, World Bank Group). The survey for this study was designed by Simon Franklin and Christina Wieser. The data collection was conducted by Frontieri Consulti PLC. The authors are grateful to Suleiman Namara (Lead Economist, Social Protection and Jobs Global Practice, World Bank Group) and Ayuba Sani (Senior Social Protection Specialist, Social Protection and Jobs Global Practice, World Bank Group) on framing the report and providing invaluable input, and the Urban Productive Safety Net Project team (Koen Maaskant, Alfredo Bohm, Wout Soer, Roman Tesfaye, and Blene Aklilu) for their feedback and support throughout the process. The team would also like to thank the Government of Ethiopia for implementing the Urban Productive Safety Net Project (UPSNP), in particular Birhanu Teshome (Head of Urban Food Security and Safety Net Office, Ministry of Urban Development and Infrastructure). The team also benefited from guidance from the Project Coordination Unit (PCU) under the leadership of Mekonnen Yaie (Project Coordinator, UPSNP PCU) and with support and contributions from Tewodros Abebe (Livelihoods Specialist, UPSNP PCU). Disclaimer This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Caption UPSNP beneficiary engaging in her sheep breeding business, which she set up as her UPSNP livelihoods activity in Adama, Ethiopia. © Christina Wieser/World Bank Table of Contents Abstract ................................................................................................................................................... 4 1. Background ..................................................................................................................................... 1 2. Program design and implementation ............................................................................................. 3 3. Data and Methods .......................................................................................................................... 6 4. Livelihoods participation and success ............................................................................................. 7 Livelihood training .............................................................................................................................. 7 Livelihood modality ............................................................................................................................. 8 Self-employment pathway .................................................................................................................. 8 Wage employment pathway ............................................................................................................. 14 Satisfaction with the livelihoods program ........................................................................................ 16 5. Empirical Strategy ......................................................................................................................... 17 6. Impact estimates........................................................................................................................... 20 Household enterprises ...................................................................................................................... 21 Household earnings and income ...................................................................................................... 22 Employment effects .......................................................................................................................... 24 Consumption and assets ................................................................................................................... 24 7. Conclusion ..................................................................................................................................... 25 References ............................................................................................................................................ 27 Annex .................................................................................................................................................... 28 Abstract Improving the livelihoods of poor households and transitioning them to the labor force is a major challenge in Ethiopia. Productive safety nets that include livelihood grants and other support services could be a promising intervention toward this end. However, the evidence on the welfare and labor market impacts of such interventions is limited. This study provides empirical evidence on the development impacts of livelihood grants and support with a focus on welfare and labor market outcomes. The data for this study come from baseline and livelihoods endline household surveys undertaken between 2016 and 2022. The results show that self-employment (compared to wage employment) is the preferred livelihood modality selected by PW beneficiaries. Yet, few UPSNP beneficiaries are successful in establishing or expanding self-employment opportunities or engaging in wage employment. This is primarily because beneficiaries tend to utilize the livelihoods grant to cover regular and emergency household expenditures, especially during challenging economic times such as during the COVID-19 pandemic. The impact evaluation results show that livelihood grants and services have a positive impact on household business ownership but impacts on earnings, consumption, and assets are insignificant. These findings have important implications for the design of future similar interventions. Keywords: safety nets, livelihood grants, livelihood services, impact evaluation, Ethiopia 1. Background Ethiopia was one of the world’s fastest-growing economies pre-pandemic and pre-conflict, making important strides in poverty reduction, but significant progress is still to be made. Gross domestic product (GDP) grew at an average annual rate of approximately 10 percent between 2007 and 2019 before taking a downturn during COVID-19. Unlike many developing economies, growth in Ethiopia has largely been pro-poor and has led to a reduction in poverty in both urban and rural areas (Franklin, Tefera, & Getahun, 2017). The most recent poverty assessment reports by the World Bank, for example, indicate that poverty in Ethiopia had declined from 44 percent in 2000 to 30 percent in 2011 (World Bank, 2015) and further to 24 percent in 2016 (World Bank, 2020). Human development outcomes also improved alongside poverty reduction, albeit from a low base. Despite the progress made, important challenges remain, and Ethiopia continues to rank among the world’s poorest countries. While poverty reduction was much stronger in urban areas (11 percentage points) versus rural ones (4 percentage points) of Ethiopia between 2011 and 2016 (World Bank, 2020), the decline in poverty falls short of making a significant dent in urban poverty owing to wage rigidities, inflation, and labor market frictions. Urban poverty remains high at 12 percent (World Bank, 2020) and there are signs of a deterioration in poverty in urban areas following the COVID-19 pandemic (Wieser et al., forthcoming). There is a growing understanding that with increasing urbanization, the problem of urban poverty becomes more salient and will require the right policy framework to translate gains from economic growth to poverty reduction (Franklin et al., 2017). Given persistent poverty and challenges in the labor market, social protection is a key part of a policy framework for the Government of Ethiopia (GoE) to focus on reducing poverty, social and economic risk, vulnerability, and exclusion. Poverty reduction, which was endorsed in the Millennium Development Goals (MDGs) and later strengthened in the Sustainable Development Goals (SGDs), is one of the key global goals. Reducing poverty in all its dimensions (SDG 1- eradicating poverty in its dimensions, SDG 2-no hunger) by 2030 is central to meeting the SDGs and achieving sustainable socio- economic transformation. In Ethiopia, the SDGs adopt a system approach that depends critically on comprehensive and integrated economic and social protection policies. The National Social Protection Policy comprises five focus areas: safety nets, tailored livelihoods support, social security, increased access to basic services by vulnerable groups, and legal protection for those who are vulnerable to abuse and violence (Ministry of Labour and Social Affairs, 2012). Ethiopia has social protection programs that have played a key role in poverty reduction in recent years. It is home to one of the largest and oldest rural safety net programs in Africa, the Productive Safety Net Program (PSNP), as well as a regular recipient of Humanitarian Food Aid (HFA). Together, both programs are Ethiopia’s main social protection programs in terms of spending. While the PSNP addresses chronic food insecurity in rural areas, HFA responds to acute food insecurity as a result of severe shocks, mainly of a climatic nature. Evidence documents the significant impacts of PSNP on poverty reduction (Endale, Pick, & Woldehanna, 2019; Gilligan, Hoddinott, & Taffesse, 2009; Hill, Inchauste, Lustig, & Tsehaye, 2015; Hirvonen, Mascagni, & Roelen, 2018; World Bank, 2020), reflecting development success of social protection in the county. According to the World Bank’s most recent poverty assessment report, the positive benefits of the PSNP on poverty reduction are attributed to its targeting performance and coverage (World Bank, 2020). PSNP is overall well targeted, with beneficiaries more likely to be poor and experiencing food shortages, owning fewer assets, and living 1 in more remote and drier places, the HFA is reasonably well targeted but with inclusion errors (World Bank, 2020). Given the success of social protection programs in the country, Ethiopia has recently embarked upon expanding safety nets to cover the urban poor. Social protection policies can help address the multifaceted nature that helps to enhance inclusive growth and development (Franklin et al., 2017). It is with the recognition of urban poverty that the UPSNP as a first social protection response for urban areas was designed to enhance inclusive growth and development in selected urban areas. The government’s urban safety net strategy aimed to reduce poverty and vulnerability among the urban poor living below the poverty line for 10 years (Franklin et al., 2017). In 2016, the Urban Productive Safety Net Project (UPSNP) was launched in 11 cities of the country (one each from nine regional states and the two chartered city administrations of Addis Ababa and Dire Dawa). The main activities of the UPSNP include labor-intensive public work (PW) opportunities combined with access to life skills, financial literacy, and business skills training and a livelihood grant. Beneficiary households without any able-bodied members who cannot participate in PW and livelihood activities will receive direct support (DS) in the form of unconditional cash transfers.1 These activities together (PW, DS, and the livelihood grant) will be referred to as the safety net component of the UPSNP. The urban safety net program focuses on providing beneficiaries with a longer-term outlook by focusing on a graduation approach. The design of the safety net component of the UPSNP is based on a graduation into sustainable livelihoods approach2 which provides an integrated development pathway for livelihoods and has been tested and implemented across several contexts. All beneficiary households that complete public works are also beneficiaries of the livelihood services. The livelihood program is implemented in stages, spanning a three year period. In the first year, beneficiaries participate in PW as well as life skills and financial skills training. They are encouraged to start saving from the beginning of the program to ensure they can later invest in their business. In the second year, beneficiaries receive support for business planning, technical training as needed, and job search support from woreda-level One Stop Service Centers (OSSCs) which specialize in supporting micro- and small-business development and receive a livelihood grant of US$500 per household. During the third year, beneficiaries devote less time to PW to ensure they can engage in building their business activities and continue to receive mentoring/coaching advice from the OSSCs to overcome potential challenges. The UPSNP has gathered substantial evidence of the effects of the program on the urban poor. A recent report documents the results of the assessment of the targeting performance of the program and the impact evaluation of the PW and DS components of the program on various development outcomes (Wieser, Franklin, & Hari, 2021). In terms of targeting performance, the reliance on a combination of geographic targeting and community-based targeting ensures that the targeting of the UPSNP was effective. The results from a rigorous impact evaluation of the PW and DS components of the UPSNP show that the PW component has short-run direct effects on a range of socioeconomic outcomes including household income, financial inclusion, and human capital. However, the direct 1 Moreover, people living in street situations receive services for their economic and social reintegration as part of the UPSNP Urban Destitute (UD) component. This component is not part of this evaluation. 2 The UPSNP approach to livelihoods was adapted from the graduation into sustainable livelihoods approach as developed by the Consultative Group to Assist the Poorest-Ford Foundation 2014. 2 support component does not generate significant impacts. This report expands on the previous impact evaluation (Wieser et al., 2021) by assessing the effectiveness and developmental impacts of the livelihood component of the UPSNP focusing on welfare and labor market outcomes. More specifically, the report seeks to: (i) provide an understanding of beneficiaries’ livelihood pathway (self or wage employment) and an overview of beneficiaries' success rates in implementing their livelihood pathway (overall, self-employment, wage employment), and (ii) estimate the impacts of the livelihoods component of the UPSNP project on welfare and employment. The rest of this report is organized as follows. Section 2 provides a brief description of the livelihood component of UPSNP. Section 3 describes the estimation method and data. Section 4 discusses the participation and success rates in relation to the livelihoods modalities. Section 5 presents the results of the impact evaluation. Section 6 concludes and synthesizes key lessons and implications for the implementation of the UPSNJP. 2. Program design and implementation The UPSNP aims at improving the incomes of targeted poor households. The UPSNP was launched in 2016 and funded by the Government of Ethiopia (GoE) with support from the World Bank. The long- term objective of the government's Urban Safety Net Strategy and Program framework is to reduce poverty and vulnerability among the urban poor living below the poverty line by implementing productive and predictable urban safety nets and complimentary livelihood interventions among the 4.7 million urban poor over a period of 10 years. These objectives are hoped to be achieved through the provision of cash transfers, financial, and technical support to access livelihood opportunities, building the capacity of institutions to effectively deliver this support, and developing core systems for the delivery of safety nets and complementary livelihood services. Consistent with the poverty reduction and economic development objectives of the Growth and Transformation Plans (GTP) and the National Social Protection Policy/Strategy, the Urban Productive Safety Net Strategy seeks to guide the implementation of interventions that will alleviate the varying needs of the urban poor. The first phase project supported by the World Bank was implemented between 2016 and 2022 focusing on putting in place basic safety net building blocks including productive and predictable transfers through public works, livelihood interventions, and capacity building. The UPSNP was implemented in the 11 largest cities of the country (Addis Ababa, Harar, Gambela, Semera, Asosa, Mekelle, Dessie, Hawassa, Jigjiga, Dire Dawa, and Adama), and targeted the poorest 12 percent of urban households with an emphasis on the public works component. Around 75 percent of the beneficiaries of the program reside in Addis Ababa given that urban poverty is concentrated in Ethiopia’s capital (Abebe, Franklin, & CarolinaMejia-Mantilla, 2018). The UPSNP was fully integrated with existing government strategies and structures. The UPSNP was implemented by the Urban Job Creation and Food Security Agency (JOBFSA) under the Ministry of Urban Development and Construction (MoUDC3 and was fully integrated into the existing government structure at the federal, regional, city, and local levels. The Ministry of Labor and Social Affairs (MoLSA) 3 Institutional arrangements for GoE have since changed and some ministries were merged and changed names since. For example, MoUDC is not the Ministry of Urban Development and Infrastructure. MoLSA was divided into the Ministry of Labor and Skills (integrating the Jobs Creation Commission) and the Ministry of Women and Social Affairs. 3 closely worked with the MoUDC to support the coordination and implementation of the DS component. The Ministry of Finance (MoF) and MoLSA also had an important role in providing overall guidance, mobilizing, managing, and allocating resources, and ensuring linkages with existing government strategies. The overall management and coordination of the project were supported by a Project Coordination Unit (PCU). The PCU reported to MoUDC and was staffed with appropriate technical and management staff. At the regional and city levels, the regional Bureaus or offices of the relevant ministries and concerned city administrations were responsible for the actual implementation of the project. The UPSNP had three major components (a) safety net support; (b) livelihood services; and (c) institutional strengthening and project management. The safety net support component of the UPSNP aims at supporting the delivery of a predictable, timely, and productive safety net through conditional and unconditional safety net transfers. Component one of the UPSNP includes conditional cash transfers and unconditional cash transfers.4 The conditional transfers target able-bodied persons in households eligible for the project support. The PW beneficiaries receive monthly transfers conditional on their participation in public works projects close to their place of residence, which could range from small-scale infrastructure projects to the provision of beneficial services to communities that are absent in urban areas. These may include urban greenery development, watershed management, solid waste management, environmental cleaning, social infrastructure, and other types of public work activities that emanate from the community (Wieser et al., 2021). The unconditional cash transfers target persons who for various reasons are unable to perform work (or participate in the PW) for example, the chronically ill, the elderly, and people with disabilities. The urban destitute include the target group who do not have access to a sustainable livelihood and often resort to begging or illicit activities to make a living. The UDs receive psychosocial and reintegration services, Direct support (DS) beneficiaries receive a monthly stipend as an unconditional cash transfer (lower than the PW payments). UPSNP payments are made through the Commercial Bank of Ethiopia, a stipulation that ensures that at least one member of the beneficiary household has a bank account. The UPSNP is envisioned as a three-phase graduation process. Each wave of beneficiaries participates in the UPSNP in three phases. In the first phase (1st year), beneficiaries received cash conditional on their participation in PWs. Those who were unable to work, received unconditional DS throughout the entire project duration. In the second phase (2nd year), conditional transfers continue and one beneficiary per household receives livelihood support including training, financial support, and guidance on the employment pathway (PW to LH pathway). In the third phase (3rd and graduation year), beneficiaries have the option to continue to engage in PW to supplement their employment income and receive coaching and mentoring services to strengthen their livelihood activities (Abebe et al., 2018). Thus, after three years, the PW beneficiaries exit from the program. The UPSNP was implemented in three waves of beneficiaries and this study looks at beneficiaries who participated during wave 1 and wave 2. The UPSNP included beneficiaries across 11 participating 4There is also another subcomponent, capital budgets for the public works sub-component, that covers capital inputs and material for public work activities. This sub-component supports the purchase of hand tools, material, small cart, equipment etc. required for urban greenery development, water shed management activities, urban agriculture, environmental cleaning activities, social facilities/services projects, construction of cobble stone roads, and building of drainages. 4 cities in three unique waves. Each wave of beneficiaries participated in a three year program (as outlined above). At the end of the three years, beneficiaries exited the program. For example, wave 1 beneficiaries roughly participated from 2017 through 2020; wave 2 beneficiaries participated from 2018 through 2021, and wave 3 beneficiaries participated from 2019 through 2022. The impact evaluation study only looks at beneficiaries who participated in wave 1 or wave 2 of the program. The livelihood services support interventions that complement the safety nets to facilitate graduation from the program and promote moving out of poverty. The livelihood services and support interventions complement PWs. The interventions focus on activities for PW beneficiaries aimed at enhancing employability skills, facilitating links to employment opportunities, and helping the urban poor unleash their productive potential both through wage and self-employment opportunities. One individual per household can receive livelihood support, which is comprised of counseling and life-skills development to support people successfully entering self-employment or wage employment. There are two phases: (i) livelihood skills, training, mentoring, and coaching, and (ii) livelihood transfer. The first phase of livelihood support focuses on enhancing financial literacy and soft skills that will be useful across a range of livelihood choices to improve employability (e.g., assessing the technical skills of beneficiaries and providing individuals with information on entrepreneurship and wage opportunities in their location). Beneficiaries receive short trainings on aspects such as learning skills on how to open a bank account, planning, budgeting, and saving, attitude and expectations at work, and business and workplace readiness. Moreover, financial literacy training is provided to all PW beneficiaries early on to ensure that they can save from the income they receive from PW during the PW years. The livelihood transfer component supports interventions that facilitate graduation from the program and promote moving out of poverty. After receiving initial training, beneficiaries are entitled to financial support and training for livelihood development. In the second phase of the livelihood support, individuals who received guidance in the first phase choose whether they would like to receive support to increase their income from self- employment (either by starting a business or increasing the profitability of an existing business) or wage employment (either by gaining access to wage employment or moving from low- to higher-wage employment). They receive financial support to pursue this livelihood pathway upon completion of a business plan (for self-employment pathway beneficiaries) or job search plan (for wage employment pathway beneficiaries). In addition to the financial support, individuals received the information, mentoring, and training necessary to develop and implement the business plan and/or the job search plan. While a lot of training was provided during the 3 years of PW regarding the LH, they started financial literacy and life skills training in year 1. The focus in year 2 was on business plan. Table 1 summarizes the number of beneficiaries by UPSNP component and year. Moreover, beneficiaries could access savings they accumulated over the three years of PW—beneficiaries were required to save 20 percent of PW payments each month—to support their business or job search. Table 1. Number of beneficiaries by UPSNP component and year Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Total PW 159,600 370,343 488,880 488,870 488,870 488,880 488,880 LH 48,854 62,705 30776 142,335 142,335 PDS 30,400 401,44 22,579 93,122 93,122 93,122 UD 3,392 8,234 9,365 20,991 5 Note: There is no complete information regarding the number of households covered. 3. Data and Methods This section discusses the sampling design employed for selecting the treatment and control households. It also presents a description of the sample type and data used in the analysis. Even though UPSNP was implemented in 11 cities across Ethiopia, the IE of the livelihood grants and service was restricted to Addis Ababa. This was because the design of the IE required the program to be rolled out randomly across woredas in order to have valid comparison groups. Moreover, given its large population, about 75 percent of the beneficiaries of the program are from Addis Ababa (Abebe et al., 2018). This was possible only in Addis Ababa since other cities started implementation with the poorest woredas. The UPSNP was randomized at the level of the woreda (urban district) within Addis Ababa. In year 1, only households residing in woredas with poverty rates above 20 percent were eligible for the program, specifically, 90 out of 116 woredas in the city. Randomization was conducted by a public draw of woreda names in November 2016, and it was stratified by 10 sub-cities (urban sectors) within the city. Of these 90 eligible woredas, 35 were selected for the program in year 1. These will be referred to as the “early” or “treatment” woredas. The remaining 55 woredas— referred to as “late” or “control” woredas—received the program in year two (around 12 months later). Year 2 beneficiaries (beneficiaries from year 2 woredas) that complete three years of PWs received livelihood grants and services one year after year 1 woredas or households. The sample of households was selected both before the start of the program and before the randomization of woredas into the program. Figure 1 below shows the outcome of the woreda level randomization. Figure 1: Woreda Level Randomization A central challenge associated with data collection was that households that would be part of the IE had to be identified before targeting of beneficiaries under the program had commenced. This was because targeting in control woredas would only take place in year two, and the baseline survey had to be conducted before the program had been implemented anywhere. The first step, therefore, was to identify households that were likely to participate in the program. This was done through a PMT module administered on approximately 32,000 households which was carried out in October 2016 across all 90 woredas. Using the PMT model, households that were predicted to be below the 6 30th percentile of the Addis Ababa consumption distribution were included in the baseline survey. Based on this exercise, a total of 6,093 households were included in the IE across treatment and control woredas. It should be noted that this sampling strategy selects those households who are most likely to be poor based on certain observable characteristics (demographics, assets, etc.) and, therefore, unlikely to be representative of all UPSNP participants. As a result, while the randomized design ensures that the results of the IE will be internally valid, it is important to pay careful attention to this issue while extrapolating these results to all UPSNP beneficiaries or locations. This analysis utilizes data collected from about 6,000 households from Addis Ababa. The impact evaluation utilizes data from the public works baseline data collected in November/December 2016 and livelihoods follow up surveys conducted in May 2021 and August 2022 with about 6,000 households. The impact evaluation is based on 4,829 households that were observed in the baseline and the livelihood follow-up surveys and excluding direct support households that are not eligible to receive livelihood grants (Table 2). The treatment groups include 1,117 year 1 households and 1,237 year 2 households. Households that did not receive livelihood grants and other support either in year 1 or year 2 are considered as control groups as they never received the project. Table 2. Sample distribution by intervention status Intervention year Total Sample Year 1 Year 2 Non UPSNP 601 828 1,429 UPSNP but no livelihoods 429 581 1,010 Livelihoods 1,117 1,273 2,390 Total 2,147 2,682 4,829 Note: The sample size is the same for endline 0 (public works baseline) and the livelihood endline surveys (endline 3 and endline 4 that refer to the livelihood endline surveys for year 1 (May 2021) and year 2 (August 2022)). Each wave of beneficiary receives support for roughly 3 years. PWs started in 2017 for wave 1 and in 2018 for wave 2 beneficiaries. The life skill and financial literacy trainings were phased into the program in the first year of PWs. The business plan and job search preparation and coaching started in the second year of PWs, which was 2018 for wave 1 and 2019 for wave 2 beneficiaries. Beneficiaries received training on business and job search plan implementation in 2019 for year 1 and in 2020 for year 2 beneficiaries. 4. Livelihoods participation and success This section provides an overview of (i) beneficiaries’ livelihood pathway (self or wage employment; (ii) beneficiaries' success rates in implementing their livelihood pathway (overall, self-employment, wage employment) reasons for success and failure; and (iii) what and how livelihood grant was used, distinguishing between successful and unsuccessful livelihood pathways and between self and wage employment. Livelihood training Participation in and satisfaction with livelihood training are high. UPSNP beneficiaries received different training as part of the program. Both participation in and satisfaction with life-skills training (e.g., topics covered: objective setting skills, motivation, interpersonal skills, working culture) and 7 financial literacy training (e.g., financial management and saving, planning) are high for year 1 and year 2 beneficiaries (Figure 2). Although participation in personal initiative training (e.g., topics related to motivation/positive thinking, mindset, or outlook) was relatively lower, satisfaction was high for both years. Concerning the intensity of training (in terms of the number of days covered), more than 75 percent of the beneficiaries received 2-7 days of life skills and financial literacy training. However, about 15 percent received 2 or fewer days of training. On the other hand, about 90 percent received up to 10 hours of personal initiative training. There are not many differences between year 1 and year 2 beneficiaries. Figure 2. Training as part of the LHG: participation, intensity, and satisfaction 94%98% 92%98% 99% 82% 7% 5% 10% 38% 40% 22% 39% 39% 47% 17% 15% 21% Life skills Financial Life-skills Financial Personal training literacy training Personal initiative training training literacy initiative training training < 2 days 2-4 days < 4 hours 4-7 hours Participation Satisfaction 5-7 days > 7 days 8-10 hours > 10 hours Source: Authors’ calculation based on endline 4 Livelihood modality Self-employment is the preferred livelihood modality compared to wage employment for UPSNP beneficiaries. Of those completing 3 years of PWs and receiving the livelihoods grants, 97 percent went to the self-employment pathway (Figure 3) and only 3 percent chose wage employment. The results from further analysis show that there is no difference in household characteristics among beneficiaries across modalities, implying that differences in household characteristics do not determine differences in livelihood modality choice. Figure 3. Livelihood modality Figure 4. Business plan purpose 4% 3% 3% 16% 22% 19% 96% 97% 97% 84% 78% 81% Year 1 Year 2 Total Year 1 Year 2 Total Self-employment Wage-employment Open a business Expand a business Source: Authors’ calculation based on endline 4 Source: Authors’ calculation based on endline 4 Self-employment pathway Business plan development 8 Of all beneficiaries in the self-employment modality, about 94 percent of the beneficiaries submitted a business plan. Of those who submitted the business plan, the vast majority—84 percent of year 1 and 79 percent of year 2 beneficiaries—wanted to open a new business rather than expanding an existing business (Figure 4). The analysis shows that more than half of beneficiaries received support in completing their business plan. Lack of economic decision-making skills appears to be a challenge for business plan development. For households who faced challenging in drafting the business plan, the main challenges include estimating the amount the business can produce and sell in both years and estimating costs including raw materials and other inputs (Figure 5). However, deciding on a product is reported to be a less important challenge in business plan development. Figure 5. Main challenges faced in drafting the business plan Figure 6. Source of business plan support Deciding on a product 13% 18% 15% Estimating costs (e.g., raw materials, inputs) 37% 36% 35% Estimating how much would be produced Estimating how much would be sold Thinking of a marketing 50% 47% 48% plan Estimating competition for the product Others (lack of space, Year 1 Year 2 Total finance, inputs) TVET college 0% 10% 20% 30% 40% Friends/Other household members Year 1 Year 2 Total OSSC Officials Source: Authors’ calculation based on endline 4 Source: Authors’ calculation based on endline 4 Nearly half of households in self-employment received business plan development support. The main source of business plan support is the One Stop Service Centers (OSSC) officials that specialize in supporting micro- and small-business development followed by social networks made up of friends, relatives, or other household members (Figure 6). The other important source of support for business plan appears to be TVET colleges. Business plan implementation Of all households in self-employment, about 87 percent reported that they implemented the business plan which was in the business plan they had prepared. Of the 87 percent that implemented the original business plan, 71 percent said the business plan was helpful in implementing their business. For the 13 percent that did not implement the original business plan (the business they opened/expanded is different from the one you had written the business plan for), challenges are mainly related to difficulties in securing inputs to open a business as per the original plan (Figure 7). Not surprisingly, for year 1 beneficiaries, who received the LH grant just a few weeks after COVID-19 hit Ethiopia, the COVID-19 pandemic is reported to be an important reason (29 percent) for not implementing the original plan while this was much lower for year 2 beneficiaries (6 percent). 9 Moreover, lack of adequate workspace and financial constraints tend to be important reasons for year 2 beneficiaries to not implement the original business plan. Figure 7. Reasons for not implementing the original business plan 8% 14% 12% 37% 43% 47% 27% 30% 32% 29% 16% 6% Year 1 Year 2 Total Other (work space and capital shortage) Difficulty of securing inputs to open the original business Original plan did not seem viable for other reasons Original plan did not seem viable due to COVID-19 Source: Authors’ calculation based on endline 4 Business-related training: participation, intensity, and satisfaction While a large share of the self-employment households received business-related training, there is a low rate of market linkage support. While some trainings are the same for households in wage and self-employment, each employment modality has some specific trainings relevant to their modality. For self-employment, these trainings are business training, technical training, and market-linkage support training. About 93 percent of the self-employment pathway households received business training—topics covered include marketing, production, and how to manage a business (Figure 8). The satisfaction rates are also high. However, there is relatively low participation in technical training related to business (43 percent) although the satisfaction rate is high (97 percent). Concerning the intensity of training, around 85 percent received more than 2 days of business training and about 80 percent received more than 2 days of technical business training. A striking result is the very low rate of market linkage support from MSEs (12 percent), although satisfaction is high. There are not many differences in the level of participation and satisfaction related to the business support training and market linkage support among year 1 and year 2 beneficiaries. Figure 8. Business training: participation, intensity, and satisfaction 97% 93% 98% 96% 8% 17% 43% 40% 27% 12% 37% 37% Participation Satisfaction Total 15% 19% Business training Technical training related to business Market linkage support (from MSEs) < 2 days 2-4 days 5-7 days > 7 days Source: Authors’ calculation based on endline 4 s Self-employment pathway success rates 10 The rate of households who established a business for the self-employment pathway is rather low. Although a high share of livelihood beneficiaries chose the self-employment pathway, the success rate—indicated by the low share of households using the grant to open or expand a business—is rather low. Only 44 percent of households in the self-employment modality opened or expanded a business using the livelihoods grant (Figure 9). More specifically, only 38 percent of year 1 and 50 percent of year 2 beneficiaries used the business grant for opening/expanding their business. The different success rates for year 1 and year 2 beneficiaries point to the challenges year 1 households faced to open/expand a business during COVID-19. For those households that used the grant to open/expand a business, analysis of the grant use pattern shows that about two-thirds used the grant to open a new business and one-third used the grant to expand an existing business. Sustainability of businesses that were opened or expanded thanks to the LH grant was relatively low. Of those households that opened or expanded a business, only half (54 percent) still operated the business 12 months after receipt of the LH grant (Figure 9). Of all households in the self- employment pathway (not just those opening or expanding a business), only 24 percent are still operating the business 12 months after receiving the grant with a stark difference between year 1 (19 percent) and year 2 (28 percent) households. Overall, success rates for the self-employment pathway beneficiaries are low but relatively higher for year 2 beneficiaries. The results from further analysis show that the COVID-19 impacts explain the lower success rate for year 1 beneficiaries who received the livelihoods grant when the spread and impacts of the pandemic in Addis Ababa were at their peak. Figure 9. Business grant use pattern and success rate by year 50% 44% 38% 28% 24% 19% Year 1 Year 2 Total Opened/expaned a business (using the grant) Source: Authors’ calculation based on endline 4 Reasons why businesses are not successful The high cost of inputs appears to be a major reason for business failure for beneficiaries who opened/expanded a business but were not able to keep it for 12 months. Regarding the low success rate of the self-employment pathway, the main reasons why the business opened/expanded using the livelihoods grant did not operate for at least 12 months includes the high cost of inputs (Table 3). The high cost of inputs is reported to be a major challenge for year 2 beneficiaries (34 percent), less so for year 1 beneficiaries (23 percent). This could be due to the high cost of inputs due to recent inflation. For example, inflation between April 2020 and March 2021—roughly the year after year 1 beneficiaries received the livelihoods grant—was 18 percent and soared to 34 percent the year after. Consistent with the results in the aforementioned discussions on the effects of COVID-19 for year 1 beneficiaries; COVID-19-induced low product demand is reported to be a challenge for 15 percent of the beneficiaries, but more important for year 1 (25 percent) than year 2 (7 percent) beneficiaries. 11 Table 3. Reasons why the business operation is discontinued Reason Year 1 Year 2 Total The person operating the business found wage employment 0% 1% 0% The person operating the business started another business 2% 3% 2% Extremely low demand for the product (due to COVID-19) 25% 7% 15% Extremely low demand for the product (due to other reasons) 13% 11% 12% The difficulty of securing a regular flow of inputs 12% 13% 12% High cost of operating the business than anticipated 23% 34% 30% Very stiff competition from similar businesses in the neighborhood 6% 10% 8% Other (e.g., lack of working place, health problems, capital shortage) 19% 22% 21% Source: Authors’ calculation based on endline 4 Reasons for not using the livelihoods grant to open/expand business When asking the 56 percent of households who did not successfully open or expand business, more than half of the self-employment households used the livelihoods grant to cover regular household expenses. Of those households that did not use the livelihood grant to open or expand a business, close to half of the beneficiaries (slightly more during year 2) indicate that they used the grant money to cover regular household expenses or to cover emergency household expenses (Table 4). For those that used the business grant to cover household expenses, reduced economic opportunities (78 percent)—likely related to the many crises happening in Ethiopia such as conflict, rising cost of living, COVID-19—and COVID-19-related income losses (16 percent) were reported to be the major cause of the shortfall in the regular household budget. The other important reason for not using the livelihoods grant for its intended purpose is that opening/expanding a business did not seem viable because of COVID-19 (more for year 1) or other reasons. Table 4. Reasons for not using the livelihoods grant to open/expand business Reason Year 1 Year 2 Total Opening/Expanding a business did not seem viable due to COVID-19 27% 8% 18% Opening/Expanding Business did not seem viable due to other reasons 11% 14% 13% Household members were busy with other wage/self-employment activities 2% 4% 3% Did not feel prepared to open/expand a business 1% 2% 2% Needed the grant money to cover regular household expenses 47% 55% 51% Needed the grant money to cover emergency household expenses 12% 17% 14% Causes of shortfall in the household regular budget: Reduced economic opportunities 75% 81% 78% COVID-19-related income losses 20% 12% 16% Other (health) 5% 6% 5% Source: Authors’ calculation based on endline 4 Note: The cause of the shortfall in the household's regular budget is asked for those who have used the livelihoods grant to cover regular household expenses. Grant sufficiency and additional finance source A majority of households in the self-employment modality indicated that the business grant was sufficient. Of all in the self-employment pathway that used the business grant to open or expand a 12 business, 70 percent indicated that the grant was sufficient to cover costs related to opening/expanding the business. Among those that said the grant was not sufficient, they sought additional sources of funds from personal savings (58 percent) or private money lenders (11 percent) to cover needs (Figure 10). Though only 17 percent of all PW beneficiary households in the self- employment pathway used savings to invest in a business, for those who did the 20 percent savings requirement was likely useful. About 8 percent rely on credits or loans from a bank, microfinance institutions, the sale of assets, and family or friends. Figure 10. An additional source of a fund when a grant is insufficient 56% 59% 58% 10% 11% 11% 11% 8% 6% Personal savings Private moneylenders Others (bank, family, microcredit, etc) Year 1 Year 2 Total Source: Authors’ calculation based on endline 4 Challenges in opening or expanding a business For households that open or expand a business using the business grant, securing inputs and finance and competition appear to be the main challenges. Additional analysis is conducted to learn about the challenges in opening/expanding a business. The results show that the main challenge for beneficiaries attempting to open/expand a business is the difficulty of securing finance (Figure 11). Unsurprisingly, competition from other businesses is mentioned to be a key challenge for opening a business. This could reflect the low success rate for the self-employment pathway households because high competition from other businesses would force them to quit their businesses or engage in less remunerative activities that have implications on their earnings from self-employment. The difficulty of securing inputs, market promotion, and managing accounts are also mentioned as important challenges for opening and operating a business. In addition, other challenges reported include lack of working space/places, financial/capital shortage, inflation, and COVID-19. Figure 11. Main challenges for opening a business Securing source of inputs 52% Securing additional financing 63% Letting people know about product 49% Managing accounts 43% Competition from other businesses 54% Source: Authors’ calculation based on endline 4 13 Future plan and support required About a third of the self-employment households, that were not successful in opening or expanding a business, have the plan to open a business in the future. Of the self-employment pathway households who did not use the business grant to open or expand their business, 32 percent report that they have a plan to open a business in the future: 25 percent for year 1 and 45 percent for year 2. This is mainly because they did not feel prepared to open/expand the business now. For those who planned to open a business in the future, respondents indicated additional needs for support in the areas of training on financial management, identifying markets, and market promotion (for year 2 beneficiaries) for the business product(s). Other areas of support include workspace and finance access (Figure 12). Figure 12. Support needed for opening a business in the future 25% 20% 22% 20% 26% 38% 40% 39% 38% 20% 13% 0% Year 1 Year 2 Total Other (working space, finance) More information about the market for your product Additional training on managing finances Additional training on informing people about product Source: Authors’ calculation based on endline 4 Wage employment pathway Writing a job search plan seems to be a major challenge for households in wage employment with less than 60 percent submitting a job search plan. While only 3 percent of the livelihood beneficiary households chose wage employment as their livelihood modality, only 59 percent of those in the wage employment pathway submitted a job search plan under the program. Only a small share of the wage employment households used the livelihoods grant for job search purposes. For those in wage employment, the livelihood grant is supposed to support the livelihood beneficiary in searching for a job but less than 10 percent of the households in wage employment reported that the money was used for job search (Figure 13). However, a large majority used the grant to cover regular household expenses (89 percent) and emergency expenses (18 percent). The share of households that used the grant for job search (and training) related purposes is only 7 percent, with no difference between year 1 and year 2 households. Surprisingly, there are few households (8 percent) in year 2 that have used the grant for other purposes including starting a business. Figure 13. Job search grant use pattern 14 88% 89% 89% 19% 18% 16% 8% 7% 7% 5% 5% 4% 4% 4% 3% 3% 0% 0% 0% 0% 0% Cover the cost Stipend to Travel Travel to work To cover regular To cover Other (start a of additional attend training allowance for job and other household emergency business) training search ancillary costs expenses expenses programs such as childcare Year 1 Year 2 Total Source: Authors’ calculation based on endline 4 Job search training: participation, intensity, and satisfaction Of trainings specific for wage employment, job search training participation was high. Among the households in the wage employment modality, receipt of job search training was higher for year 2 households (82 percent) than year 1 households (72 percent). However, satisfaction with the job search training is high in both years. Regarding the intensity of job search training, all of year 2 and about 80 percent of year 1 households received up to 4 days of training (Figure 14). Around 19 percent of year 1 households report that they have attained 5 or more days of job search training. Figure 14. Wage employment training: participation, intensity, and satisfaction 97% 100% 98% 81% 0% 10% 72% 76% 19% 80% 67% 55% 26% 20% 23% Year 1 Year 2 Total Received a job search training Year 1 Year 2 Total Satisfied with the job search training 2 days or less 3-4 days 5 or more days Source: Authors’ calculation based on endline 4 Only half of households in wage employment received guidance or information on employment opportunities. There is a stark difference between year 1 (44 percent) and year 2 (59 percent) households in terms of receiving guidance or information on employment opportunities (Figure 15). Although One Stop Service Centers (OSSC) appear to be the major source of information for or receiving guidance on employment opportunities, there are still differences across years: 74 percent for year 1 and 91 percent for year 2. Existing social networks (friends and family) are reported to be the second most important source of information for employment opportunities, particularly for year 1 (21 percent) households. This leaves job boards to be the third important source of information for employment opportunities. Figure 15. Receipt of guidance or information on employment opportunities 15 5% 5% 12% 21% 5% 59% 5% 51% 44% 91% 74% 83% Year 1 Year 2 Total Received guidance or information on employment opportunities Through existing networks (friends and family) Through job boards Year 1 Year 2 Total At OSSC Source: Authors’ calculation based on endline 4 Wage employment success rates The share of households in the wage employment modality that found a wage job is very low with only 8 percent of households starting a new job. Differences for year 1 and year 2 are large, though low in both cases and only 5 percent of year 1 and 11 percent of year 2 households have started a new job with the livelihoods grant (Figure 16). When considering sustainability of wage employment, results are slightly more encouraging with around 67 percent of those who found wage employment still in wage employment 12 months after receiving the grant. Of all households in the wage employment modality, only 5 percent are still employed, with no difference across years. There is a clear need for improving the wage employment pathway. The low success rate for the wage employment pathway could be due to the low selection rate of the pathway and the low rate of grant use for job search. Figure 16. Wage employment pathway success rates 100% 67% 50% 5% 5% 11% 5% 8% 5% Year 1 Year 2 Total Started a new job (wage employment) in the past 12 months Started a new job and still employed in that new job (of those employed) Started a new job and still employed in that new job (of all in wage track) Source: Authors’ calculation based on endline 4 Satisfaction with the livelihoods program Despite the disappointing performance of the program, there is a high level of satisfaction among beneficiary households. The results so far demonstrate low success rates for both self-employment and wage employment regarding the use of their grants for opening/expanding a business or for securing a job. Despite this, there is a high level of satisfaction with the livelihoods grant that the beneficiaries received as part of the program and the way payments were made (Figure 17). There is not much difference in satisfaction rates among year 1 and year 2 beneficiaries. However, satisfaction is relatively low for the support provided by woreda officials. 16 Figure 17. Satisfaction with the livelihoods and woreda officials supports 92% 94% 93% 92% 96% 94% 80% 84% 82% Satisfied with the livelihoods grant Satisfied with how payments were Satisfied with the support provided by received made for the LHG woreda officials Year 1 Year 2 Total Source: Authors’ computation from endline 4. 5. Empirical Strategy This section provides a discussion of the empirical strategy used for the impact evaluation of the livelihood grants and service. Two alternative strategies are presented to estimate impacts by exploiting the randomization of the UPSNP and the timing of the livelihood grant receipt. It also includes a simplified theory of change that illustrates the impact pathways through which the livelihood intervention would affect the outcomes of interest. Rigorous impact evaluations are an important aspect of the M&E strategy of the UPSNP. To understand the effectiveness of the livelihoods grants and support service, it is vital to rigorously evaluate its impact. Such evaluations can help clarify what aspects of the program worked well, which ones were not effective, and the reasons for their success or failure. This information is valuable in helping to select program design features which have the most beneficial impact and can help inform the expansion of the program to other parts of the country in the future. The impact evaluation of the livelihood component of UPSNP is undertaken focusing on welfare, assets, and labor market outcomes using rigorous impact evaluation techniques. The analysis provides both the short-run and medium- run direct effects of the livelihood program, where effects are captured one and two years after the livelihood grants and other service (training) were offered to beneficiaries. The rest of this sub-section is organized as follows: first, we provide an overview of the impact evaluation (IE) design, followed by a theory of change which provides a framework to understand the potential impact of the program and the channels through which impacts are mediated. This is followed by a description of the data and sampling strategy. The impact evaluation addressed three key questions. The key questions this study seeks to answer include: (i) Did the livelihood grants and livelihood support services succeed in creating better outcomes for UPSNP beneficiaries or in achieving livelihood opportunities (ii) Has successful opening/expansion of businesses made a difference in outcomes compared to those who received the grant but did not open/expand a business? (iii) Is there a difference in beneficiaries who received the grant in year 1 and those who received the grant in year 2 with respect to setting up their livelihood opportunities, consumption, labor market outcomes, and assets? 17 The main outcomes of interest considered include welfare (consumption, earnings), labor market (household enterprises ownership, employment rate, hours of employment, employment earnings), and asset wealth (asset index). Two identification strategies, both with advantages and disadvantages, are applied for the impact evaluation. Given the nature of the intervention, two complementary approaches were applied for the quantitative assessment of the short-term effect of the livelihood component of the UPSNP. First, a difference-in-differences (DID) approach was applied, based on the comparison of the changes in outcome indicators (household enterprise ownership, consumption, earnings, assets, and labor market outcomes) of households in the treated and control woredas before and after the intervention.5 The second approach is a post-intervention comparison of households in treatment and control woredas that involves comparing UPSNP households in year 1 treated woredas to UPSNP households in year 2 treated woredas (that did not yet receive the livelihood grant at the time of endline 3 but received it at the time of endline 4). The results presented in this report reflect the effects on first- and second-year beneficiaries. The impact evaluation for year 1 using endline 3 and for year 2 using endline 4 provides an estimate of the short-run direct effects because impacts are estimated one year after the intervention. The estimation for year 1 beneficiaries using endline 4 (two years after receiving the treatment) provides estimates of the medium-run direct effects, it also provides a means of testing whether any of the short-run effects persist over time. Table 5. Two identification strategies Status Year 1 Year 2 No UPSNP A B UPSNP but no LHG C D UPSNP and LHG E F The first identification strategy involves comparing UPSNP livelihood beneficiaries to non UPSNP (non-livelihood beneficiary households). The first identification strategy to estimate the causal impact of the livelihood grants and services is using a difference-in-difference (DID) approach that involves comparing the outcomes of UPNSP livelihood beneficiaries and non-beneficiaries before and after the livelihood intervention (A+C vs. E for year 1 and B+D vs. F for year 2 in Table 5). DID considers not only post-intervention difference but also pre-intervention differences between the two groups. A DID estimate provides a consistent and unbiased estimate of the average causal effect of the intervention if and only if the temporal trends in the absence of the intervention are the same in both groups. Since participation in the PW to LH pathway component of the UPSNP could be nonrandom, the impact evaluation combined the DID approach with controls for baseline characteristics. The identification strategy has the advantage that non-UPSNP households are not in public works and so are a “purer” control group. To ensure that impacts are correctly identified, we have to assume that there are no spillovers and that UPSNP beneficiaries are on the same trajectory that they would have been without the existence of the UPSNP. However, the approach has some limitations because the assignment to UPSNP was not random. The non UPSNP households, however, looked the same based 5 As discussed above, treated woredas are those that started the program in year 1 while we see control woredas as those where the program started in year 2. 18 on the PMT-like characteristics. Therefore, the analysis is not based on a randomized comparison but rather relies on an approach with a weaker validity compared to a randomized comparison. The DID approach has the following form = + + ( ∗ ) + + + where indicates households (in treatment or control groups) in year t (baseline or endline 3 or endline 4). takes a value of 1 for the livelihoods endline (endline 3 or endline 4) and 0 for the baseline. is the treatment dummy indicating whether the household belongs to the UPSNP livelihoods group or not. In this analysis, “treatment” includes year 1 beneficiaries in endline 3 and endline 4 or year 2 beneficiaries in endline 4, while “control” includes non-UPSNP beneficiaries either in endline 3 or endline 4. For endline 3, the treatment group are year 1 beneficiaries that received the livelihoods grant and support services 1 year ago, and the control group are year 2 beneficiaries who have not yet received the livelihood grants or support services and non-UPSNP or non-livelihood households. For endline 4, the treatment group are year 1 and year 2 beneficiaries that received the livelihoods grant and support services, and the control group are non-UPSNP households. X is a vector of baseline household characteristics that include age and sex of head, presence of disabled member, head employment, head education, highest education level of members, housing status (rented from kebele or not), housing feature (quality of floor), improved sanitation, number of living rooms, weekly expenditure on food, business enterprise ownership, and household size. The baseline controls are added to account for differences in characteristics between the treatment and control groups in the baseline. Lastly, is a vector of household fixed effects and is the error term. The coefficient of primary interest is that of the interaction term, , which provides an estimate of the average treatment effect on the treated (ATT). The dependent variable Y is our welfare (consumption expenditure, poverty, food security) or labor market outcomes. The results from the estimation of equation (1) are intended to guide inference on whether the livelihood contributed to household welfare and improvement in labor market outcomes (self-employment or wage employment). To remove bias, the DID model is estimated with controls. The second identification strategy involves comparing year 1 and year 2 livelihoods participants. This strategy entails comparing all participants in the livelihoods in year 1 woredas to year 2 woredas (E vs. F in Table 5), using livelihoods endline 3 and endline 4 data. For year 1 woredas, the analysis looks at whether they said they received livelihoods grant and services in livelihoods endline 3 and endline 4. For year 2 woredas, we look at whether they said that they received livelihood grants and services in livelihoods endline 4. This approach helps us unveil the immediate (one year after livelihood grant and support receipt) vs. medium-term (two years after the livelihood grant and support receipt) effect. Year 1 beneficiaries received their livelihoods grant several weeks after the first case of COVID- 19 was confirmed, which was followed by the introduction of a State of Emergency and stringent restrictions on movement and assembly. Given this context, the analysis helps us to understand if the COVID-19 pandemic impacted year 1 beneficiaries differently since their livelihoods grant was paid while restrictions on movement and assembly were in place while they had already been lifted by the time that year 2 beneficiaries received their livelihoods grant. The first advantage of the second identification strategy is that we only look at people who received the livelihood grants and support services. The second advantage is that assignment to year 1 and year 2 was random, which means this comparison is “randomized”. 19 There are several mechanisms through which the livelihoods intervention would affect the outcomes of interest, in the short, medium, and long term. Given that the livelihoods component of UPSNP builds on the PW program, the channels that mediate impacts of PWs could also work for the livelihood component. The income support to beneficiaries under the PW provided livelihood beneficiaries access to income-earning opportunities, thereby increasing their income and risk management capacity through income diversification. Moreover, the livelihood grants could relax credit or income constraints, thereby enabling beneficiaries invest in remunerative activities that would in turn contribute to increased income. The income support to beneficiaries that could increase the access to income-earning opportunities is also expected to contribute to increase in earnings and consumption. At the same time, households could reallocate labor supply away from the private sector and towards PW. While the grants help establish businesses or secure employment (jobs), the increased income could allow beneficiaries to increase their expenditure and purchase household assets. Given that grants are fungible, the channel is less likely to work if grants are not used for intended purposes. Financial literacy and other trainings could help beneficiaries improve their financial management. This could also promote financial inclusion and savings since the PW program has as a requirement minimum savings and opening of bank accounts. The UPSNP requirements on minimum savings and opening of bank accounts led to increased savings and increased financial inclusion as every beneficiary without a bank account opened a bank account to receive PW payments. 6. Impact estimates This section discusses the results of the impact evaluation. This evaluation estimates the short- and medium-term effects of the effect of completing three years of PW and receiving a livelihood grant on household welfare and labor market outcomes. As discussed in chapter 4, the livelihoods grant recipients used grants to start small enterprises, many of which were still operating at endline, but we did not see much success in securing wage employment. Yet, the most important question is, did the livelihood grant and support services lead to better outcomes compared to those not in the UPSNP? To estimate the impact of the livelihood grant, we have compared recipients of the livelihoods to a control group. In this case, the control group refers to households who were not included in the UPSNP. Table 5 summarized the estimated impacts on the outcome indicators for year 1 and year 2 households obtained from the DID estimations (full results are provided in Table A.1 in the Annex). Table 6 provides estimates from a post-intervention comparison of households in treatment and control woredas i.e., UPSNP households in year 1 treated woredas to UPSNP households in year 2 treated woredas by exploiting the randomized nature of the UPSNP (full results are provided in the Table A.2 in the Annex). Given that the self-employment pathway is taken up by almost all beneficiaries (97 percent), the main results are essentially estimates of the effect of the self- employment modality. Table 6. Effects on Livelihood Beneficiary Households: Results from the DID model Endline 3 Endline 4 Outcome indicator Year 1 Year 1 Year 2 Effect Magnitude Effect Magnitude Effect Magnitude Household enterprise No effect Positive 11.1 pp Positive 14.5 pp ownership 20 Hired household members No effect Positive 0.04 Positive 0.09 (total) Hired nonmembers (paid) Positive 0.01 No effect No effect Average monthly profit No effect No effect Positive 140.5 Total earnings No effect No effect Positive 365 Total earnings in Birr (per Negative -115.6 Negative -113 No effect working age members) Nonpublic work earnings Negative -114.8 No effect No effect (per working age member) Self-employment earnings No effect No effect No effect Wage earnings No effect No effect Positive 322.8 Non-public works hours No effect No effect No effect (per working age members) Monthly expenditure per Negative -68.3 No effect No effect adult equivalent Asset index No effect No effect No effect Source: Authors’ computation from endline 4 and using a DID approach; The specifications include baseline controls. Note: No effect means the coefficient is not significant; pp refers to percentage points. Earnings, profits, and expenditure are in Birr per month. Hours are per week. The asset index is constructed using the using inverse covariance weighting with all household assets covered in the survey. Table 7. Effects on Livelihood Beneficiary Households: Results from post-intervention comparisons regression Endline 3 Endline 4 Outcome indicator Effect Magnitude Effect Magnitude Household enterprise ownership No effect No effect Hired household members (total) No effect Negative 0.7 Hired nonmembers (paid) No effect No effect Average monthly profit No effect No effect Total earnings No effect No effect Total earnings (per working age No effect No effect members) Nonpublic work earnings (per working age No effect No effect member) Self-employment earnings No effect Negative 114.7 Wage earnings No effect No effect Non-public works hours (per working age No effect Negative 1.4 members) Monthly expenditure per adult equivalent Negative 96.2 Negative 32.1 Asset index negative 0.11 No effect Source: Authors’ computation from endline 4 and using a post-intervention comparison; All regressions include baseline controls. Note: No effect means the coefficient is not significant; pp refers to percentage points. Earnings, profits, and expenditure are in Birr per month. Hours are per week. The asset index is constructed using the using inverse covariance weighting with all household assets covered in the survey. Household enterprises The UPSNP livelihoods component has significant impacts on household enterprise ownership. The estimation results from the DID estimation suggest that the livelihoods component of the UPSNP lead to increase in household business enterprise ownership associated with hiring household members (Table 5). While the results show that the program does not lead to enterprise creation in the first 21 year of the program (for year 1 beneficiaries), year 1 woredas (beneficiaries) do experience enterprise creation in year 2 of 11 percentage points. Likewise, year 2 woredas experience a 15-percentage point increase in household enterprise ownership, over a mean of 18.7 (which is a nearly 100 percent increase). Overall, the livelihoods services help beneficiary households increase their engagement in business enterprises. While nearly 20 percent of households in the control group run a household enterprise, this rises to just over 35 percent in the treatment group (year 2 recipients of the livelihoods). The significant impact on household enterprise ownership corresponds to more household members being hired in the enterprise. Overall, very few households hire non-household members on their enterprise. Treated households are even less likely to hire. Figure 18. Impacts on business enterprise ownership Source: Authors’ calculation based on endlines 3 and 4. Notes: Household enterprise (left figure) and enterprise and household (right figure) refer to household enterprise ownership. Disaggregation by year shows evidence that households in year 1 suffered negative effects. In endline 3 – year 1 woredas did not start enterprises (Table 6). This is because year 1 was the year of COVID-19. Analysis using data from endline 4 show that, the next year, there were enterprises opening. But the effect for year 1 is smaller than for year 2. This indicates that year 1 woredas are doing worse compared to year 2 woredas due to COVID-19. Household earnings and income There are no discernable effects on total household income. The impact evaluation result show that the livelihood intervention did not bring a significant impact on total household income (in birr per working age member). The data show that earnings from self-employment are generally low, and even less comes from household enterprise profits (Figure 19). Also, the effects on profits are muted. This could be associated with allocation of the livelihoods grant to household expenditure, that constraint investment on remunerative income generative business enterprises. As discussed later, the main type of enterprises operated (opened and operated) by the livelihood grant and support service beneficiaries are petty businesses with low income generating capacity. Figure 19. Impacts on earnings and profit Figure 20. Types of enterprises 22 Source: Authors’ calculation based on endline 4 Source: Authors’ calculation based on endline 4 The main types of enterprises households open are involved in petty trade with less prospect for income generation. Further analysis shows that the main types of enterprises households open include baking of injera, guilet (petty trade in fixed marketplace, for example, selling of vegetables, charcoal, etc.), and brewing of coffee and beer. Baking of injera and brewing of coffee and beer are petty businesses with few prospects for high incomes. Guilets are also often petty and roadside markets where business owners are predominantly female, business owners have no access to credit which could facilitate running their businesses. The endline 4 data show that 13 percent of year 2 and 9 percent of year 1 households have guilets related business, leading to 3 percentage points difference. Likewise, 4.5 and 2.7 percent of the year 2 and year 2 beneficiaries, respectively, have baking business, with 2 percentage points difference. Overall, the impact evaluation results using endline 4 data show that the livelihood grant led to increase in the likelihood of owning guilet and baking type of businesses more for year 2 than year 1 households. While UPSNP livelihood households have more enterprises, the average profitability per enterprise is much lower than for non-UPSNP households. The data show that livelihood beneficiaries are almost twice as likely to have a household enterprise compared to non-beneficiaries of the control group. However, the main types of enterprises for UPSNP are in petty trade with low incomes. Overall, non- UPSNP households generate higher average profits from their enterprises that include shops/kiosk and sewing/weaving service providers than UPSNP households (Table 8). Moreover, for UPSNP beneficiaries who used the grant to expand an existing business have higher average monthly profits (ETB 1,714) compared to those who established a new business (ETB 1,437). Table 8 also depicts those enterprises started by the livelihood grants are less profitable than enterprises owned by the control group. This is more pronounced for year 1 beneficiaries than for year 2 beneficiaries. This could be because UPSNP businesses are established only recently, and businesses are not often profitable during their earlier periods of operation. Table 8. Average profits for households with Figure 21. Distribution of profits by group enterprises 23 Treatment Year 1 Year 2 Total status Control (non- 2,779 2,680 2,740 livelihoods) 252 163 415 Treatment 1,820 1,666 1,755 (livelihoods) 466 346 812 Total 2,157 1,991 2,088 718 509 1,227 Source: Authors’ calculation based on endline 4 Source: Authors’ calculation based on endline 4 Employment effects Although the UPSNP generate positive and significant impact on enterprise ownership, the impacts on employment effects are muted. Compared to the impacts on household enterprises and hiring of household members, the impacts of the livelihoods on employment (hours worked) are muted. The estimation results suggest that the livelihoods component of UPSNP does not affect total working hours. Moreover, there are no significant impacts on self-employment or wage employment hours. On the contrary, the estimation results indicate that the households’ total working hours and per capita working hours are less compared to non-UPSNP households. It is found that UPSNP households work 4.6 hours per week less than non-UPSNP households, which translates to less work of 42 minutes for every household member. Figure 22. Employment (hours worked) effects of Figure 23. Impacts of livelihoods by year of livelihoods intervention Source: Authors’ calculation based on endline 4 Source: Authors’ calculation based on endline 4 Overall, year 1 households are doing slightly worse compared to year 2 households. Comparing year 1 and year 2 beneficiaries, the results from the impact evaluation show that year 1 households have worse employment outcomes compared to year 2 households. Consumption and assets There was no significant impact on consumption and assets. The DID estimation results show that the livelihoods component of the UPSNP does not generate significant impacts on household consumption or asset wealth. The main objective of UPSNP is to increase consumption and thus 24 reduce poverty. The impact estimates on consumption show that this objective was not achieved. In fact, we observe slightly negative consumption effects for year 1 households, meaning that consumption for UPSNP households reduced (rather than increased) compared to non-UPSNP households. This could be resulting from the fact that PW payments stopped (due to the exit of the project) for year 1 households at the time COVID-19 hit. As households were not able to successfully set-up or expand businesses, they had to compensate their income shock with reducing consumption while non-UPSNP households were able to sustain their consumption levels. We see that negative consumption effects disappear in the second year of the program (for both year 1 and year 2 households). This shows that, even though UPSNP did not lead to increased consumption compared to non-UPSNP households, UPSNP participation did not make beneficiaries worse off. For asset wealth index, the DID estimation results show negative coefficients but the impacts were not significant. 7. Conclusion Self-employment is the preferred livelihood modality for UPSNP households, however there is low success rates for either modality. Around 97 of livelihood households selected the self-employment modality. However, of all households in self-employment, the proportion of households that opened or expanded a business with the business grant was only 44 percent and the proportion that were operating the business 12 months later was only 24 percent. This reflects the low success rates in self- employment pathway. The main challenges for opening and operating a business enterprise include securing inputs or finance, high competition, and lack of account management skills. Thus, alleviating business opening, and operation constraints could improve impacts. In this improving business plan development skills (e.g., estimating costs, output, sales) and training on financial management could be given emphasis. Nearly half of the households used the grants to cover household expenditures (due to COVID-19 and other reasons). Our results show even lower participation in and success rates for the wage-employment modality. While only 3 percent of all UPSNP beneficiaries chose the wage employment pathway, of those who did, only 8 percent started a new job with roughly two-thirds still in that employment 12 months later. The wage rate for PW was carefully considered during the design stage and adjusted throughout the project life cycle but likely not sufficiently to compensate for the large increases in the cost of living in Ethiopia. During the design of the UPSNP, the daily wage rate for PW was set at ETB 60, below the market wage rate at the time based on a study of urban labor markets. Given high levels of inflation in the country, the wage rate was adjusted based on increases in food prices as reflected in the official consumer price index (CPI). The aim of these adjustments was to ensure that purchasing power of the transfer does not erode over time (rather than increasing the real value of the transfer). The UPSNP became effective in July 2016 and started disbursing transfers in April 2017. A first adjustment to the transfer value was made in June 2018, when the PW transfer was revised to ETB 75 per day. In July 2020, the wage rate was further adjusted to ETB 90 per day (Wieser et al., 2021). Despite these adjustments in wage rates, the inflationary pressure on households was large. For example, year 1 households participated in PW roughly between April 2017 and March 2020 and received only one wage rate adjustment in June 2018. Yet, between June 2018 and March 2020, food prices increased by 36 percent. As a result, households’ purchasing power eroded over time which may not have allowed them to smoothen consumption which could explain why around 50 percent of households 25 used their livelihood grant for regular household expenses and not for the intended purpose to establish or expand a business. While the livelihoods program led to increase in households’ ownership of enterprises, effect on earnings, employment, and profits is low. Though we see an increase of household enterprise resulting from the program, there are no effects on earnings or employment and there was no discernable effect on total household income. Few households hire non-household members on their enterprise. Moreover, enterprises started by the livelihood grant are less profitable than those of non- UPSNP beneficiaries. We also do not observe an overall change in total working hours. Overall, year 1 beneficiaries are doing worse compared to year 2 beneficiaries related to the challenging context that households had to operate in during COVID-19. Regardless of the year of intervention, there is no impact on consumption or assets. The low impact of the livelihoods grants on business ownership and earnings could be partly due to beneficiaries selecting low paying but labor-intensive businesses. Additional support could be provided in these areas of improving logistic systems within the market along wholesalers, guilet merchants, and guilet service providers to improve efficiency. The lack of impacts on welfare and asset wealth could be because the grants are used for other purposes than investing in remunerative activities or securing employment. Moreover, the impact of the COVID-19 pandemic during the first year of the livelihood intervention could explain the lack of significant impacts. 26 References Abebe, G., Franklin, S., & CarolinaMejia-Mantilla. (2018). Public works and cash transfers in urban Ethiopia: Evaluating the Urban Productive Safety Net Program. Retrieved from https://documents1.worldbank.org/curated/en/666211557829787683/ Endale, K., Pick, A., & Woldehanna, T. (2019). Financing Social Protection in Ethiopia: a Long-Term Perspective. In OECD Development Policy Papers (No. 15). https://doi.org/https://doi.org/10.1787/24140929 Franklin, S., Tefera, G., & Getahun, T. (2017). Monitoring and Evaluating Ethiopia’s Urban Productive Safety Net Project (UPSNP). In World Bank, Washington, DC. License: Creative Commons Attribution CC BY 3.0 IGO. Retrieved from www.worldbank.org. Gilligan, D. O., Hoddinott, J., & Taffesse, A. S. (2009). The Impact of Ethiopia’s productive safety net programme and its linkages. Journal of Development Studies, 45(10), 1684–1706. https://doi.org/10.1080/00220380902935907 Hill, R., Inchauste, G., Lustig, N., & Tsehaye, E. (2015). Fiscal Incidence Analysis for Ethiopia. Hirvonen, K., Mascagni, G., & Roelen, K. (2018). Linking taxation and social protection: Evidence on redistribution and poverty reduction in Ethiopia. International Social Security Review, 71(1), 3– 24. https://doi.org/10.1111/issr.12159 Ministry of Labour and Social Affairs. (2012). National Social Protection Policy of Ethiopia. Addis Ababa. Wieser, C., Franklin, S., & Hari, S. (2021). Effectiveness of the Urban Productive Safety Net Project. In World Bank Group. World Bank. (2015). Ethiopia Poverty Assessment. https://doi.org/10.1201/9781482293500-22 World Bank. (2020). Ethiopia Poverty Assessment: Harnessing Continued Growth for Accelerated Poverty Reduction. Washington DC. 27 Annex Table A.1. Estimated impacts of livelihood services: DID results Outcome Estimator Method Coeff. SE CM N Household enterprise ownership Endline 3 - Year 1 PW D-D 0.018 0.013 0.035 5570 Household enterprise ownership Endline 4 - Year 1 PW D-D 0.111 0.021 0.187 5170 Household enterprise ownership Endline 4 - Year 2 PW D-D 0.145 0.020 0.187 5656 Hired household members (total) Endline 3 - Year 1 PW D-D 0.013 0.013 0.035 2785 Hired household members (total) Endline 4 - Year 1 PW D-D 0.037 0.020 0.121 2585 Hired household members (total) Endline 4 - Year 2 PW D-D 0.086 0.020 0.121 2828 Hired nonmembers (paid) Endline 3 - Year 1 PW D-D 0.011 0.005 0.003 2785 Hired nonmembers (paid) Endline 4 - Year 1 PW D-D -0.004 0.009 0.018 2585 Hired nonmembers (paid) Endline 4 - Year 2 PW D-D -0.007 0.008 0.018 2828 Average monthly profit Endline 3 - Year 1 PW D-D 64.696 132.726 155.672 2785 Average monthly profit Endline 4 - Year 1 PW D-D 14.132 71.680 483.932 2585 Average monthly profit Endline 4 - Year 2 PW D-D 140.518 71.618 483.932 2828 Total earnings Endline 3 - Year 1 PW D-D 199.944 218.795 2068.418 5570 Total earnings Endline 4 - Year 1 PW D-D 58.931 230.521 3419.666 5170 Total earnings Endline 4 - Year 2 PW D-D 365.146 198.720 3419.666 5656 Total earnings (per working age members) Endline 3 - Year 1 PW D-D -115.566 59.878 626.089 5550 Total earnings (per working age members) Endline 4 - Year 1 PW D-D -113.048 63.784 1015.954 5152 Total earnings (per working age members) Endline 4 - Year 2 PW D-D -7.655 54.483 1015.954 5637 Nonpublic work earnings (per working age member) Endline 3 - Year 1 PW D-D -114.752 59.860 623.617 5550 Nonpublic work earnings (per working age member) Endline 4 - Year 1 PW D-D -92.212 63.780 987.310 5152 Nonpublic work earnings (per working age member) Endline 4 - Year 2 PW D-D 11.332 54.443 987.310 5637 Self-employment earnings Endline 3 - Year 1 PW D-D 51.757 93.932 491.006 5570 Self-employment earnings Endline 4 - Year 1 PW D-D -26.774 105.391 641.762 5170 Self-employment earnings Endline 4 - Year 2 PW D-D 107.121 100.566 641.762 5656 Wage earnings Endline 3 - Year 1 PW D-D 151.041 203.197 1570.015 5570 Wage earnings Endline 4 - Year 1 PW D-D 154.956 217.023 2690.278 5170 Wage earnings Endline 4 - Year 2 PW D-D 322.800 181.863 2690.278 5656 Self-employed members Endline 3 - Year 1 PW D-D 0.058 0.038 0.200 5570 Self-employed members Endline 4 - Year 1 PW D-D 0.038 0.057 1.395 5170 Self-employed members Endline 4 - Year 2 PW D-D 0.101 0.051 1.395 5656 Wage employed members Endline 3 - Year 1 PW D-D 0.149 0.059 0.549 5570 Wage employed members Endline 4 - Year 1 PW D-D 0.170 0.065 1.187 5170 Wage employed members Endline 4 - Year 2 PW D-D 0.059 0.059 1.187 5656 Non-public works hours (per working age members) Endline 3 - Year 1 PW D-D 0.816 0.824 9.238 5550 Non-public works hours (per working age members) Endline 4 - Year 1 PW D-D 1.023 0.889 17.138 5152 Non-public works hours (per working age members) Endline 4 - Year 2 PW D-D 0.146 0.821 17.138 5637 Monthly expenditure per adult equivalent (real) Endline 3 - Year 1 PW D-D -68.273 25.919 704.394 5569 Monthly expenditure per adult equivalent (real) Endline 4 - Year 1 PW D-D 11.064 21.542 609.581 5169 28 Outcome Estimator Method Coeff. SE CM N Monthly expenditure per adult equivalent (real) Endline 4 - Year 2 PW D-D -4.562 20.310 609.581 5655 Asset index Endline 3 - Year 1 PW D-D -0.018 0.056 0.119 5567 Asset index Endline 4 - Year 1 PW D-D -0.040 0.058 0.156 5169 Asset index Endline 4 - Year 2 PW D-D -0.028 0.052 0.156 5655 Enterprise types: Shop Endline 4 - Year 1 PW D-D -0.012 0.007 0.027 2585 Shop Endline 4 - Year 2 PW D-D -0.002 0.007 0.027 2828 Sewing Endline 4 - Year 1 PW D-D -0.004 0.008 0.029 2585 Sewing Endline 4 - Year 2 PW D-D -0.001 0.007 0.029 2828 Baking Endline 4 - Year 1 PW D-D 0.016 0.005 0.010 2585 Baking Endline 4 - Year 2 PW D-D 0.028 0.006 0.010 2828 Gulit Endline 4 - Year 1 PW D-D 0.026 0.011 0.047 2585 Gulit Endline 4 - Year 2 PW D-D 0.052 0.011 0.047 2828 Street Endline 4 - Year 1 PW D-D 0.021 0.007 0.013 2585 Street Endline 4 - Year 2 PW D-D 0.011 0.006 0.013 2828 Brewing coffee or beer Endline 4 - Year 1 PW D-D 0.025 0.007 0.013 2585 Brewing coffee or beer Endline 4 - Year 2 PW D-D 0.017 0.006 0.013 2828 Source: Authors’ computation from endline 4 and using a DID approach. Note: CM is control group mean. Bold coefficients are significant (at less than 10%). Note: CM is control group mean. Bold coefficients are significant (at less than 10%). Earnings, profit, and expenditure are in Birr per month. Hours are per week. The asset index is constructed using the using inverse covariance weighting with all household assets covered in the survey. 29 Table A.2. Estimated impacts of livelihoods: year 1 vs. year 2 comparisons Outcome Estimator Method Coeff SE CM N Household enterprise ownership Endline 3 - Year 1 LHG Y1/2 -0.017 0.011 0.057 1793 Household enterprise ownership Endline 4 - Year 1 LHG Y1/2 -0.027 0.022 0.366 1793 Hired household members (total) Endline 3 - Year 1 LHG Y1/2 -0.004 0.016 0.041 1793 Hired household members (total) Endline 4 - Year 1 LHG Y1/2 -0.072 0.025 0.240 1793 Hired nonmembers (paid) Endline 3 - Year 1 LHG Y1/2 0.003 0.002 0.000 1793 Hired nonmembers (paid) Endline 4 - Year 1 LHG Y1/2 0.005 0.006 0.006 1793 Average monthly profit Endline 3 - Year 1 LHG Y1/2 18.366 147.461 138.547 1793 Average monthly profit Endline 4 - Year 1 LHG Y1/2 -83.289 70.171 666.306 1793 Total earnings Endline 3 - Year 1 LHG Y1/2 0.956 128.713 1702.000 1793 Total earnings Endline 4 - Year 1 LHG Y1/2 -95.706 138.982 2989.489 1793 Total earnings (per working age members) Endline 3 - Year 1 LHG Y1/2 -43.945 33.985 504.707 1788 Total earnings (per working age members) Endline 4 - Year 1 LHG Y1/2 -29.428 37.313 874.879 1788 Nonpublic work earnings (per working age member) Endline 3 - Year 1 LHG Y1/2 -21.000 33.567 475.159 1788 Nonpublic work earnings (per working age member) Endline 4 - Year 1 LHG Y1/2 -28.221 37.222 872.155 1788 Self-employment earnings Endline 3 - Year 1 LHG Y1/2 -46.352 59.259 474.034 1793 Self-employment earnings Endline 4 - Year 1 LHG Y1/2 -114.737 64.324 717.446 1793 Wage earnings Endline 3 - Year 1 LHG Y1/2 124.570 109.710 1140.508 1793 Wage earnings Endline 4 - Year 1 LHG Y1/2 22.241 131.746 2263.213 1793 Self-employed members Endline 3 - Year 1 LHG Y1/2 -0.067 0.026 0.303 1793 Self-employed members Endline 4 - Year 1 LHG Y1/2 -0.053 0.047 1.502 1793 Wage employed members Endline 3 - Year 1 LHG Y1/2 0.002 0.038 0.523 1793 Wage employed members Endline 4 - Year 1 LHG Y1/2 0.007 0.045 1.129 1793 Non-public works hours (per working age members) Endline 3 - Year 1 LHG Y1/2 -0.363 0.583 9.004 1788 Non-public works hours (per working age members) Endline 4 - Year 1 LHG Y1/2 -1.367 0.660 17.818 1788 Monthly expenditure per adult equivalent (real) Endline 3 - Year 1 LHG Y1/2 -96.182 19.479 610.293 1793 Monthly expenditure per adult equivalent (real) Endline 4 - Year 1 LHG Y1/2 -32.094 14.214 520.500 1793 Asset index Endline 3 - Year 1 LHG Y1/2 -0.109 0.046 0.016 1793 Asset index Endline 4 - Year 1 LHG Y1/2 -0.031 0.044 -0.037 1793 Enterprise types: Shop Endline 4 - Year 1 LHG Y1/2 -0.008 0.007 0.031 1793 Sewing Endline 4 - Year 1 LHG Y1/2 -0.003 0.008 0.026 1793 Baking Endline 4 - Year 1 LHG Y1/2 -0.018 0.009 0.045 1793 Gulit Endline 4 - Year 1 LHG Y1/2 -0.032 0.015 0.126 1793 Street Endline 4 - Year 1 LHG Y1/2 0.014 0.009 0.025 1793 Brewing coffee or beer Endline 4 - Year 1 LHG Y1/2 0.010 0.010 0.038 1793 Source: Authors’ computation from endline 3 and 4 and using a post-intervention comparison. Note: CM is control group mean. Bold coefficients are significant (at less than 10%). Earnings, profit, and expenditure are in Birr per month. Hours are per week. The asset index is constructed using the using inverse covariance weighting with all household assets covered in the survey. 30